When a model rewrites text that is already in its context, such as a file it is editing, the engine can guess the next tokens by finding the matching run of words (an n-gram) earlier in the prompt and copying what followed. The model then checks the whole guess in one step and keeps only what it agrees with. The answer is still the model's own: in our code-edit tests the output was byte-identical with the lookup on and off. It helps most on code edits and does little for free chat.
Measured gains
| Task | MTP only | MTP + prompt lookup | Change |
|---|---|---|---|
| Code edit | 42.8 | 134.8 | 3.15x |
| Document summary | 33.3 | 32.4 | none |
| Free chat | 37.3 | 37.1 | none |
On the code edit the model accepted 1,926 of 1,940 drafted tokens (99.3 %), and all 12 outputs were byte-identical across the settings. In our long-context tests the gain shrank as the context grew, because checking a draft costs more over a large KV cache, while almost every draft was still accepted.
Agents usually sample instead of picking the single most likely token. For that case the engine uses exact speculative sampling, and a statistical test found no difference from normal sampling while catching a deliberately wrong version. On our own 4-bit Qwen3.8-Flash-Next build at temperature 1.0 (M5 Max Mac Studio, 2 November 2026), all 591 drafted tokens on a code rewrite were accepted and decode reached 96.7 tokens per second.
Where it doesn't help
- Quoting or summarising a document: drafts fire, but on the 27B model they bought nothing, as the table shows.
- Free chat: there is nothing to copy, so there are almost no drafts, and no gain or loss.
How to turn it on
Each model's profile in the MLX engine settings has a "Prompt lookup (n-gram)" control: Auto, On or Off, plus the shortest n-gram, the longest n-gram and the longest draft. Auto keeps the engine's default for that model family. For Qwen 3.5, 3.6 and 3.8 that default is on, matching runs of 8 to 64 tokens and drafting up to 31, so Auto already gets the gain. Changing it restarts the engine.
Prompt lookup works on one Mac only. It is not available on a two-Mac split or on a model served by another Mac over LeanZero Link.
How it relates to MTP
MTP drafts tokens from a small prediction head that some models ship with. Elzi turns it on when the model folder contains that head. On those models, prompt lookup runs inside MTP and the two add up: on the 27B code edit, MTP alone doubled the speed (21.4 to 42.8 tokens per second) and prompt lookup tripled it again. Non-hybrid models without an MTP head can use a separate suffix-decoding lane (hybrid models without one get neither), but Auto leaves that lane off because on Mellum2.1-12B (26 October 2026) it was slower and changed a chat answer.
Facts on this page were checked against our own test records on 6 November 2026.