These are speeds we measured ourselves, through Elzi's engine on our own Macs. Every row names the model, the Mac and the date, because the same model can run twice as fast on another Mac or with another setting. Decode is how fast the answer is written. Prefill is how fast the prompt is read. Time to first token is how long you wait before the answer starts, and it grows with the length of the prompt.
One Mac: a 27B model
| Task | Plain | With MTP | MTP + prompt lookup |
|---|---|---|---|
| Code edit | 21.4 | 42.8 | 134.8 |
| Document summary | 22.1 | 33.3 | 32.4 |
| Free chat | 22.0 | 37.3 | 37.1 |
| Prompt | First token | Prefill | Decode |
|---|---|---|---|
| 217 tokens | 1.19 s | 182 tok/s | 22.1 tok/s |
| 31,996 tokens | 184.9 s | 173 tok/s | 16.0 tok/s |
In an agent loop, where each turn sends the same long prompt plus a little more, the prompt cache matters more than raw speed. On that MacBook, time to first token per turn fell from about 25-29 seconds to 17.5-18 seconds when the engine started caching the changing tail of the prompt (engine lz.2, Elzi 3.0.16).
One Mac: Qwen3.8-Flash-Next at very long context
| Prompt | First token | Prefill | Decode |
|---|---|---|---|
| 455,700 tokens, cold | 691.5 s | 659 tok/s | about 27 tok/s |
| 521,727 tokens, cold | 656.9 s | 794 tok/s | 28.4 tok/s |
| 653,246 tokens (131,519 new on a cached prefix) | 317.1 s | 415 tok/s | 26.5 tok/s |
Across that session decode stayed between 25 and 31 tokens per second from 455k to 653k tokens of context, and swap stayed flat. In the same session, sampling at temperature 1.0 with prompt lookup on a code rewrite, all 591 drafted tokens were accepted and decode reached 96.7 tokens per second.
Over LeanZero Link and across two Macs
Qwen3.8-27B (our Atlassian fine-tune) running on the M3 Ultra Mac Studio answered a chat from an M4 Max MacBook Pro over LeanZero Link at 29.9 tokens per second, reading the prompt at 314 tokens per second (24 September 2026).
Splitting one model across two Macs is for fitting a model that is too big for one, not for speed; the distributed serving page has the measurements.
What makes it faster
- Prompt lookup and MTP, for text the model copies or rewrites.
- The prompt cache, for agent loops and new chats that share a system prompt.
- The memory lock, when other apps compete for memory during long chats.
Facts on this page were checked against our own test records on 6 November 2026.