On one Mac, Elzi loads the model into the LeanZero MLX engine and serves it from that Mac's own memory. Before it mounts a model, the app measures the memory macOS can give right now. A mount that won't fit is refused with the reason and, where there is one, an alternative such as a two-Mac split. During long chats a memory guard plans every step, so a request that grows too big gets a clear answer instead of pushing the Mac out of memory.
Mounting a model
The app measures available memory the way Activity Monitor does: free memory plus the file cache and purgeable pages macOS can reclaim. If that check fails, the mount doesn't start. While the model loads, you see each phase (making room, starting, loading, warming up) and how many bytes of weights are in. Stop works during loading too.
The memory guard
The engine plans memory before every prefill chunk, every decode step and every time a request joins a batch. It counts the weights, the KV cache, the allocator cache and the scratch space attention needs. When a step won't fit, it clears the allocator cache, evicts prompt-cache entries, shrinks the chunk or holds the newest request back. It only refuses a request that can't fit on its own, and then it says so with a context-length error.
We built it after a 64 GB MacBook Pro ran out of memory in the middle of a single growing session in October 2026. The guard also yields when macOS reports memory pressure.
Settings you can tune
The Memory tab of the LeanZero MLX settings holds the levers: the Metal memory limit, a free-memory floor (5 % of RAM by default), the prompt-cache size, the prefill step size, the maximum context, the guard's margin and how many requests run at once. Each one maps to an engine flag. The same tab has a button that raises or restores macOS's limit on GPU memory.
How much context fits
| Prompt | First token | Decode | Peak memory |
|---|---|---|---|
| 455,700 tokens, cold | 691.5 s | about 27 tok/s | 105.8 GiB |
| 521,727 tokens, cold | 656.9 s | 28.4 tok/s | 111.5 GiB |
| 653,246 tokens, 131,519 new on a cached prefix | 317.1 s | 26.5 tok/s | 114.2 GiB |
Swap stayed flat through all three, and the model found the needle each time.
How we count context
These are single test runs, not a promise for every prompt. Before we rely on a size in daily work, we also check that a real session growing to it works, because a cold test can pass at a size a real session can't reach.
Before a model goes into use
- Right build: every file's checksum, the tokenizer and the chat template match the lab's official release.
- Every capability works end to end: images answered correctly, and the MTP and prompt-lookup counters actually moving.
- The largest context is measured in a growing session, as above.
- Speed levers are proven not to change the output, or they stay off.
- Memory settings are set for each Mac it runs on.
Facts on this page were checked against our own test records on 6 November 2026.