Elzi runs local models with the LeanZero MLX engine, our fork of the open-source Rapid-MLX server for Apple silicon. You don't install or start it yourself. Each Elzi release pins the engine version it was tested with, builds the launch command from the model you pick, starts the engine on your Mac and restarts it if it stops. The app talks to it over an OpenAI-compatible API on your Mac's loopback address.
Why a fork of Rapid-MLX
Rapid-MLX began as vLLM-MLX, an Apache 2.0 project, and took its current name in March 2026. We chose it as Elzi's engine on 31 August 2026 after a scored comparison with oMLX: it stayed stable with eight requests running at once, its prefix cache cut the wait for the first token when a prompt repeated, and it used less memory.
We keep our own fork because we fix things there first. A few examples: a request that asked for log probabilities could stop an engine that was running a model's MTP head (fixed in lz.4), a hidden 32,768-token cap on answers was removed so an answer can run to the end of the context (lz.5), and hybrid models gained an 8-bit and 4-bit KV cache (lz.3).
What the app does for you
- It pins one engine version per app release. Elzi 4.1.3 ships engine 0.14.3+lz.32, downloaded as a single package from LeanZero's download server.
- It starts the engine with uv. You need uv installed once (for example with brew install uv); the app sets up the engine's Python environment itself.
- It writes the launch command from the model's profile: prefix cache on, how many requests run at once, the model's sampling defaults, the right tool-call parser, MTP when the model has an MTP head, and the adapter if you use one.
- It watches the engine with health checks and restarts it, with a breaker so a broken model doesn't restart forever.
If you typed your own launch command, the app never replaces it. A saved command that matches an old default moves to the new pinned version on update.
What it serves
- Chat with tool calls, through an OpenAI-compatible API on 127.0.0.1.
- Images, for vision models. Qwen3.8-Flash-Next reads images on the fast text lane; other vision models start text-only by default to keep batching and the prompt cache, and you can switch that per model.
- Speculative decoding: the model's own MTP head when it has one, and n-gram prompt lookup on top.
- A prompt cache that is always on. A new chat with the same system prompt and tools picks up where the cached part ends, and an optional disk tier keeps older entries on disk instead of dropping them.
- Thinking, set per model and off unless you turn it on.
Models we have run on it
| Model | Mac | Date | What we checked |
|---|---|---|---|
| Qwen3.8-27B, our Atlassian fine-tune, 8-bit | Mac Studio (M3 Ultra, 96 GB) | 26 Oct 2026 | MTP and n-gram prompt lookup speed |
| Qwen3.8-Flash-Next, our own 4-bit build | Mac Studio (M5 Max, 128 GB) | 2 Nov 2026 | Images, MTP, prompt lookup, a 653k-token needle test |
| Qwen3.5-9B, 4-bit | Mac Studio (M5 Max, 128 GB) | 6 Nov 2026 | The memory lock in the released Elzi 4.1.3 |
One Mac or two
The engine has two release lines. The single-Mac line serves a model from one Mac's memory (single-Mac serving). A separate pipeline line splits a model that is too big for one Mac across two Macs joined by Thunderbolt (distributed serving).
Facts on this page were checked against our own test records on 6 November 2026.