Topic hub

Local AI

Local models on Apple Silicon, benchmarked rather than quoted.

§01

What this covers

Running a capable model on your own hardware is mostly an argument about memory — how much you actually have, given that 96GB of marketing is 77.76 GiB you can spend, and what you are willing to give up to fit inside it. The benchmarks here are run, not quoted: MLX against llama.cpp on one machine and one set of weights, MXFP8 against Q8 on the same model, speculative decoding measured instead of assumed. Every number arrives with the command that produced it and the hardware it ran on, because a tokens-per-second figure missing either one is not a measurement. Results that contradicted what I expected are left in. Those tend to be the useful ones.

§02

Series

Read in order

Local Model Benchmarks

5 parts

Measured runs on real hardware: runtimes, quantisation formats and the memory ceiling nobody quotes.

§03

Everything on this topic

Articles

ArticleAI Codinggoose Local Edition

goose swarm: pytest | head -80 exits 0 when nothing ran, and pipefail only trades the lie

A worker in my local-model swarm ran pytest against a file that did not exist, twice, and finished the task green. The pipe had eaten the exit code. Measured on bash and zsh, on pytest and cargo — including why turning pipefail on just moves the lie to the other side.

Aug 18, 202614 min read

ArticleAI CodingLocal AI

What 145 MCP Tools Cost Before the Model Reads Your Question

MCP went stateless on 2026-07-28. I probed ten real servers: none implement it. Then I measured the thing that actually costs you — 145 tool definitions, 40,784 tokens, and the 30% that is one server repeating itself.

Aug 17, 202614 min read

ArticleAI CodingLocal AI

96 GB is 77.76 GiB: the real memory ceiling on an M3 Ultra

Metal will not give you the RAM on the box, and the number it does give is not the 75% everyone repeats. I measured the three ceilings on a 96 GB Mac Studio, then measured what modern hybrid-attention models actually spend against them — including a Gemma 4 cache that quietly holds three times its own sliding window.

Aug 13, 202613 min read

ArticleAI Codinggoose Local Edition

goose compaction: my 27B re-read the file 95% of the time — quoting it in the summary didn't help

Our swarm workers kept re-reading files they had already read. Context compaction was summarising the tool output that held the file. Pasting the file into the summary does not fix it; returning the last turns verbatim does. Measured three ways on the same 27B.

Aug 11, 202612 min read

ArticleAI CodingLocal AI

llama.cpp vs MLX on Qwen3.6-27B: MTP is 1.04x here, not 1.85x

Multi-token prediction is merged in llama.cpp and still an open PR in mlx-lm. I measured both on an M3 Ultra with the same model. Every default MTP setting was slower than no MTP at all, and the runtime that deletes the MTP head outright is still the fastest thing on the box.

Aug 10, 202615 min read

ArticleAI CodingLocal AI

Ling 3.0 Flash won't load on a Mac Studio. Qwen3-Coder-Next does, at 73 tok/s.

Ant Group's 124B/5.1B-active hybrid-linear MoE hit Hugging Face on 2 August. The memory arithmetic fits a 96 GB Mac with room to spare, and mlx-lm still refuses it. I counted exactly which tensors block it — 385 of 62,237 — then measured what that sparsity actually buys on the models that do run.

Aug 5, 202618 min read

ArticleAI CodingLocal AI

MXFP8 vs Q8: 10x the weight error, 1% the perplexity

I quantised real Qwen3-Coder weights both ways on an M3 Ultra. MXFP8 reconstructs them about 10x worse than 8-bit affine at identical size — and then costs only 1% perplexity end to end. Both numbers are true, and the gap between them is the interesting part.

Aug 1, 202614 min read

ArticleAI Codinggoose Local Edition

Making the goose swarm predictable: 602 commits, 100 levers, and three bugs I found writing this

Three weeks after the swarm shipped its first honest builds, the work stopped being about making small local models smarter and became about making them stop lying to me. Here is what 602 commits bought, what the desktop looks like now, and the three defects I found in my own honesty machinery while writing this post.

Jul 20, 202614 min read

TutorialAI Codinggoose Local Edition

swarm-gym for goose: grading local models by running the code they write

A hands-on walkthrough of swarm-gym — the harness that drives goose local-edition's swarm through real coding tasks and grades the result by running it. Set it up, run both modes, read every output, and tune the swarm from what you find.

Jul 3, 20265 min read

ArticleAI Codinggoose Local Edition

MLX vs GGUF on Apple Silicon: Benchmarking the Same Local Model Two Ways

We built a harness that makes local coding agents produce real software, grades it by running it, and ran the same model as GGUF and MLX. Here's the harness, its modes and archetypes — and which build wins.

Jul 2, 202612 min read

TutorialAI Codinggoose Local Edition

Inside goose-swarm: How We Turned One Local Model Into a Self-Verifying Fleet

The engineering teardown of goose local-edition: how the scheduler, the parallel planner, the CONTRACTS discipline, the model-free judge, and the post-run smoke/AST gates are actually implemented, why each exists — and the self-driving test harness that found the failure behind every one of them.

Jul 1, 20265 min read

ArticleAI Codinggoose Local Edition

The goose swarm, self-verifying: small local models shipping real software

We forked Block's goose into a multi-agent swarm that decomposes a hard app spec into a task DAG, runs it across three small local models on LM Studio, and makes them verify their own work by actually running it. Over six days and 300-plus commits, the last real limit stopped being the swarm's coordination and became the small model's raw coding ability — and even that ceiling moved.

Jul 1, 20265 min read

ArticleAI CodingLocal AI

Qwen3-Coder-Next + MXFP8: The 128GB Local LLM That Runs Predictably

A deep dive into Qwen3-Coder-Next, the 80B MoE model that brings powerful local development within reach. Learn why this 2026 release matters for developers who value control, predictability, and consistent results.

Feb 4, 202610 min read