Ling 3.0 Flash won't load on a Mac Studio. Qwen3-Coder-Next does, at 73 tok/s.
Mihai Perdum
Author
18 min readAugust 5, 2026
Key takeaways
Ling 3.0 Flash is 127,486,405,600 parameters with 5.10B active per token. I counted both from the shipped safetensors headers and they reproduce the model card exactly.
97.1% of the weights are routed experts, so MXFP4 on the experts alone brings it to roughly 73 GB — which fits 96 GB of unified memory.
mlx-lm 0.31.3 still answers `ValueError: Model type bailing_hybrid not supported.` The open PR for that exact model_type was written for Ling 2.6 and mismatches 385 tensors: 0.6% of the file, and 11 of the 13 attention tensors in each of 35 layers.
On the models that do run: Qwen3-Coder-Next (80B total, 3B active) decoded at 73.39 tok/s against 23.37 for dense Qwen3.6-27B and 19.67 for Gemma-4-31B, on the same machine and the same prompts.
My grading harness had two bugs that faked six model failures out of fifteen gradings. When several models fail one case identically, suspect the harness.
Ant Group announced Ling-3.0-Flash from Hangzhou on 27 July 2026, promising the weights would follow the free-API period that ended 3 August. They landed a day early. The commit that matters is Add files using upload-large-folder tool at 2026-08-02 16:18:49 UTC — 24 safetensors shards, 255.0 GB, MIT licence. Everything since has been README edits. When I pulled the index three days later the repo had 111 likes and 25 downloads (read 5 August), which is about right for a quarter of a terabyte of bf16.
I have a Mac Studio with 96 GB of unified memory. The interesting thing about a 124B model that activates only 5.1B parameters per token is that it is precisely the shape this machine should like: enormous at rest, cheap per token. So I went to run it.
It does not run. That turned out to be the more interesting result, and it is far more specific than "MLX doesn't support it yet". I can tell you which 385 tensors out of 62,237 stand in the way, and why the remaining work is smaller than anyone staring at a 255 GB download would guess.
Then, because a model you cannot load teaches you nothing about speed, I put the models that do run on this machine head to head on the same prompts — one sparse 80B against two dense ~30Bs — to measure what the sparsity Ling is selling actually buys.
Reading 255 GB without downloading 255 GB
Every claim below about Ling 3.0 Flash's insides comes from the safetensors headers rather than the model card. You do not need the weights for this. A safetensors file opens with an 8-byte little-endian header length followed by that many bytes of JSON describing every tensor's dtype, shape and byte offsets. An HTTP range request fetches you just that part.
All 24 shard headers together came to 7.96 MB — the complete tensor manifest of a 255 GB model, every name and shape and dtype, at about 32,000:1 against the weights. For the shape spot-checks further down I only needed two shards, which is 0.53 MB.
The first thing worth doing with it is checking the vendor's arithmetic.
text
1TOTAL incl MTP : 127,486,405,600 (127.49 B)
2MTP layer : 3,072,194,048 ( 3.07 B)
3TOTAL excl MTP : 124,414,211,552 (124.41 B) <- card says 124B
4ACTIVE per token: 5,103,302,112 ( 5.10 B) <- card says 5.1B activated
Both land. The card's "124B" excludes the multi-token-prediction layer, which the config declares as num_nextn_predict_layers: 1 and which the index carries as layer 42 on top of num_hidden_layers: 42. The "5.1B activated" figure comes out at 5.10B if you count 8 of the 512 routed experts per token, skip the MTP layer, and treat the embedding matrix as a lookup rather than a matmul. When a vendor's headline numbers reproduce from first principles, the rest of the card earns more benefit of the doubt — and, more usefully, it tells me my counting method is sound before I use it for anything they did not publish.
The distribution is what decides everything downstream:
Component
Parameters
Share
Routed experts
123,815,854,080
97.1%
Attention, all 43 layers
2,462,471,648
1.9%
of which KDA-specific
405,292,512
0.32%
Embeddings, router, norms, dense MLPs
about 1.2 B
about 1.0%
5 rows × 3 columnsHeader row enabled
Ninety-seven percent of this model is expert FFN weight. That is what makes the memory arithmetic work, because expert weights are the easiest thing in a transformer to quantise hard.
The arithmetic fits
At bf16 the model is 255.0 GB and hopeless. Quantise the experts and it changes character entirely:
Scheme
Size
Fits 96 GB?
bf16 everywhere
255.0 GB
no
MXFP4 experts, bf16 everything else
73.1 GB
yes
4-bit affine group-64 everywhere
71.7 GB
yes
MXFP4 experts, 8-bit everything else
69.7 GB
yes
5 rows × 3 columnsHeader row enabled
MXFP4 is 4-bit E2M1 elements with one E8M0 scale per group of 32, so 4.25 bits per weight. Because the experts are 97.1% of the parameters, the expert format sets the file size almost single-handedly and everything else can stay in bf16 essentially for free.
Somebody has already done it. olka-fi/Ling-3.0-flash-MXFP4 appeared on 4 August at 20:24 UTC and publishes 77.6 GB, a few gigabytes above my 73.1 GB estimate — I would attribute the gap to packing overhead and to whatever else they kept at higher precision beyond what I modelled. Either figure fits in 96 GB with room for a working context.
So memory is not the blocker.
Warning
That repo caught me for a minute and it may catch you. Hugging Face's search surfaced it to me as an MLX conversion. It is not. The card says library_name: transformers, the tags say compressed-tensors and vllm, and the measurement quoted on it is KV cache on a B200. It is a vLLM build, and MLX cannot read compressed-tensors. Read the card, not the search result.
The actual blocker
Here is mlx-lm's answer, from the config file alone:
python
1import json
2from mlx_lm.utils import _get_classes
34cfg = json.load(open("config.json"))5print(cfg["model_type"])# bailing_hybrid6_get_classes(cfg)7# ValueError: Model type bailing_hybrid not supported.
mlx-lm resolves architectures by importing mlx_lm.models.<model_type> directly, through a 9-entry remapping table for aliases. Version 0.31.3 ships 107 architecture modules. bailing_hybrid is not one of them and is not in the remap table, so the import fails and you get that ValueError. Thirty seconds, no download.
Most write-ups would stop there. But bailing_hybrid is not an unknown string to the mlx-lm project — PR #1227 adds exactly that model_type, open since 29 April 2026 and last touched 6 July. It was written for Ling-2.6-flash, which the PR describes as "a 104B / 7.4B-active hybrid that mixes MLA with Lightning-style linear attention (1:7 ratio, layers 7/15/23/31 are MLA) and a sigmoid noaux_tc MoE (256 experts, 1 shared, group-limited top-8)".
Ling 3.0 Flash keeps the model_type string and changes the architecture underneath it. So the real question is not "is it supported" but "how much of the existing port survives".
Measuring the gap instead of guessing at it
I installed the PR's bailing_hybrid.py into mlx-lm, built the model at miniature scale from Ling-3.0-flash's own config — same layer pattern, tiny hidden size so it fits in memory — enumerated the parameter names it expects, and diffed them against the 63,783 real tensor names in the shipped index.
My first pass overstated the damage, and why is worth saying. The PR carries a sanitize() that stacks the 512 per-expert tensors into MLX's switch layers, drops the trailing MTP layer, renames the router gate and splits the MLA kv_b_proj. Counting any of those as mismatches would have been my error, not the PR's. So I replicated sanitize() as a pure name transform and diffed what was left:
text
1Ling-3.0-flash: 63783 shipped tensors
2 after the PR's own sanitize(): 62237 (1553 MTP-layer tensors dropped)
34RESIDUAL MISMATCH — shipped tensors with nowhere to go (11 templates):
5 35 x model.layers.N.attention.q_conv1d.weight
6 35 x model.layers.N.attention.k_conv1d.weight
7 35 x model.layers.N.attention.v_conv1d.weight
8 35 x model.layers.N.attention.A_log
9 35 x model.layers.N.attention.dt_bias
10 35 x model.layers.N.attention.f_proj.weight
11 35 x model.layers.N.attention.b_proj.weight
12 35 x model.layers.N.attention.k_proj.weight
13 35 x model.layers.N.attention.v_proj.weight
14 35 x model.layers.N.attention.o_proj.weight
15 35 x model.layers.N.attention.o_norm.weight
1617PR parameters Ling-3.0-flash never ships (4):
18 model.layers.N.attention.query_key_value.weight
19 model.layers.N.attention.g_norm.weight
20 model.layers.N.attention.query_layernorm.weight
21 model.layers.N.attention.key_layernorm.weight
2223Unplaceable after sanitize: 385 of 62237 tensors (0.6%)
24Matched: 61852 of 62237 (99.4%)
99.4% of the file lands where the existing port already expects it. The MoE — 61,440 tensors carrying 120.8 billion parameters — is a solved problem. What does not land is 385 tensors carrying 0.405B parameters, 0.32% of the model.
Those 385 tensors are 11 of the 13 attention tensors in each of 35 layers. Only q_proj and g_proj survive, and they match by template coincidence against a different layer type's parameters. Not one of those 35 layers could run.
The mismatch list reads like a diagnosis once you know the two families. The PR's LinearAttention is gated linear attention, Lightning-style: it wants one fused query_key_value projection and a g_norm. Ling 3.0 ships split q_proj, k_proj and v_proj, three depthwise conv1d kernels, an A_log and a dt_bias. Those last two are the decay and step-size parameters of a delta rule. It is not the same operator. Ant swapped Lightning attention for Kimi Delta Attention between 2.6 and 3.0 and kept the config string.
The pieces are already in the box
Kimi Delta Attention comes from Moonshot's Kimi Linear: An Expressive, Efficient Attention Architecture (arXiv:2510.26692), which describes KDA as Gated DeltaNet with finer-grained gating and pairs it with full MLA at 3:1. Ling 3.0 Flash runs the same idea at 5:1.
And mlx-lm already implements it. mlx_lm/models/kimi_linear.py contains a KimiDeltaAttention module backed by a hand-written Metal kernel in gated_delta.py. So I compared Ling's KDA tensors against what that module exposes:
Ling-3.0-flash ships
mlx-lm KimiDeltaAttention
verdict
q_proj, k_proj, v_proj
same names
exact match
b_proj, A_log, dt_bias
same names
exact match
o_norm, o_proj
same names
exact match
q_conv1d, k_conv1d, v_conv1d
q_conv.conv, k_conv.conv, v_conv.conv
rename only
f_proj, one matrix
f_a_proj plus f_b_proj
structural
g_proj, one matrix
g_a_proj plus g_b_proj
structural
7 rows × 3 columnsHeader row enabled
Eight of thirteen match by exact name. Three are pure renames. The two structural differences have their explanation sitting in the config: Ling sets no_kda_lora: true and use_kda_lora: false, so where mlx-lm factors the forget gate and the output gate into low-rank pairs, Ling uses single full-rank matrices.
The shapes confirm that rather than my inferring it. On KDA layer 0, with hidden_size 2560 and 32 heads of 128:
f_proj goes 2560 to 4096 in one hop, full rank, exactly as no_kda_lora promises. q_conv1d is [4096, 1, 4] — depthwise, kernel 4, matching short_conv_kernel_size: 4. A_log is one value per head, dt_bias one per channel.
The layer census from the same headers matches the card line by line. Layers carrying A_log are KDA; layers carrying kv_b_proj are MLA:
Five KDA then one MLA, seven times over, and layer 42 — the MTP layer — is MLA because the config says mtp_use_kda: false. The card's "35 KDA + 7 Gated MLA (5:1)" is exactly right, and the eighth MLA layer is the MTP head it does not count.
One more detail the shapes give away. On a KDA layer, g_proj is [4096, 2560], a full gate per channel. On MLA layer 5 the same name is [32, 2560] — one scalar per head. That is gated_attention_proj_granularity_type: "head_wise", and it means a port cannot treat g_proj uniformly across layer types.
So the distance between mlx-lm and a 124B model is: three renames, one boolean branch for the no-LoRA case, and per-layer-type gate handling, on top of a MoE stack that already works. Nobody has walked it yet.
Note
This is a static analysis of names and shapes, and it should be read as exactly that. Matching every tensor name proves a port would load. It does not prove the numerics are right — gate conventions, normalisation placement and the precise delta-rule recurrence can all differ subtly and still fit the same shapes. I did not download 73 GB to check, so I am not making a quality claim and nobody should read one into this.
llama.cpp is approaching from the other end: BailingMoE support landed for the earlier Ling models in PR #12634, and KDA is being added separately in the Kimi-K3 work. Two GGUF conversions of Ling 3.0 Flash appeared while I was writing this — prometheusAIR/Ling-3.0-flash-GGUF was created at 03:50 UTC on 5 August, about ninety minutes before I looked. I have not run either, and I would want somebody to verify the KDA path produces sane logits before trusting output from them.
What the sparsity would have bought
The shape Ling is selling — a very large expert bank with a very small active slice — is not exotic, and this machine already runs it. Qwen3-Coder-Next is 80B total with 3B activated, 512 experts with 10 firing per token plus one shared, in a 48-layer stack Qwen writes as 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE)). Architecturally that is a cousin of Ling 3.0 Flash: the same 512-expert bank, the same hybrid-linear idea, 3:1 where Ling runs 5:1.
So I ran it against the two dense models of roughly a third its size that live on the same disk. Same machine — Mac Studio, M3 Ultra, 96 GB — same prompts, greedy decoding at temperature 0 so a rerun reproduces, mlx 0.32.0 and mlx-lm 0.31.3 in a venv. That is not LM Studio's bundled MLX engine, which is a different build and can report different numbers. Everything was unloaded first; a background download alone is enough to halve these figures.
Qwen3-Coder-Next
Qwen3.6-27B
Gemma-4-31B
Parameters
80B total, 3B active
27B dense
30.7B dense
Quantisation
MLX 6-bit, group 64
MLX 8-bit, group 64
MLX 8-bit, group 64
Weights in memory
64.75 GB
28.58 GB
32.62 GB
Load time
17.6 s
5.0 s
6.2 s
Decode, short prompt
73.39 tok/s
23.37 tok/s
19.67 tok/s
Prefill, ~16k tokens
1476.34 tok/s
312.06 tok/s
241.43 tok/s
Peak memory at 16k
68.03 GB
33.67 GB
39.30 GB
8 rows × 4 columnsHeader row enabled
The sparse model decodes 3.14x faster than the dense 27B and 3.73x faster than the dense 31B, and prefills 4.73x and 6.11x faster respectively — while holding 2.27x more weight in memory than Qwen3.6.
That is the whole trade in one table. Decode on Apple Silicon is memory-bandwidth bound and at batch 1 you only pay for the weights you touch, so sparsity converts almost directly into tokens per second. The naive ratio flatters it, though, and it is worth being precise about why. Qwen3-Coder-Next touches roughly a ninth of the parameters per token that dense Qwen3.6 does, and it delivered 3.14x, not 9x. The difference is everything that is read regardless of sparsity — attention, embeddings, norms and the shared expert fire on every token — plus routing overhead and the fact that a gather-based expert matmul is a less efficient kernel than a plain dense one. Prefill is compute bound rather than bandwidth bound, and there the same structural advantage came out at 4.73x.
Both models are quantised, at different widths (6-bit against 8-bit), so I am deliberately not quoting a bytes-per-token roofline here. The measured ratios are the claim; the parameter counts are only the reason to expect one.
This is the argument for Ling 3.0 Flash in one table, which is why the 385 tensors are so annoying: 5.1B active out of 124B is a sparser ratio still.
The cost is that you must hold all of it. Paying 64.75 GB of weights to get the speed of a 3B model is a bargain on a 96 GB Mac and impossible on a 32 GB one. Note the peak column too — Qwen3-Coder-Next was already at 68.03 GB with a 16k context. Ling 3.0 Flash at 73 to 78 GB of weights would leave this machine very little room for context, which is a real constraint the "it fits!" arithmetic above quietly understates.
The coding tasks, and the one that lost
Throughput without correctness is a benchmark for nobody. Each model got three Python tasks — run-length encode/decode with a round-trip property, an integer-interval merge checked against a brute-force oracle, and a quoted-value config parser. I took the longest fenced code block and graded by running it. Thinking is off here for the two models that have it — the next section measures what turning it on costs.
Task
Qwen3-Coder-Next
Qwen3.6-27B, thinking off
Gemma-4-31B, thinking off
rle round-trip
PASS, 196 tok
PASS, 276 tok
FAIL, 184 tok
merge_intervals vs oracle
PASS, 180 tok
PASS, 175 tok
PASS, 197 tok
parse_kv quoted values
PASS, 340 tok
PASS, 416 tok
PASS, 392 tok
Total
3/3, 716 tok, 9.9 s
3/3, 867 tok, 37.6 s
2/3, 773 tok, 40.0 s
5 rows × 4 columnsHeader row enabled
Gemma-4-31B is the one that lost, and it lost for a reason worth reading. Its rle_encode is correct. Its decoder is not:
python
1defrle_decode(s):2# Find all pairs of (character, digits)3 pairs = re.findall(r'([a-zA-Z])(\d+)', s)4return"".join(char *int(count)for char, count in pairs)
The encoder is alphabet-agnostic and will happily emit a run of spaces as " 1". The decoder's character class is [a-zA-Z], so it silently drops every run whose character is not a letter. My property test draws from an alphabet that includes a space, and the round-trip broke on the first string containing one:
That is a genuine and very ordinary bug class: a regex character class narrower than the data, paired with a producer that has no such restriction, failing silently rather than raising. Neither Qwen model made that assumption.
Thinking mode cost one model 12.5x and rescued the other
Qwen3.6-27B and Gemma-4-31B both expose an enable_thinking flag in their chat template. Qwen3-Coder-Next does not — its card states it "supports only non-thinking mode and does not generate <think></think> blocks", which my template check confirms. Comparing a reasoning model against a non-reasoning one at a single setting is not a comparison, so I measured both.
tasks passed
tokens
wall clock
Qwen3.6-27B, thinking off
3/3
867
37.6 s
Qwen3.6-27B, thinking on
3/3
10,879
478.4 s
Gemma-4-31B, thinking off
2/3
773
40.0 s
Gemma-4-31B, thinking on
3/3
5,133
274.9 s
5 rows × 4 columnsHeader row enabled
Throughput barely moves — 23.37 against 23.15 tok/s for Qwen3.6 — because thinking changes how many tokens you generate, not how fast. What changes is the bill. Qwen3.6 spent 12.5x the tokens and 12.7x the wall clock to pass exactly the tasks it already passed. Gemma spent 6.6x and genuinely bought something: the rle decoder came out correct with thinking on.
That 12.5x is a floor rather than a measurement. Qwen3.6's thinking run on parse_kv hit my 4,096-token generation cap (finish_reason: length) and was still going. It had already emitted a complete, correct code block by then, so it passes — but its true token cost is unknown and higher than what I am reporting. Gemma's runs all stopped on their own.
So the honest reading is that thinking rescued the model that was getting it wrong and was pure overhead on the model that was getting it right. Which is not a result you can apply in advance, and is a decent argument for running the cheap pass first and escalating only on failure.
Qwen3-Coder-Next, with no thinking mode at all, completed all three tasks in 9.9 seconds — 48x faster than Qwen3.6 with thinking on, for the same 3/3.
The tests that were wrong, not the models
I have to report something that nearly went into this article as a finding about models. My grading harness had two bugs, and between them they faked six model failures out of fifteen gradings.
The first showed up as all three models failing parse_kv on the identical assertion. Three independently trained models failing one case the same way is a very loud signal, and what it was signalling was my bug. The test embedded its input as parse_kv('b="say \"x\""'). That literal sits inside a Python test file, so Python consumed the backslashes before parse_kv was ever called and handed it b="say "x"" — a string containing no escapes at all and no consistent parse. I was asking for behaviour the input could not exhibit.
I found it by writing the reference implementation myself and watching it fail the same assertion:
The fix is one character — make the literal raw, parse_kv(r'b="say \"x\""') — after which my reference passes and the task is answerable.
The second bug was worse because it looked like a real finding. Qwen3.6 with thinking on "failed" rle after 3,141 tokens. The algorithm it wrote was completely correct. It had emitted the fenced block indented under a list item, my extractor kept the leading whitespace, and Python raised IndentationError on line 1. I had a headline sitting there — thinking mode makes Qwen3.6 worse at a trivial task — and it was entirely my own whitespace handling.
Both fixes are three lines: dedent the extracted block, and make the test literal raw. Because I had stored every raw generation, re-grading cost no inference at all:
text
1[qwen3-coder-next/default] parse_kv PASS 340 tok <-- CHANGED (pass1=FAIL)
2[qwen36-27b/nothink] parse_kv PASS 416 tok <-- CHANGED (pass1=FAIL)
3[qwen36-27b/think] rle PASS 3141 tok <-- CHANGED (pass1=FAIL)
4[qwen36-27b/think] parse_kv PASS 4096 tok <-- CHANGED (pass1=FAIL)
5[gemma4-31b/think] rle PASS 815 tok <-- CHANGED (pass1=FAIL)
6[gemma4-31b/think] parse_kv PASS 3148 tok <-- CHANGED (pass1=FAIL)
Every number in the two tables above is post-fix. The one surviving failure — Gemma's [a-zA-Z] decoder — is the only one that was ever real.
The generalisable rule: when several models fail the same case in the same way, suspect the harness before the models. A shared failure across independently trained models is far more likely to be your bug than their blind spot. And store the raw generations, so that finding out costs you a re-grade instead of a rerun.
What I would actually run today
If you have 96 GB of unified memory and you want the thing Ling 3.0 Flash is promising, Qwen3-Coder-Next is that thing, today, at 73 tok/s. Apache 2.0, 70.6 on SWE-bench Verified with the SWE-Agent scaffold by Qwen's own reporting, 3/3 on my tasks in under ten seconds, and no thinking mode to misconfigure.
Ling 3.0 Flash looks the better bet on paper. 5.1B active against 3B is close enough to a wash, but 124B of expert capacity against 80B is not, Ant's benchmark claims are aggressive, and MIT is more permissive than most of its peers. None of that is reachable from MLX right now, and the gap is three renames, a boolean and per-layer-type gate handling away from a codebase that already contains both halves of the answer.
If you write MLX ports, that is a well-specified afternoon with a very large model at the end of it. If you do not, wait for somebody who does — and stay sceptical of the GGUF conversions that appeared in the last twenty-four hours until someone demonstrates the KDA path produces sane logits rather than merely loading.
We do this kind of measurement at LeanZero because the same discipline decides whether an Atlassian app ships or embarrasses somebody: read the primary artefact, count the thing you are claiming, and distrust a number you did not produce yourself. The subject changes, the reflex does not.
Reproducing this
Everything above came off one Mac Studio — Apple M3 Ultra, 28 CPU cores, 96 GB unified memory — with mlx 0.32.0 and mlx-lm 0.31.3 in a venv.
Since writing this I went back and measured the memory side of the same box properly — the three ceilings Metal actually enforces, and what the KV cache of each of these architectures really costs at full context. It is in 96 GB is 77.76 GiB: the real memory ceiling on an M3 Ultra, and the short version is that Qwen3-Coder-Next fits at its full 262,144-token context with 11 GiB to spare.
1
Read the manifest, not the model
fetch each shard's safetensors header with an HTTP Range request. 7.96 MB gives you every name, shape and dtype in a 255 GB repo, and two shards is enough for a spot-check.
2
Check the vendor's arithmetic first
sum the shapes. If your method reproduces the numbers they published, you can trust it for the numbers they did not.
3
Ask mlx-lm before downloading
_get_classes(config) on the config alone tells you in about a second whether the architecture exists.
4
Diff a candidate port properly
install the module, build it at miniature scale from the real config, and replicate its own sanitize() before comparing names, or you will blame it for expert-stacking it already handles.
5
Benchmark on a quiet machine
unload everything, mx.synchronize() around the timings, and report decode and prefill separately. They are bound by different resources.
6
Grade code by running it, and write the reference solution yourself
otherwise a broken test masquerades as a model failure. Store the raw generations so re-grading is free.
Tip
mx.quantize.__doc__ prints the authoritative quantisation mode table for the MLX version you actually have — which modes exist, their group sizes, their scale types. Read that before trusting any blog post about MXFP4, this one included.
96 GB is 77.76 GiB: the real memory ceiling on an M3 Ultra
Metal will not give you the RAM on the box, and the number it does give is not the 75% everyone repeats. I measured the three ceilings on a 96 GB Mac Studio, then measured what modern hybrid-attention models actually spend against them — including a Gemma 4 cache that quietly holds three times its own sliding window.
