Benchmark run

MiMo-V2.6-Flash 4-bit · 2-Mac split · text-only, 120k ctx — 0.5894 on Gauntlet 7.2

mlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8 · scorer sb-7.2

copper-ocelot-d8abmlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8 · single agentOct 27, 2026
Overall
0.5894
Excellent
no
Benchmark
Gauntlet 7.2
Model build
272m 29s
Prompt tok
7.0M
Gen tok
334.0k

Token usage

Tokens: 7M in · 334k out (exact, engine telemetry, reported by the publisher)

7,321,289 tokens over 130 model calls: every call of the run, the model's own and the harness's helper calls, counted from engine telemetry. Reported by the publisher's app; LeanZero did not count them.

  • mlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8

    In 7M · Out 334k · 130 calls

Tier breakdown

A · structure & runtime1.00
B · behaviour1.00
C · vendor contract0.95
D · finesse0.90

Scoring detail

How this score was built

Per-check scorer output, as posted by the app. Expand a section to inspect its checks.

Earned credit before admission

(0.88 × 0.8231 core + 0.12 × 1.00 gate × 0.70 excellence) × 1.0000 critical = 0.8083

The core (88% of the total) is the weighted mean of the thirteen measured tiers. The last 12% is the excellence slice: it unlocks in proportion to the perfection conditions below (16 of 16 met here), then pays out at the excellence tier's own measured mean. Core 0.7243 + excellence 0.0840.

j_first_use 1.00j_workflow_journey 1.00j_error_state 1.00j_empty_state 1.00console_clean 0.00v_responsive_375 1.00v_dates_readable 1.00t_scene_binding 1.00x_conservation_residual 1.00r_no_row_loss 1.00p_drag_frames 1.00p_idle_flatness 1.00p_stream_apply 1.00p_under_stream 1.00p_api_latency 1.00p_sync_wall 1.00

Earned score 0.808 · Admission ceiling 0.599 · Final score 0.589

Final score is the lower of earned credit and the admission ceiling. Passing admission adds no points.

Visible: not metMatching: not metGood: not metVisual excellence: not metBackend excellence: passed

Visible data-backed 3D: s_visible_surface (maximum 0.599)

Tower structure, currency mapping and scene truth: s_visible_surface, s_tower_geometry, s_currency_collar, t_height_pixels, t_vs7dbg_truth (maximum 0.609)

3D interaction and overview legibility: t_camera_math, t_labels_culling, t_brush_link, q_overview_legibility, q_inspector_framing (maximum 0.699)

Event animation and backend recovery: m_committed_event_replay (maximum 0.899)

Run notes and corrections

Ran locally in Goose across two Macs (M3 Ultra 96 GB with layers 30-47, M5 Max 128 GB with layers 0-29) over Thunderbolt 5 RDMA. Known limits of this run: the MLX build used is text-only (its vision encoder was stripped), so the model could not view the screenshots it took, and the 120k context led to 5 compactions where the cloud run had none. Most of the gap to the cloud score (0.79) is one 3D rendering task that needed visual checking. A re-run with the full multimodal build and a 256k context follows.

Token rates

Measured by the engine itself, one record per completed model call: prefill rate is prompt tokens over time-to-first-token, generation rate is completion tokens over the decode window. Medians per node.

NodeCallsPrompt tokGen tokPrefill tok/sGen tok/s
mlx1306,987,332333,95712192.727.4
fleet1306,987,332333,95712192.727.4

Graded browser recording

Loading duration…

Watch the full graded browser recording at its original speed: the payment field, structure inspection, committed updates and replay. The check results and screenshots on this page provide the wider evidence. Pause or scrub to inspect a view.

Each tower represents a payment: height encodes its amount, cap color its status, and the collar its currency. The selected detailed tower is expected to animate its collar after a committed backend update or replay. These are the task requirements; the check results show what this app actually achieved.

Ready to play0:00 / —:—
Open recording file

Drag the timeline to seek. Keyboard: arrow keys move 5 seconds; Home and End jump to the start and end.

Graded browser recording re-encoded to 848x530 at 25 fps to fit 4 MiB, same 192.7 s; graded original sha256 726a095ba7ca1b1d2c5683304b18be716de49077c56220f3e76bc3cbe13fc082

Screenshots

Captured from the built application during browser grading. Captions identify the recorded views and checks; these images do not represent repair rounds.

1First captured render
2Latest captured render
3After sync
4Error state
5Mobile · 375px

Run details

Model
mlx-community/MiMo-V2.6-Flash-RL-mxfp4-q8
Engine events
0
Repair rounds
0
Started
Oct 27, 2026, 02:44 PM
Finished
Oct 27, 2026, 07:26 PM

How the benchmark is scored