Benchmark run

Qwen3.8-Flash-Next oQ4e DWQ (LeanZero) · lz.27 · vision+YaRN · workmule, 512k — 0.6932 on Gauntlet 7.2

LeanZero/Qwen3.8-Flash-Next-oQ4e-dwq · scorer sb-7.2

proud-iguana-cf02LeanZero/Qwen3.8-Flash-Next-oQ4e-dwq · single agentNov 4, 2026
Overall
0.6932
Excellent
no
Benchmark
Gauntlet 7.2
Model build
137m 57s

Token usage

Tokens: 28.3M in · 255k out (exact, harness session counters, reported by the publisher)

28,529,838 tokens over 138 model calls: every call of the run, the model's own and the harness's helper calls, counted from harness session counters. Reported by the publisher's app; LeanZero did not count them.

  • LeanZero/Qwen3.8-Flash-Next-oQ4e-dwq

    In 28.3M · Out 255k · 138 calls

Tier breakdown

A · structure & runtime1.00
B · behaviour0.98
C · vendor contract0.93
D · finesse0.90

Scoring detail

How this score was built

Per-check scorer output, as posted by the app. Expand a section to inspect its checks.

Earned credit before admission

(0.88 × 0.9316 core + 0.12 × 0.89 gate × 0.60 excellence) × 1.0000 critical = 0.8841

The core (88% of the total) is the weighted mean of the thirteen measured tiers. The last 12% is the excellence slice: it unlocks in proportion to the perfection conditions below (14 of 16 met here), then pays out at the excellence tier's own measured mean. Core 0.8198 + excellence 0.0644.

j_first_use 1.00j_workflow_journey 1.00j_error_state 0.30j_empty_state 1.00console_clean 0.00v_responsive_375 1.00v_dates_readable 1.00t_scene_binding 1.00x_conservation_residual 1.00r_no_row_loss 1.00p_drag_frames 1.00p_idle_flatness 1.00p_stream_apply 0.00p_under_stream 1.00p_api_latency 1.00p_sync_wall 1.00

Earned score 0.884 · Admission ceiling 0.699 · Final score 0.693

Final score is the lower of earned credit and the admission ceiling. Passing admission adds no points.

Visible: passedMatching: not metGood: not metVisual excellence: not metBackend excellence: not met

Tower structure, currency mapping and scene truth: t_vs7dbg_truth (maximum 0.699)

3D interaction and overview legibility: t_pick_buffer, t_click_semantics, t_labels_culling, t_brush_link, t_stream_diff, q_overview_legibility (maximum 0.699)

Event animation and backend recovery: x_l1_no_invented_states, r_b3_sigkill_resync (maximum 0.869)

Run notes and corrections

This run was local, on one Mac Studio with an M5 Max and 128 GB of memory. It ran in harness version 3.0.151 on LeanZero's MLX engine v0.14.3-lz.27, with vision on and a 524,288-token context (YaRN factor 4). It used 138 of its 150 model calls and never compacted. The model is our own DWQ build of Qwen's official Qwen3.8-Flash-Next weights. It keeps the 4-bit oQ4e layout, and we retrained the scales and biases of the routed experts layer by layer against Qwen's BF16 release. The tier asks for medium reasoning effort, but that setting doesn't reach a local model, so this score wasn't taken at a pinned effort. The build first scored 0.5909. We re-scored the saved build on harness version 3.0.154, whose screenshot checks now handle a canvas that sits on a fractional pixel boundary. With that fix the visible 3D surface check and the tower legend text pass, and the score rose to 0.6932. Two checks that vary from one scoring to the next (the camera wheel check and the mastery rung that depends on it) also came out higher in the re-score; they don't decide the result. There's one caveat on the build itself. On a clean test set it sits closer to Qwen's BF16 model than our plain 4-bit build (KL down 10.6% overall, all of it on tool-call text). On that same tool-call text, though, its perplexity moves slightly further from the original's, and we don't know why yet.

Graded browser recording

Loading duration…

Watch the full graded browser recording at its original speed: the payment field, structure inspection, committed updates and replay. The check results and screenshots on this page provide the wider evidence. Pause or scrub to inspect a view.

Each tower represents a payment: height encodes its amount, cap color its status, and the collar its currency. The selected detailed tower is expected to animate its collar after a committed backend update or replay. These are the task requirements; the check results show what this app actually achieved.

Ready to play0:00 / —:—
Open recording file

Drag the timeline to seek. Keyboard: arrow keys move 5 seconds; Home and End jump to the start and end.

Full graded browser recording: payment field, structure inspection, committed updates and replay

Screenshots

Captured from the built application during browser grading. Captions identify the recorded views and checks; these images do not represent repair rounds.

1First captured render
2Latest captured render
3After sync
4Error state
5Mobile · 375px

Run details

Model
LeanZero/Qwen3.8-Flash-Next-oQ4e-dwq
Engine events
0
Repair rounds
0
Started
Nov 3, 2026, 09:21 PM
Finished
Nov 3, 2026, 11:49 PM
Call budget
With wrap-up nudge, version 2: as the call budget ran low, the agent harness told the model to start no new work and finish
Re-scored
Nov 4, 2026, 05:03 AM (same build, corrected scorer)

How the benchmark is scored