Cloud baseline

Nemotron 3.5 Super VL — 0.3781 on Gauntlet 7.2

nvidia/nemotron-3.5-super-vl · scorer sb-7.2

Nemotron 3.5 Super VL, single model via OpenRouternvidia/nemotron-3.5-super-vl · single agentOct 18, 2026
Overall
0.3781
Excellent
no
Benchmark
Gauntlet 7.2
Model build
24m 7s
Prompt tok
14.1M
Gen tok
179.6k

Recorded inference cost

$1.21

From the saved OpenRouter generation billing summary (146 recorded requests). Includes auxiliary model calls made during the build. Scoring reuses the saved build; this amount describes the original inference spend.

Tier breakdown

A · structure & runtime1.00
B · behaviour0.68
C · vendor contract0.89
D · finesse0.87

Scoring detail

How this score was built

Per-check scorer output, as posted by the app. Expand a section to inspect its checks.

Earned credit before admission

(0.88 × 0.4208 core + 0.12 × 0.62 gate × 0.42 excellence) × 0.9429 critical = 0.3781

Critical defects compound a multiplier on the whole score (pre-severity 0.4010):

j_workflow_journey 0.86 → ×0.94

b_money_rendered: no additional penalty (vacuous:sync_completeness); wrong money — wrong exponent/digits or a cross-currency sum.

b_buckets_dst: no additional penalty (vacuous:sync_completeness); Incorrect daily bucket counts in the incomplete initial dataset (2112/12289 payments).

x_conservation_residual: no additional penalty (vacuous:sync_completeness); wrong money — unexplained minor units created/destroyed after dupe/loss attribution.

x_no_lost_write: no additional penalty (vacuous:sync_completeness); wrong money — an acknowledged mutation absent from final state.

r_no_dupe_effect: no additional penalty (vacuous:sync_completeness); wrong money — a ledger effect applied twice.

r_cache_truth: no additional penalty (vacuous:sync_completeness); data loss — 304-vs-cache mismatch served as fresh.

The core (88% of the total) is the weighted mean of the thirteen measured tiers. The last 12% is the excellence slice: it unlocks in proportion to the perfection conditions below (9 of 16 met here), then pays out at the excellence tier's own measured mean. Core 0.3703 + excellence 0.0307.

j_first_use 1.00j_workflow_journey 0.86j_error_state 0.00j_empty_state 1.00console_clean 0.00v_responsive_375 0.00v_dates_readable 1.00t_scene_binding 0.00x_conservation_residual 0.00r_no_row_loss 1.00p_drag_frames 1.00p_idle_flatness 1.00p_stream_apply 0.00p_under_stream 1.00p_api_latency 1.00p_sync_wall 0.00

Earned score 0.378 · Admission ceiling 0.599 · Final score 0.378

Final score is the lower of earned credit and the admission ceiling. Passing admission adds no points.

Visible: not metMatching: not metGood: not metVisual excellence: not metBackend excellence: not met

Visible data-backed 3D: s_visible_surface (maximum 0.599)

Tower structure, currency mapping and scene truth: s_visible_surface, s_tower_geometry, s_currency_collar, t_layout_basis, t_scene_binding, t_height_pixels, t_vs7dbg_truth (maximum 0.599)

3D interaction and overview legibility: t_draw_budget, t_pick_buffer, t_pick_real_pass, t_click_semantics, t_camera_math, t_coast_identity, t_coast_reality, t_labels_culling, t_brush_link, t_stream_diff, q_overview_legibility, q_inspector_framing (maximum 0.699)

Event animation and backend recovery: m_committed_event_replay, x_l1_no_invented_states, x_l2_per_key_order, x_l3_monotonic_reads, x_l4_convergence, x_l5_group_atomicity, x_m1_amount_immutability, x_m3_terminal_conservation, x_conservation_residual, x_no_lost_write, x_ooo_dup_forged, r_b6_outbox_atomic, r_notifier_exactly_once, r_no_dupe_effect, r_cache_truth (maximum 0.799)

Token rates

Measured by the engine itself, one record per completed model call: prefill rate is prompt tokens over time-to-first-token, generation rate is completion tokens over the decode window. Medians per node.

NodeCallsPrompt tokGen tokPrefill tok/sGen tok/s
gemini19725976.947.6
nemotron14514,080,900179,59063308.0163.5
fleet14614,081,872179,59563112.8163.2

Graded browser recording

Loading duration…

Watch the full graded browser recording at its original speed: the payment field, structure inspection, committed updates and replay. The check results and screenshots on this page provide the wider evidence. Pause or scrub to inspect a view.

Each tower represents a payment: height encodes its amount, cap color its status, and the collar its currency. The selected detailed tower is expected to animate its collar after a committed backend update or replay. These are the task requirements; the check results show what this app actually achieved.

Ready to play0:00 / —:—
Open recording file

Drag the timeline to seek. Keyboard: arrow keys move 5 seconds; Home and End jump to the start and end.

Full graded browser recording: payment field, structure inspection, committed updates and replay

Screenshots

Captured from the built application during browser grading. Captions identify the recorded views and checks; these images do not represent repair rounds.

1First captured render
2Latest captured render
3After sync
4Error state
5Mobile · 375px

Run details

Model
nvidia/nemotron-3.5-super-vl
Engine events
0
Repair rounds
0
Started
Oct 18, 2026, 04:16 PM
Finished
Oct 18, 2026, 04:57 PM

How the benchmark is scored