Cloud baseline

Gemini 4 Carbon — 0.7981 on Gauntlet 7.2

google/gemini-4-carbon · scorer sb-7.2

Gemini 4 Carbon, single model via OpenRoutergoogle/gemini-4-carbon · single agentNov 11, 2026
Overall
0.7981
Excellent
no
Benchmark
Gauntlet 7.2
Model build
79m 24s
Prompt tok
14.9M
Gen tok
213.8k

Recorded inference cost

At least $8.98

From the saved OpenRouter generation billing summary (86 recorded requests). This is a lower bound; the billing record is incomplete. Includes auxiliary model calls made during the build. Scoring reuses the saved build; this amount describes the original inference spend.

  • 5 telemetry call(s) carried no response id to bill
  • The harness counted tokens no recovered generation id accounts for: prompt +653968, cached +646342, completion +3145

Token usage

Tokens: at least 14.2M in · 211k out · 11.3M cached (lower bound, OpenRouter records)

14,453,086 tokens over 91 model calls: every call of the run, the model's own and the harness's helper calls, counted from OpenRouter records. 5 calls have no token count, so these are a lower bound. Reasoning tokens as the provider reports them: 99,754 (not added to the total).

5 calls of the run have no readable OpenRouter record; their tokens are not in the totals. OpenRouter's records for google/gemini-4-carbon differ from Elzi's session counters (-654,942 in, -646,342 cached, -3,150 out), so they are not exact.

  • google/gemini-4-carbon

    In 14.2M · Out 211k · Cached 11.3M · Reasoning 99.8k · 90 calls

  • google/gemini-2.5-flash

    In 974 · Out 5 · Cached 0 · Reasoning 0 · 1 call

Tier breakdown

A · structure & runtime1.00
B · behaviour0.98
C · vendor contract0.94
D · finesse0.90

Scoring detail

How this score was built

Per-check scorer output, as posted by the app. Expand a section to inspect its checks.

Earned credit before admission

(0.88 × 0.9801 core + 0.12 × 1.00 gate × 1.00 excellence) × 1.0000 critical = 0.9825

The core (88% of the total) is the weighted mean of the thirteen measured tiers. The last 12% is the excellence slice: it unlocks in proportion to the perfection conditions below (16 of 16 met here), then pays out at the excellence tier's own measured mean. Core 0.8625 + excellence 0.1200.

j_first_use 1.00j_workflow_journey 1.00j_error_state 1.00j_empty_state 1.00console_clean 0.00v_responsive_375 1.00v_dates_readable 1.00t_scene_binding 1.00x_conservation_residual 1.00r_no_row_loss 1.00p_drag_frames 1.00p_idle_flatness 1.00p_stream_apply 1.00p_under_stream 1.00p_api_latency 1.00p_sync_wall 1.00

Earned score 0.983 · Admission ceiling 0.799 · Final score 0.798

Final score is the lower of earned credit and the admission ceiling. Passing admission adds no points.

Visible: passedMatching: passedGood: not metVisual excellence: not metBackend excellence: not met

3D interaction and overview legibility: q_overview_legibility (maximum 0.799)

Event animation and backend recovery: r_b3_sigkill_resync (maximum 0.899)

Token rates

Measured by the engine itself, one record per completed model call: prefill rate is prompt tokens over time-to-first-token, generation rate is completion tokens over the decode window. Medians per node.

NodeCallsPrompt tokGen tokPrefill tok/sGen tok/s
gemini9014,897,339213,8399616.1139.9
fleet9014,897,339213,8399616.1139.9

Graded browser recording

Loading duration…

Watch the full graded browser recording at its original speed: the payment field, structure inspection, committed updates and replay. The check results and screenshots on this page provide the wider evidence. Pause or scrub to inspect a view.

Each tower represents a payment: height encodes its amount, cap color its status, and the collar its currency. The selected detailed tower is expected to animate its collar after a committed backend update or replay. These are the task requirements; the check results show what this app actually achieved.

Ready to play0:00 / —:—
Open recording file

Drag the timeline to seek. Keyboard: arrow keys move 5 seconds; Home and End jump to the start and end.

Full graded browser recording: payment field, structure inspection, committed updates and replay

Screenshots

Captured from the built application during browser grading. Captions identify the recorded views and checks; these images do not represent repair rounds.

1First captured render
2Latest captured render
3After sync
4Error state
5Mobile · 375px

Run details

Model
google/gemini-4-carbon
Engine events
0
Repair rounds
0
Started
Nov 11, 2026, 08:34 PM
Finished
Nov 11, 2026, 10:06 PM
Call budget
With wrap-up nudge, version 2: as the call budget ran low, the agent harness told the model to start no new work and finish

How the benchmark is scored