Cloud baseline

GPT-6.2 Sol, single model via OpenRouter

openai/gpt-6.2-sol · scorer forge-2.0

openai/gpt-6.2-sol · single agentNov 18, 2026
Overall
0.9988
Benchmark
Forge 2.0
Model build
23m 27s

Recorded inference cost

$2.28

From the saved OpenRouter generation billing summary (82 recorded requests). Whether auxiliary model calls were included was not retained. Scoring reuses the saved build; this amount describes the original inference spend.

Token usage

Tokens: 9.9M in · 87.1k out · 9.7M cached (exact, OpenRouter records)

10,004,786 tokens over 82 model calls: every call of the run, the model's own and the harness's helper calls, counted from OpenRouter records. Reasoning tokens as the provider reports them: 33,816 (not added to the total).

  • openai/gpt-6.2-sol

    In 9.9M · Out 87.1k · Cached 9.7M · Reasoning 33.8k · 81 calls

  • google/gemini-2.5-flash

    In 1.4k · Out 3 · Cached 0 · Reasoning 0 · 1 call

Tier breakdown

Each bar is the tier's mean, from 0 to 1 (for E, its mean × the excellence gate). The percentage is the tier's share of the score. Mean × share, added over all tiers, is the tests earned figure below.

L · Lint and bundles1.001.06%
K · Platform currency1.001.32%
T · Event pipeline1.002.11%
R · Reconcile1.001.85%
S · Storage1.001.06%
B · Resolvers and permissions1.001.58%
U · UI function1.002.11%
V · Visual1.001.06%
A · Rovo1.001.06%
E · Excellence0.93441.8%
R1 · Migration1.0012%
R2 · Dosing1.0013%
R3 · Time limits1.008%
R4 · Changing world1.0010%
R5 · Admin panel1.0010%
R6 · CI web trigger1.008%
R7 · Custom field1.005%
R8 · Forge LLM1.004%
R9 · Boot budget1.005%
R10 · Native JQL1.0010%

Scoring detail

How this score was built

Per-test scorer output, as posted by the app. Expand a section to inspect its tests.

Correctness combines the lowest score for each test across the scoring fixtures; economy measurements use their mean. The combined result can therefore be lower than every individual fixture’s final score. Critical defects reduce the weighted result, followed by the reliability multiplier, before any score cap applies.

The rules behind these numbers: Forge 2.0 requirements, weights and score caps.

The score, step by step

0.9988 tests earned × 1.0000 critical defects × 1.0000 reliability = 0.9988 final score

tests earned = 0.15 × 0.9921 v1 + 0.8500 v2 (out of 0.85)

v1 = 0.88 × 1.0000 tiers L–A + 0.12 × 1.0000 gate × 0.9344 excellence

Reliability: every tier in the breakdown above, except Excellence, is a group of tests. Each group of tests multiplies the score by 1 − 0.10 × the shortfall of its worst test: by 0.90 when that test fails completely, by 0.95 when it scores 0.5. No group of tests multiplied this score.

v1 is this run's score on the tests carried over from Forge 1.0: 15% of the total. v2 is the requirements R1–R10, each at its share of the score: the other 85%. Critical defects multiply the whole score, and so does each group of tests, by 1 − 0.10 × the shortfall of its worst test.

Excellence gate 1.0000: the average of the 18 conditions below, each counted at the value shown. Green is met in full; red is not, and lowers the gate. The 3 excellence tests are paid in proportion to the gate; together they are 1.8% of the total.

t_event_rows 1.00r_heal_dropped 1.00u_widget_numbers 1.00u_ledger_table 1.00b_comment_adf_as_user 1.00k_consumer_shape 1.00k_current_apis 1.00k_dashboard_widget 1.00k_entity_declared 1.00k_llm_model_current 1.00k_manifest_semantics 1.00k_rovo_mcp 1.00k_rovo_skill 1.00k_runtime_risks 1.00k_widget_edit_bridge 1.00v_console_clean 1.00v_csp_clean 1.00l_lint_warnings 1.00

Earned score 0.9988 · No cap applies · Final score 0.9988

Under a cap the final score is min(earned, cap − 0.05 × (1 − earned)): always below the cap, and closer to it the more the run earned. With no cap it is the earned score. Avoiding a cap adds no points.

Few Jira requests in the first scheduled run and per relevant change in triggers and consumers, and surfaces that paint after one round trip — paid only in proportion to the excellence gate.

e reconcile economy

1.00

mean over 3 scoring sites: backfill 30 background Jira calls (by lineage, every phase) / optimum 20 (incl. 2 forced 429 repeat(s)) = 1.50x; 1.0 at <= 2.45x; 5 2.0 call(s) not counted (field-value writes, person-facing, w

e event economy

0.8031

mean over 3 scoring sites: 757 Jira calls (by lineage, every phase) / baseline 214 (213 delivered relevant change(s) x 1 read + 1 forced 429 repeat; a normalizer, not a reachable optimum) = 3.54x; 1.0 at <= 2.86x; 144 fi

e ui round trips

1.00

mean over 3 scoring sites: round trips before first paint: {'widget-view': 0, 'widget-edit': 1, 'sprint-action': 1}

Graded browser recording

Loading duration…

Watch the full graded browser recording at its original speed: the dashboard widget, the sprint action, the not-started sprint and the Forge LLM cases, as the scorer drove them on the run's own seed, the first of the three scoring sites. The test results and screenshots on this page provide the wider evidence. Pause or scrub to inspect a view.

The UI Kit admin panel is not in the recording: it is graded from its component tree.

Ready to play0:00 / —:—
Open recording file

Drag the timeline to seek. Keyboard: arrow keys move 5 seconds; Home and End jump to the start and end.

Full graded browser recording: dashboard widget (before configuration, edit and Save, light and dark at 380 px, a second instance at 1180 px), sprint action (light and dark, sort, issue link, select, comment post, double click, forbidden post, explain, close), the not-started sprint and the Forge LLM cases, where the app has each. The UI Kit admin panel is not in the recording.

Screenshots

Pictures from one of the three scoring sites, the one on the run's own seed. The dashboard widget and the sprint action are captured from the built app in a browser during grading. The admin panel picture is drawn by the benchmark from the app's component tree, not by Jira, and nothing is graded from that picture. A test keeps the score of its worst site (the excellence measurements keep their mean), so a test can fail on a site these pictures do not show. A detail that opens "worst of …" names the seed of the site the test scored worst on, which can be the pictured one.

1Dashboard widget · light
2Dashboard widget · dark
3Sprint action · light
4Sprint action · dark
5Admin panel (UI Kit) · light: drawn by the benchmark from the app's component tree, not by Jira.

Run details

Model
openai/gpt-6.2-sol
Engine events
0
Repair rounds
0
Started
Nov 18, 2026, 06:05 AM
Finished
Nov 18, 2026, 06:49 AM
Call budget
With wrap-up nudge, version 2: as the call budget ran low, the agent harness told the model to start no new work and finish

How the benchmark is scored