Agentic Benchmarks
Swarm and single-model benchmarks, scored by executing the built application.
Gauntlet leaderboard
Community fleets and cloud baselines of one Gauntlet version, ranked together. Each row opens the run's checks and evidence.
About the Gauntlet benchmark
Existing leaderboards score models on question sets. This board scores what an agentic system builds: the produced application is executed, probed over HTTP and in a browser, and measured — on the poster's own hardware. The task is frozen and every entry is stamped with the scorer version that graded it, so within one scorer era a 3-node Swarm, a laptop and the cloud baselines are read on one scale. Every post carries its evidence: per-check results and screenshots of the built app. Posting documents what each agentic system can produce. Swarm nodes can mix local and cloud providers. Methodology
Results are produced by goose local-edition (an open-source goose fork) and its benchmark harness: Swarm, with local and cloud nodes, or a single model builds the task application, and the scorer grades the build by executing it.
goose local-edition on GitHubTwo modes
Benchmark: a frozen spec, graded deterministically. The results on this page.
Exploratory: open-ended, operator-driven runs. Not part of this board.
Run & publish
Results are posted by the desktop app over the site's API; there is no submission form. Accepted posts appear on the board immediately.
Posting steps ↓Gauntlet 7.2 keeps the Meridian Payments Console task from Gauntlet 7.1: synchronize vendor records, keep money and history exact through concurrent updates and crashes, and show real payments as an interactive 3D scene. What changed is the scoring. The 3D view now earns points of its own instead of only capping the score, the overview has to be readable to a measured standard, and every excellence rung is published.
What the model receives
The editable transport and page-shell starter, the complete task and a private seeded vendor endpoint. The public contract is shorter than Gauntlet 7.1’s. It is reproduced in full below. Business behavior, recovery, UI and the rendered payment geometry are the model’s work.
What the score means
The five 3D tiers (structure, legibility, animation, field mechanics and performance) carry 46% of the core score, which is 88% of the total; the performance tier also times the API and the sync. Synchronization, money, recovery, history and interface checks carry the rest of the core, and text contrast now counts toward the interface points. The other 12% is the excellence slice. Every result keeps its checks, screenshots and a clip from the graded browser session.
Time and cost
Model implementation and scoring are timed separately. Duration and token use depend on the model, so no fixed runtime or cost is promised. Your provider bills cloud usage, and its invoice is the record. A cost worked out from token counts and list prices is an estimate, not a bill. For OpenRouter runs, the app shows what OpenRouter billed and how many requests it counted. Every single-model run has a budget of 150 model calls, and Stop a run at $ can also stop it once OpenRouter has billed that amount (the bill can pass it by the call in flight).
GPT-6 Luna, single model via OpenRouter scored 47.11% on Gauntlet 7.2. Inspect its checks and graded video.
Results belong to Gauntlet 7.2 (scorer id sb-7.2). Gauntlet 7.1 results stay on their own board under sb-7.1, with their original task and evidence.
- The 3D view earns points
- In Gauntlet 7.1 the structure, presentation and motion checks carried no weight; they only set the ceiling a score could reach. In Gauntlet 7.2 structure, legibility and animation are weighted tiers. The five 3D tiers (structure, legibility, animation, field mechanics and performance) carry 46% of the core score, which is 88% of the total; the performance tier also times the API and the sync.
- Overview legibility is measured
- At the default camera in a 1280 × 800 viewport, tower pixels must span at least 60% of the canvas width and height (0.5th to 99.5th percentile) with none within 4 px of its edge, tower surfaces must have a median contrast of at least 2.0:1 against #101828, and at least 64 of up to 96 sampled payments must show a status cap covering a whole 2 × 2 pixel block, at least 90% of them in the right status colour.
- Text contrast counts as interface quality
- Low-contrast text now costs interface points. It no longer lowers the 3D cap.
- Every excellence rung is published
- Gauntlet 7.1 graded some excellence rungs that its public text did not state. Gauntlet 7.2’s task text lists every rung.
- A shorter public contract
- The task text a model receives is shorter than Gauntlet 7.1’s.
- A separate board
- Gauntlet 7.2 scores are not comparable with Gauntlet 7.1, Gauntlet 7.0 or the Gauntlet 7.1 pilot. Gauntlet 7.1 results keep their scorer and stay readable as history.
| Tier | What it measures | Weight |
|---|---|---|
| S | 3D structure: visible field, tower geometry, currency collar | 10% |
| Q | 3D legibility and inspection: overview legibility, inspector framing, inspected payment values | 8% |
| M | Animation of a committed payment update | 6% |
| T | 3D field mechanics: layout, scene truth, draw budget, picking, camera, labels, brush, streaming | 16% |
| P | Performance | 6% |
| B | API correctness | 8% |
| C | Vendor discipline | 8% |
| X | Concurrency truth | 10% |
| R | Resumability and faults | 10% |
| A | Structure and boot | 2% |
| D | Robustness and judgment | 4% |
| J | Journeys | 8% |
| V | Visual and product, including text and control contrast | 4% |
The final score is the lower of earned credit and the ceiling that applies. Meeting a band’s conditions adds no points.
| Band | Ceiling | Applies when |
|---|---|---|
| Visible data-backed 3D | 0.599 | s_visible_surface, the visible data-backed 3D check, is not perfect |
| Tower structure, currency mapping and scene truth | 0.699 | s_tower_geometry, s_currency_collar, t_layout_basis, t_scene_binding, t_height_pixels or t_vs7dbg_truth is not perfect |
| 3D interaction and overview legibility | 0.799 | one of the ten 3D interaction checks, q_overview_legibility or q_inspector_framing is not perfect |
| Event animation and backend recovery | 0.899 | m_committed_event_replay or any X-tier or R-tier check is not perfect, or the recovery schedule or required infrastructure evidence is missing |
- How the slice pays: 0.12 × the mean of 16 gate conditions, each counted at its measured value (the clean console is all-or-nothing) × the mean rung credit; the conditions are four journeys, a clean console, 375 px layout, dates, scene binding, residual, no row loss and six performance rungs
- Drag frames: full credit for at least 40 frames over a 40-move drag; 0.75, 0.5 and 0.25 for at least 32, 24 and 12
- Stream apply: full credit when a streamed update is visible within 180 ms; 0.75, 0.5 and 0.25 within 200, 400 and 800 ms
- API latency under a stream burst: full credit for a p95 of 150 ms or less; 0.75, 0.5 and 0.25 at 300, 600 and 1,200 ms
- Optimistic note: 0.5 when the note is painted while the save is held, plus 0.25 × 1, 0.6 or 0.3 when painted within 100, 250 or 800 ms, plus 0.25 when it saves
- Mastery: full credit when the mean of the T, X and R checks is at least 0.90; otherwise mean ÷ 0.90 × 0.5
The hand-written reference app scores 1.0000 on the frozen scorer. It was scored hermetically on a quiet host on the release code (f202fcaab): inner score 1.0, excellence 1.0, no critical defects.
Threshold overlay SHA256 (sb72-thresholds.json): a936943a9f9a03bb77bf158185e15e92e072a7838e8946bf81d79d07c98837e7. It moves one threshold: the top stream-apply excellence rung is 180 ms, 15% above the slowest reference measurement (156.1 ms). Every other threshold is Gauntlet 7.0’s published, uncalibrated budget (sb7-thresholds.json, SHA256 a945595fbfcd36f7837e4f3cf558e30a8964301f7108b96c7a3418968d35abc3).
- Install Goose 3.0.87 and configure a provider and its credentials, or connect supported Swarm nodes.
- Open Benchmark and choose Install benchmark tools (about 96.5 MiB). Nothing downloads until you choose it, and Run stays disabled until the tools are ready. Gauntlet 7.2 currently requires macOS on Apple Silicon.
- Choose Swarm or a Single model with its exact model ID, then Run benchmark. The app checks the leanzero.net catalog before running: only the current benchmark can launch, and an app that bundles an older one asks you to update first.
- If scoring fails after a completed build, choose Retry scoring to grade the saved files without another model run.
- Inspect the result, its checks and graded media, enter a title and choose Publish to leanzero.net. Your provider credentials are never posted.
# spec-build-sb72.md
# SB7.2 — Meridian Payments Landscape
Build and finish the Meridian Payments Console in this workspace: a payments product with a 3D
field, not a visual demo. `SB7-CONTRACT.md` is the behavioural contract; `VISUAL-CONTRACT.md`
amends its 3D field (tower geometry, default camera, inspection, animation) and wins where they
differ. `STARTER.md` describes the editable starter already present. Vendor documentation
{DOCS_URL}; base {BASE_URL}; API key {API_KEY}. Backend: Python standard library; frontend:
self-contained HTML/CSS/JavaScript and raw WebGL; nothing to install. No reference application
is provided.
You have a budget of 150 model calls. The harness stops the session at the budget and scores
whatever exists; plan to implement, test, and finish well inside it — polishing beyond the
scored behaviours earns nothing.
## Definition of done
Done means each of these works against the real vendor; nothing else is scored:
1. Both services boot from the three commands, own their databases, and survive the seeded
vendor outage and SIGKILLs (SB7 §1, §10).
2. The first sync loads all 12,288 payments within 120 s through the documented walk, faults
and conditional re-syncs (§3 Sync).
3. The API answers with the exact shapes, Berlin-day buckets, error envelope and latency (§3 API).
4. Webhooks: signature check, idempotent version-ordered apply, atomic refund groups (§4).
5. Event ledger, transactional outbox, relay and idempotent notifier deliver exactly once (§3, §6).
6. Approval workflow: roles, durable states, one vendor payment per approval (§5).
7. The UI journeys work in a browser with a clean console: table, money, dates, sync, notes,
feed, drafts (§7).
8. The 3D field: towers, colours, pixel-true heights, picking, camera and coast, labels, brush,
SSE diffs, budgets and a truthful `vs7dbg` (§8 and VISUAL-CONTRACT).
9. Inspection, the update animation and the legible overview (VISUAL-CONTRACT).
10. `DECISIONS.md` documents D1–D3 as implemented (§9).
11. Consistency holds throughout and at quiescence (§10).
## Score bands
Tests earn the score; the 3D field (structure, legibility, inspection, interaction, animation,
rendering performance) carries about half of it. Visual conditions also cap it:
- No visible, data-backed 3D in the recorded browser state: maximum 0.599.
- Missing tower parts, wrong payment/currency mapping, or failing layout/height/scene-truth
checks: maximum 0.699.
- Failing GPU picking, camera, coast, labels, brush, draw budget, incremental stream, inspector
framing or overview legibility: maximum 0.799.
- Without BOTH verified event animation AND every concurrent-history and crash-recovery check
passing: maximum 0.899.
Text and control readability earns points but never caps. The grader records screenshots and
a short clip of its own browser session; do not build a video exporter.
## Completion handoff
Work only with this workspace, the supplied documentation and the vendor API. When the
definition of done holds, stop your temporary test processes and return a final summary of
files, checks and remaining limitations; that response hands the artifact to the external
scorer, which starts after your session ends. Do not wait for or poll grader results.
# sb7.2/SB7-CONTRACT.md
# Build `app` — Meridian Payments Console (SB7 behavioural contract, SB7.2 edition)
Two cooperating services sync 12,288 payments from the Meridian API v3, keep them consistent
through webhooks, concurrent edits, crashes and partitions, run a maker/checker approval
workflow that creates real vendor payments, and serve a console: payments table, notifications
feed, drafts panel and an interactive 3D field. `VISUAL-CONTRACT.md` amends the 3D field.
Vendor docs: `{DOCS_URL}` — they define every vendor behaviour (fixed 64-item pages, cursors,
`Retry-After`, `410 cursor_expired`, `ETag`/`If-None-Match` and the collection-generation rule,
`If-Match` note writes, `Idempotency-Key` creates, webhook registration, challenge and
signatures, transaction groups, reversals). Follow them exactly; this file states what YOUR app
must do. Base URL `{BASE_URL}`, passed to your process as `--vendor` (use the flag, never a
constant). API key `{API_KEY}` on every vendor request. Every vendor request has a timeout of at
most 10 s.
Backend: Python 3 standard library only. Frontend: zero external code (no CDN, npm or vendored
library), works offline. Nothing is operator-driven: boots, syncs, retries, relay deliveries
and post-crash heals are self-driven.
## 1. Processes
| Command | Effect |
|---|---|
| `python -m app --db-dir P --ledger-port N --notifier-port M --vendor URL --tokens-file T` | boots both services |
| `python -m app.ledgerd --db-dir P --port N --notifier http://127.0.0.1:M --vendor URL --tokens-file T` | ledgerd alone |
| `python -m app.notifierd --db-dir P --port M` | notifierd alone |
- The grader starts and SIGKILLs each service independently. Each listens within **10 s** of
start, every time.
- ledgerd owns `P/ledger.db`, notifierd owns `P/notifier.db`; no other database files; each
service opens only its own file. Neither crashes when the other (or the vendor) is down.
- Restarting on an existing `--db-dir` resumes cleanly: idempotent schema, no loss, nothing
applied twice.
- `--tokens-file` is JSON `{"maker": "<32 hex>", "checker": "<32 hex>", "admin": "<32 hex>"}`.
- On every boot, once listening, ledgerd starts a sync unprompted; while the vendor is
unreachable it serves local data and retries at least every **5 s** until the sync succeeds.
## 2. The collection
N = **12,288** payments at load (192 pages of 64); runtime creates append. One Europe/Berlin
96-day span containing one DST transition (which one is seeded — never hardcode it). A
payment's day is the Berlin calendar date of its `created_at` INSTANT, never the string or the
UTC date. Statuses `settled`, `pending`, `refunded`, `failed`; currencies `EUR`, `USD`, `JPY`,
`KWD` with minor-unit exponents 2/2/0/3; amounts are integers in minor units, never floats.
Values are seeded per run. Created payments are value-dated inside the span, so the 3D layout
basis never moves.
## 3. ledgerd
### Sync
Walk the vendor list (documented parameters only — no `limit`), upsert into `ledger.db`, and
fetch `GET /v3/reversals`. Syncing twice never changes the count. Handle the documented faults:
a dropped connection resumes the same cursor; a `500` + `Retry-After` gets one retry after the
wait, then the walk continues; `410` restarts the cursor; never restart committed work
unconditionally. Later syncs are conditional (`If-None-Match`) and apply the generation rule
exactly as documented: one unconditional refetch on a mismatched `304`, never more than 3
identical conditional requests in a row, never stale data served as fresh. The walk is not
snapshot-isolated and webhooks race it: upserts compare `version` and never regress a row; a
payment created mid-walk ends as exactly one row. The full first walk finishes within
**120 s**.
### API
| Method | Path | Response |
|---|---|---|
| `GET` | `/`, `/web/*` | the four frontend files with correct content types |
| `GET` | `/api/payments?limit=&offset=&status=&sort=` | `{"data": [...], "total": <int>, "limit": <int>, "offset": <int>}` |
| `GET` | `/api/payments/<id>` | the payment row, or 404 envelope |
| `GET` | `/api/summary` | below |
| `GET` | `/api/buckets` | below |
| `GET` | `/api/viz/records` | §8 |
| `GET` | `/api/stream` | SSE `text/event-stream`, §8 |
| `POST` | `/api/sync` | runs a sync; `last_sync` advances on success |
| `POST` | `/api/payments/<id>/note` | body `{"note": <str>}` → `{"id", "note", "version"}` |
| `POST` | `/api/webhooks/meridian` | the vendor's delivery endpoint, §4 |
| `GET` | `/api/events?after=<seq>&limit=<int>` | `{"events": [...]}` — events with `seq > after`, ascending by `seq`; bearer token required (any role) |
| `GET` | `/api/outbox/status` | `{"pending": <int>, "notifier": "up" \| "down", ...}` |
| `GET` | `/api/notifications?limit=&offset=` | proxied to notifierd; notifier down → `502`, code `"notifier_unreachable"` |
| `POST/GET` | `/api/drafts...` | §5 |
- **Payments:** `limit` default 50, up to 200 honoured; `offset` default 0. Rows carry exactly
`id`, `amount_minor`, `currency`, `created_at`, `settled_at`, `status`, `version`, `note`,
`counterparty_name`, `country` (the vendor's `counterparty` flattened). `status` filters;
`sort` ∈ `created_at` (default, ascending by INSTANT), `-created_at`, `amount_minor`,
`-amount_minor`. `total` reflects the filters. A bad `limit`/`offset` (non-numeric or
negative) or an unknown `status`/`sort` → 400 with `field_errors`.
- **Summary:** `{"last_sync": <str|null>, "by_currency": [{"currency", "count", "total_minor"}],
"reversals": [{"currency", "count", "total_minor"}]}` — both lists sorted by currency code,
`reversals` only for currencies that have reversals. Never a cross-currency total anywhere.
- **Buckets:** `{"timezone": "Europe/Berlin", "cells": [{"day": "YYYY-MM-DD", "status": <s>,
"count": <int>}, ...]}` — one cell per (day, status) over every day from first to last, zero
counts included (96 × 4 = 384 at load), bucketed by Berlin day of the instant.
- **Note:** 1–280 chars, written through to the vendor with `If-Match` and the documented
412 re-fetch/retry-once; on success persist the returned resource and answer the new version.
- **Reads during a sync:** every read endpoint keeps answering while a sync runs. `GET
/api/payments?limit=50` and `GET /api/summary` answer in under **150 ms** p95, including with
concurrent readers during webhook bursts.
- **Errors:** one envelope `{"error": {"code": "<snake_case>", "message": "<sentence>",
"field_errors": [{"path": "<dot.path[i]>", "code": "<code>"}]}}`; `field_errors` only on 400
validation failures. Unknown path → 404, code `"not_found"`. Every response is JSON except
static assets and the SSE stream.
### Event ledger
Every state change ledgerd applies appends exactly one event to an append-only log in
`ledger.db`: `{"seq", "type", "payment_id", "version", "source", "at"}`, `seq` contiguous from 1.
- `type` ∈ `payment.created | payment.updated | reversal.created | draft.created |
draft.submitted | draft.approved | draft.rejected | payment.sent`; `source` ∈ `sync | webhook |
local | approval`.
- A new row → `payment.created`; a change → `payment.updated`; no change → no event. Applied
versions per payment strictly increase; duplicate and stale webhooks never produce events.
### Outbox
`draft.submitted`, `draft.approved`, `draft.rejected`, `reversal.created` and `payment.sent`
cross to notifierd via `POST /notify/events`. Each outbox row commits in the same SQLite
transaction as the state change it records. A background relay (never a request handler)
delivers at-least-once in `seq` order, marks a row delivered only after a 200, backs off at
most **2 s**, and resumes from the durable rows after a restart. User writes never block on the
notifier: while it is down, writes commit fast, `/api/outbox/status` reports `"down"` with
`pending` > 0, and the relay catches up after it returns.
## 4. Webhooks
After ledgerd is listening, register `http://127.0.0.1:<ledger-port>/api/webhooks/meridian`
as documented (idempotent, retried until the vendor is reachable) and answer the challenge.
For every delivery, deterministically: verify the signature over the raw body first — missing
or wrong → `401`, state untouched; an already-processed event id or a version not above the
stored row → `200`, state untouched; otherwise apply, append the event, `200`. Answer within
**3 s**; never call the vendor from the handler. Apply transaction groups atomically as
documented: stage parts, apply the complete group in one local transaction; no read may ever
observe half a group. Payments keep the four statuses; reversals appear in the summary
`reversals` block and the notifier.
## 5. Approval workflow
Bearer tokens (`Authorization: Bearer <token>`) on every drafts endpoint and on `/api/events`.
Missing or unknown token → `401` (`"unauthorized"`); known token, wrong role → `403`
(`"forbidden"`). `admin` reads drafts and events and writes nothing.
| Method | Path | Role | Effect + ledger event |
|---|---|---|---|
| `POST` | `/api/drafts` | maker, checker | `{"amount_minor", "currency", "counterparty": {"name", "country"}, "note"}` → `draft.created` |
| `POST` | `/api/drafts/<id>/submit` | maker, checker | `submitted` → `draft.submitted` |
| `POST` | `/api/drafts/<id>/approve` | checker | `approved` → `draft.approved`, then SEND |
| `POST` | `/api/drafts/<id>/reject` | checker | `rejected` → `draft.rejected` |
| `GET` | `/api/drafts?state=` | any | `{"data": [...], "total": <int>}`, filtered by state |
- States `draft → submitted → approved | rejected`, `approved → sent`. A draft object carries
at least `id`, `state`, `amount_minor`, `currency`, `counterparty`, `note`, `created_at`. An
unknown draft id → 404. Invalid draft input (e.g. a non-integer `amount_minor`, a missing
field) → 400 with `field_errors`.
- **SEND:** after committing `approved`, create the vendor payment (`POST /v3/payments`) with
an `Idempotency-Key` stored with the draft before the first attempt; on 2xx append
`payment.sent`. Any retry (crash, timeout, stall) reuses the stored key: exactly one vendor
payment per approved draft. The payment returns through webhook/sync like any other.
- **Durability:** `submitted` and `approved` are durable once their 200 is sent. After a
SIGKILL right after either — including mid-send — the restart finds the state intact, the
outbox event preserved, and the send completed with the same key.
## 6. notifierd
| Method | Path | Response |
|---|---|---|
| `POST` | `/notify/events` | batch of ledger events from the relay |
| `GET` | `/health` | `{"status": "ok", "duplicate": <int>, ...}` |
| `GET` | `/notify/processed?after=<seq>` | `{"processed": [{"seq": <int>, "type": <str>}...]}` |
| `GET` | `/notify/notifications?limit=&offset=` | `{"data": [{"id", "event_seq", "kind", "message", "at"}...], "total": <int>}`, newest first |
Idempotent by ledger `seq`: a seq already in the processed set changes nothing (counted in
`duplicate`), and a mixed batch still applies its new events. The processed set and the
notifications are durable in `notifier.db` across SIGKILL; exactly-once is graded on them.
Exactly `draft.submitted`, `draft.approved`, `draft.rejected` and `reversal.created` produce
one notification each (`kind` = event type); `payment.sent` is processed but notifies nothing.
## 7. Frontend — `web/`
Four files served by ledgerd and referenced with relative paths: `web/index.html`,
`web/styles.css`, `web/app.js`, `web/viz.js` — together at most **150 KB**. Style it as a
product: your own stylesheet, a non-default font, distinct solid background colours on the
header, table and controls, and a branded header bar `#app-header` with the product name.
- **Summary:** per currency an element `.cur-total[data-currency]` showing that currency's
payment count and total in its own currency. A sync button `#sync-now` calls
`POST /api/sync`, shows an in-flight state (disabled, `data-state="syncing"`) and, on
completion, re-enables and refreshes the table and summary.
- **Table:** columns in this order: Date, Amount, Status (the status word), Counterparty,
Note. Server-paginated through `/api/payments` at most 50 rows per page (never
`/api/viz/records`), with `#prev`/`#next` and a `showing X–Y of TOTAL` readout. Clicking the
Date header toggles ascending/descending via `sort`. The first rows render within **2 s** of
page load. Rows carry `data-id` and `data-brushed`
(`"true"` while brushed); clicking a row toggles it in the brush (§8), except the Note cell.
Status badges use the 3D hex values: `settled #059669`, `pending #D97706`, `refunded #7C3AED`,
`failed #B91C1C`, distinct in computed colour.
- **Notes, optimistically:** clicking a Note cell opens an inline text input (never
`prompt()`); on confirm (Enter or a Save button) the new text paints in the cell
immediately, before the network answers, with the row at `data-state="saving"`, then
`data-state="saved"` after the 200.
- **Notifications feed `#notifications`:** reads only `/api/notifications`, polling at least
every **5 s**; entries newest first, each carrying `data-event-seq` and showing its kind and message. `data-state="live"`; `"degraded"`
while the proxy answers 502; back to `"live"` without a reload within **5 s** of the notifier
returning.
- **Drafts panel:** a token input `#role-token` (the bearer for drafts calls); `#draft-form`
with amount, currency, counterparty name and country, and note; `#draft-list` rows with
`data-draft-id` and `data-state`, click to select; `#submit-btn`, `#approve-btn`,
`#reject-btn` act on the selected draft. The full journey works through the UI: maker →
create → submit; checker → approve or reject; the feed shows the submitted/approved/rejected
notifications; the sent payment appears in the table.
- **States:** an empty database shows an empty state with a call to sync (corner D3); a
backend that is unreachable or erroring shows an error a user can act on (e.g. retry); never
a blank page. Zero console errors and unhandled rejections in every journey.
- **Dates** render human-readable in the user's locale — never raw ISO-8601 strings.
- **Money** renders in each amount's own currency with exactly its exponent's decimals and the
stored digits (`129900 EUR → €1,299.00`, `JPY → ¥129,900`, `KWD → KWD 129.900`), with a
currency symbol or code.
- **375 px** viewport: no horizontal scroll, rows still rendered.
## 8. The 3D field
Raw WebGL (no library) on `<canvas id="viz3d">`, main thread, every payment one instance on a
day × in-day-rank grid, with GPU picking, an inertial camera, culled labels, a brush linked to
the table and live SSE diffs. The grader recomputes all of this and compares it with your API,
pixels, picks and GL call stream.
### Data → scene
`GET /api/viz/records` → `{"count": N, "id": [...], "amount_minor": [...], "currency": [...],
"status": [...], "created_at": [...], "day": [...], "version": [...]}`, all length N, order
`(created_at instant ASC, id ASC)`; `day` is the server-computed Berlin date, which the
frontend uses and never recomputes.
- `n` = stable arrival index: initial records in serve order (0-based), each streamed create
appends at `n = count`. `n` never re-sorts. Picks, digest, labels and `vs7dbg` key on `n`/`id`.
- Layout basis, locked at the first non-empty render: `vs7dbg.layout()` →
`{"d0": <first day>, "D0": 96, "R0": <max in-day count at load>}`; unchanged by creates.
- `d` = day − `d0` in calendar days; `r` = in-day rank at load by `(created_at, id)`; a
streamed create takes `r` = current in-day count.
```
Δ = 1.2; x = (d − (D0 − 1)/2)·Δ; z = (r − (R0 − 1)/2)·Δ; base y = 0
a_major = amount_minor / 10^exp(currency) exp: EUR 2, USD 2, JPY 0, KWD 3
h = clamp(0.9 + 0.55·log10(a_major), 0.2, 4.2)
```
Rendered tops are measured in device pixels (±3 px, JPY and KWD included). Every payment
renders every frame (frustum culling only; no LOD or decimation outside VISUAL-CONTRACT's
inspection mode).
**Colours** (exact, ±8 per channel): status `settled (5,150,105)`, `pending (217,119,6)`,
`refunded (124,58,237)`, `failed (185,28,28)`; face shading per VISUAL-CONTRACT; brushed-dim
(brush non-empty, instance not in it) `round(0.30·c)` applied before the face factor;
background `#101828 (16,24,40)` — every non-tower pixel, no floor, grid, axes or in-canvas text.
**Scene digest** `vs7dbg.sceneDigest()` → `{count, Sh: Σh, Sh2: Σh², Sx: Σx, Sz: Σz,
Sxh: Σx·h, Szh: Σz·h, brushedCount}` over all current records, float64, rounded to 4 decimals;
graded within `max(0.5, 1e-4·|expected|)` and against the rendered pixels.
### Rendering
- Context `webgl` or `webgl2`, `{antialias: false, alpha: false}`; backing store
`clientWidth × devicePixelRatio` by `clientHeight × devicePixelRatio`. Depth testing on.
- **Draw budget:** at most **8** default-framebuffer draw calls per rendered frame (every
draw entry point and instancing extension is counted). Over a 40-move drag (pointerup + one
rAF), default-framebuffer draws ≤ `8·max(frames, 1)` and ≤ `8·(40 + 8)`, and the scene draws
at least **0.8 frames per move**.
- **Demand rendering:** draw on load, input, coast and data change only; at rest **0**
default-framebuffer draws in any 500 ms window.
- A "realloc" is any `bufferData` over 4096 bytes.
### Pick buffer
GPU truth, never a CPU raycast: an offscreen RGBA8 + depth framebuffer sized to the drawing
buffer, no MSAA, depth-tested, every instance drawn in identity colour
`idNum = n + 1`, `r = idNum & 255`, `g = (idNum >> 8) & 255`, `b = (idNum >> 16) & 255`;
0 = background. `vs7dbg.pick(sx, sy)` → `{id, index}` or null and `vs7dbg.pickPixel(sx, sy)` →
raw `[r, g, b, a]`, both at device pixel `(round(sx·DPR), Hdev − 1 − round(sy·DPR))`, and both
agree with the analytically nearest surface (occlusion in either index order). After each
invalidation (camera change or applied batch) the first pick does ≥ 1 offscreen draw and ≥ 1
offscreen `readPixels`, at most **4** offscreen draws per refresh; later picks may use the
cached readback; a pick never draws to the default framebuffer.
A pointerup within **5 px** and **300 ms** of its pointerdown is a click: on an instance it
toggles that record in the brush; on background it clears the brush.
### Camera
```
θ = yaw·π/180 φ = pitch·π/180 T = (0, 1, 0)
eye = T + distance·(cos φ·sin θ, sin φ, cos φ·cos θ)
f = normalize(T − eye) r = normalize(f × (0,1,0)) u = r × f
q = p − eye; xc = q·r; yc = q·u; zc = q·f; zc ≤ 0.5 → not projected
fovY = 50°, k = 1/tan(fovY/2), aspect = Wcss/Hcss, near 0.5, far 1000
sx = ((k/aspect)·xc/zc + 1)/2·Wcss sy = (1 − k·yc/zc)/2·Hcss (CSS px from canvas top-left)
```
Default camera: VISUAL-CONTRACT. Clamps: pitch `[5, 85]`, distance `[15, 340]`; yaw unbounded
(compared modulo 360).
- **Drag:** per pointermove, `yaw ← yaw − 0.30·Δx`, `pitch ← clamp(pitch + 0.30·Δy)`.
- **Wheel:** `distance ← clamp(distance·exp(0.0012·deltaY))`; the canvas consumes the event,
the page never scrolls.
- **Double-click:** reset to the defaults and zero all angular velocity.
- **Inertia:** at pointerup, `(vyaw, vpitch)` in deg/s = `0.30·Δpx/Δt` from the last two
moves. Then `v(t) = v0·e^(−t/τ)`, `yaw(t) = yaw0 + v0·τ·(1 − e^(−t/τ))`, τ = **0.4 s**, using
real elapsed time (a per-frame constant decay fails); stop when `|vyaw| < 2` and `|vpitch| < 2`.
Pitch clamps apply during the coast and zero `vpitch`. pointerdown and double-click cancel the
coast; wheel does not. Graded: `yaw_rest − yaw(t) = v(t)·τ` at mid-coast samples within
`max(1.0°, 0.15·|v·τ|)`; a fast flick keeps moving ≥ **3°** in the drag direction, visibly;
a release under **6 px/s** moves ≤ 0.5°; the coast settles within
`τ·ln(max(v0, 2)/2) + 0.7 s`, capped at **2.5 s**.
### Labels
Candidates: the **12** records with the highest `a_major` (ties by `id` ASC). Anchor
`A = project(x, h, z)` (top centre, live camera). Eligible iff `A` is inside the canvas and
`pick(A)` returns that instance. Each shown label is a DOM element `.viz-label[data-id]` in
`#viz-labels` over the canvas, border-box exactly **110 × 18** CSS px, top-left at
`(A.sx + 10, A.sy − 9)` ± 2 px, text containing the amount in its own currency. Consider
candidates by `a_major` DESC, `id` ASC; show one iff eligible and its rect intersects (≥ 1 px)
no already-shown label, otherwise hide it (never move it). Labels update every rendered frame
and after `vs7dbg.setCamera`.
### Linked brush
One set of record ids; `vs7dbg.brush()` returns it sorted by id; `#brush-count` shows its size.
A table row click toggles the record (row `data-brushed="true"` while in the set). While the set
is non-empty, non-members render dimmed and members keep their exact colours. An instance click
toggles the record and navigates the table to the page containing it under the current sort,
with the row `data-brushed="true"` and scrolled fully into view. A background click clears the
set and the dim. Whether a brushed record stays brushed after a streamed mutation is corner D1.
### Streaming diffs
`GET /api/stream` (SSE, concurrent subscribers) delivers every committed payment change
(including note writes) in exactly one message, each message one atomic batch:
`{"batch": <int, increasing>, "records": [{"id", "amount_minor", "currency", "status",
"created_at", "day", "version"}, ...]}`. The initial dataset comes from `/api/viz/records`.
- Applying a batch touches only the changed instances: a status change touches 1; a create
appends at `n = count`, `r` = current in-day count; nothing re-ranks or re-sorts.
- During a batch apply, uploaded bytes (`bufferData` + `bufferSubData`) ≤ `|S|·stride + 4096`
and no realloc.
- Within **250 ms** of receipt the store, digest and pixels show the change.
### `window.vs7dbg` — required, synchronous, truthful
```js
layout() // {d0, D0, R0}
sceneDigest() // {count, Sh, Sh2, Sx, Sz, Sxh, Szh, brushedCount}
camera() // {yaw, pitch, distance, vyaw, vpitch} — live values
setCamera(yaw, pitch, distance) // clamps, cancels coast, renders, re-culls labels
pick(sx, sy) // {id, index} | null
pickPixel(sx, sy) // [r, g, b, a]
brush() // ids, ascending
frames() // frames rendered since load, agrees with the counted draws
```
Each answer must agree with what the canvas shows.
## 9. `DECISIONS.md`
Decide each corner, implement it, and document it under exactly these headings, two or three
sentences each (the choice and why); the run exercises all three and an undocumented or
contradicted corner fails:
- `## D1` — does a brushed record stay brushed after a streamed mutation, or drop out?
- `## D2` — is a rejected draft terminal or resubmittable?
- `## D3` — before the first sync completes, does the table render empty-with-progress or block?
## 10. The graded run
Faults are seeded; all of them are normal operation. The vendor is down for the first seconds
of a boot. The first walk gets one dropped connection and one `500` + `Retry-After`, with
webhooks racing it: mutations to served and unserved pages, an out-of-order pair, a duplicate,
a forged signature, a mid-walk create and a two-part refund group. A later sync gets a `304`
with a mismatched generation. ledgerd is SIGKILLed mid-sync; notifierd is SIGKILLed while
ledgerd commits more events; ledgerd is SIGKILLed between an outbox commit and its delivery,
right after a submit's 200, and right after an approve's 200 with the send in flight. Every kill
is followed by a restart with the same flags.
The grader polls your API throughout and replays your event log and the notifier's processed
set against the vendor's history. At every instant: every served or applied
`(payment_id, version)` exists in the vendor's history; served versions never decrease; no
read observes half a refund group (per currency, reversal totals equal the refunded rows'
amounts); amounts never change. At quiescence: every version and status equals the vendor's,
counts and per-currency totals (reversals included) equal the vendor's ground truth (fixture,
scripted mutations and your creates), every write answered 2xx is present, and no event,
vendor payment or notification is applied twice.
# sb7.2/VISUAL-CONTRACT.md
# SB7.2 — payment towers, a legible overview and committed-update inspection
This amends `SB7-CONTRACT.md` §8. Everything there still holds (data, layout, arrival indices, raw WebGL, GPU picking, camera laws, brush, labels, SSE diffs, budgets) except the column shape, the default camera and the inspection mode below.
## Overview: a payment is a structured tower
Keep the SB7 center `(x,z)`, height `h` and base `y=0`. Render each payment as the union of four centered, axis-aligned closed solids (widths in world units, heights as fractions of `h`):
| Part | Width and depth | Vertical interval |
|---|---|---|
| Pedestal | 0.90 | 0 to 0.12h |
| Shaft | 0.54 | 0.12h to 0.90h |
| Status cap | 0.90 | 0.90h to h |
| Currency collar | EUR 0.62; USD 0.70; JPY 0.78; KWD 0.86 | 0.76h to 0.84h |
Visible rendering and the pick pass both use this union (never its bounding box): a ray through the open shoulder beside the shaft hits what lies behind it.
Colour is the SB7 status colour times a face factor, rounded per channel, brush dim applied first: top surface `y=h` 1; X-normal vertical faces 0.55; Z-normal vertical faces 0.72; every other horizontal surface 0.82. No gradients or textures. The digest still counts one payment with one `h`, and the four parts fit within the SB7 draw and upload budgets.
## Inspection: the ledger spire
Inspection renders the same real payment, at its real center, height and status, in detail on the same canvas — the only exception to SB7's no-LOD rule. Coordinates are relative to the payment center; heights are fractions of its `h`:
- Pedestal y=0..0.12h and status cap y=0.90h..h, octagonal: `abs(x)<=0.45`, `abs(z)<=0.45`, `abs(x)+abs(z)<=0.80` (clipped vertical corners).
- Recessed core shaft: width/depth 0.38, y=0.12h..0.90h.
- Four separate support ribs: width/depth 0.06, centered at x=±0.24, z=±0.24, y=0.12h..0.90h.
- Hollow currency frame: outer width EUR 0.62, USD 0.70, JPY 0.78, KWD 0.86; four rails 0.04 thick around an open square; y=0.76h..0.84h at rest. Core, ribs and background seen through the opening survive depth testing and picking.
Materials: core (42,55,73); pedestal and ribs (190,207,223); cap the status RGB; frame EUR (37,99,235), USD (6,182,212), JPY (234,88,12), KWD (147,51,234). Factor 0.55 on X-normal faces, 0.72 on Z-normal faces, 0.82 on horizontal faces except the cap's top (1); a clipped corner with normal `(nx,0,nz)` uses `(0.55*abs(nx)+0.72*abs(nz))/(abs(nx)+abs(nz))`. Round each channel. Inspection is undimmed. Visible and identity passes both use this detailed union while inspecting.
## Controls and cameras
Default overview camera: **yaw 70, pitch 50, distance 190**, target `(0,1,0)`; SB7's projection, clamps, drag, wheel and coast laws are unchanged. Double-click and **Full field** return to it.
`#inspector-controls` holds a text input `#inspect-payment` (a payment ID), **Inspect payment** `#inspect-open`, **Full field** `#field-view` and **Replay payment update** `#replay-event`. Inspection starts only from `#inspect-open`, so field clicks keep brushing. A visible legend `#tower-legend` explains the encoding.
Inspecting shows only the chosen payment's structure on `#viz3d`, camera target `(x,h/2,z)`, yaw 35, pitch 25, distance 6, vertical field of view 50°; `vs7dbg.camera()` reports the real camera and `vs7dbg.setCamera` accepts distance 6 while inspecting (it is checked at yaw 35 and 125). Entering inspection does not change the stored scene or digest and hides the full-field labels. `#inspect-details` shows `#inspect-id`, `#inspect-currency`, `#inspect-amount` (formatted in its currency), `#inspect-status` and `#inspect-version` of that payment. Full field restores target `(0,1,0)` and the default camera.
`#tower-annotations` holds four visible callouts with `data-part` `pedestal`, `shaft`, `cap`, `collar`, each naming its part and its actual width (shaft 0.38 in inspection), placed beside the tower at its part's projected height, inside the canvas, covering neither the tower nor each other. The desktop inspector canvas is at least 600 × 460 CSS px; at yaw 35 and 125 the seeded tallest payment per currency spans 40–90% of the canvas height, unclipped. Text in the legend, details, callouts and controls is at least 12 CSS px with at least 4.5:1 contrast against its composed background, unclipped and uncovered.
## Animate an actual committed payment update
When the inspected payment receives a newer committed SSE version, apply its data immediately and animate only its currency frame. With `t` the seconds since that version arrived, clamped to `[0,1]`, and `s=t²(3−2t)`, for exactly one second the frame's bottom is `(0.20 + 0.56s)h` and its top `(0.28 + 0.56s)h`, width unchanged; it then rests at `[0.76h,0.84h]` and demand rendering resumes. Pedestal, core, ribs, cap, height, amount and status never move; the camera stays put. A newer version of that payment restarts the motion; duplicate or stale versions change nothing. **Replay payment update** replays the inspected payment's latest observed update the same way, with no write. Leaving inspection cancels the animation, restores the default camera and idle rendering, and hides the callouts.
## A legible overview
At the default camera in a 1280 × 800 desktop viewport, measured on a composited screenshot with the whole canvas in view, label boxes excluded:
- Framing: pixels showing a published tower colour (status RGB × 1, 0.82, 0.72 or 0.55, ±8 per channel) span at least 60% of the canvas width and 60% of its height (0.5th to 99.5th percentile), and none lies within 4 CSS px of the canvas edge.
- Lighting: at up to 200 seeded pixels where the published geometry predicts a tower surface, the median WCAG contrast against `#101828` is at least 2.0:1.
- Status: of up to 96 seeded payments the grading schedule does not mutate, at least 64 must show a cap top covering a whole 2 × 2 block of screen pixels, and in at least 90% of those all four pixels must read as the payment's status: nearest published status colour, within RGB distance 40.
## Excellence rungs
0.12 × gate fraction × mean credit (1/.75/.5/.25): drag frames over 40 moves ≥40/32/24/12; stream visible ≤180/200/400/800 ms; burst p95 ≤150/300/600/1200 ms; optimistic note .5 painted while held, +.25×(1/.6/.3) by ≤100/250/800 ms, +.25 saved; T+X+R mean ≥.90 (else half, pro rata). Gate: 16 conditions (four journeys, clean console, 375 px, dates, scene binding, residual, no row loss, six P rungs).
## Evidence and grading
Framebuffer pixels and GPU picks are compared with independently computed geometry: the overview at the default camera, and one seeded payment per currency in inspection at yaw 35 and 125, where every part and every gap (cut corner, open frame) shows its published colour within ±8. Stream latency runs from SSE receipt to the first drawn frame showing the new status. The animation is exercised with a real note edit.
# sb7.2/STARTER.md
# Starter
The starter supplies argument parsing, the three service entrypoints, a launcher, HTTP request dispatch with raw body bytes, four static asset routes and an empty page shell, using only the Python standard library and browser JavaScript. Every file is editable or replaceable.
`create_application(args)` in each service returns an object whose every request raises `NotImplementedError`, answered as HTTP 501 `not_implemented`: unfinished work, never an acceptable final response. No database or product state exists. SSE needs a streaming response path the simple JSON transport does not have. Sync, recovery, vendor calls, storage, events, webhooks, outbox, authentication, drafts and the notifier are yours.
The frontend supplies markup, IDs, neutral layout and matrix/shader-compilation helpers (`MeridianGL` in `web/viz.js`); it draws nothing and has no fetching, formatting, interaction, WebGL context, geometry, picking, camera, labels, brush or streaming. Replace the starter notice once the product works.
# bench/browser-self-test.mjs
// Public entrant utility. It contains no scorer logic or private fixtures.
import { createRequire } from 'node:module';
import path from 'node:path';
const args = process.argv.slice(2);
const gate = args[0] === 'load';
const [url, screenshot = 'self-test.png'] = gate ? args.slice(1) : args;
if (!url || !process.env.BENCH_BROWSER_MODULE || !process.env.BENCH_BROWSER_EXECUTABLE) {
throw new Error('Usage: node browser-self-test.mjs http://127.0.0.1:PORT [screenshot.png]');
}
const { chromium } = createRequire(import.meta.url)(process.env.BENCH_BROWSER_MODULE);
const browser = await chromium.launch({
headless: true,
executablePath: process.env.BENCH_BROWSER_EXECUTABLE,
args: ['--enable-unsafe-swiftshader'],
});
try {
const page = await browser.newPage({ viewport: { width: 1440, height: 1000 } });
const errors = [];
const sources = [];
page.on('pageerror', error => { errors.push(error.message); sources.push(error.stack || ''); });
page.on('console', message => {
if (message.type() === 'error') { errors.push(message.text()); sources.push(message.location().url || ''); }
});
// Payment updates keep an SSE connection open; network-idle is not page readiness.
await page.goto(url, { waitUntil: 'domcontentloaded' });
let readinessError = null;
try { await page.locator('tbody tr').first().waitFor({ state: 'visible', timeout: 10000 }); }
catch (error) { readinessError = error.message; }
await page.screenshot({ path: path.resolve(screenshot), fullPage: true });
const renderedRowCount = await page.locator('tbody tr').evaluateAll(rows =>
rows.filter(row => row.getClientRects().length > 0 && row.querySelectorAll('td').length > 1).length);
console.log(JSON.stringify({ title: await page.title(), errors, readinessError, screenshot: path.resolve(screenshot),
renderedRowCount, consoleErrors: { count: errors.length, texts: errors, sources } }));
} finally {
await browser.close();
}
# model-prompt-appendix.md
A bundled browser is available for your own tests. Read BROWSER-TESTING.md for the runtime paths and screenshot command. It contains no private tests.
# BROWSER-TESTING.md
# Browser self-testing
Python 3, Node, Playwright and a headless Chromium browser are supplied. Start your app with its documented command, then run `node browser-self-test.mjs http://127.0.0.1:PORT screenshot.png`. The helper reports page errors and saves a screenshot. You may modify it or write your own Playwright tests. Load Playwright with `require(process.env.BENCH_BROWSER_MODULE)` and launch Chromium with `executablePath: process.env.BENCH_BROWSER_EXECUTABLE`. These paths work inside the same isolation boundary as your app.
Public input source: 8cc978d73e193310a486be9a3a131722dc171dc7. Release manifest SHA256: 7f6ef437e3558c80b3ce1bd304db396d49209506da1d1d450a363b2ea284f524.
- 2026-10-02 · Gauntlet 7.2: Gauntlet 7.2 replaces Gauntlet 7.1 as the current benchmark. Same Meridian payments task; the scorer is rebalanced so the 3D view earns points directly, overview legibility is measured, text contrast scores as interface quality, the excellence rungs are published and the public contract is shorter. Gauntlet 7.1 results stay on their own board.
Results are posted by the goose Local Edition desktop app over the site's API. The site has no submission form. The posted payload contains the score, per-tier means, per-check rows, admission evidence, run metadata, screenshots and available video — the data the run cards display. Accepted posts are live on the board immediately.
- 1.RunOpen Benchmark in Goose 3.0.3 or later. The app verifies the current catalog before Run and Retry scoring. Only the latest stable benchmark can launch; update Goose if its bundled benchmark is older. Historical results stay readable.
- 2.BuildSwarm or your selected single model builds the application in a working directory on your machine. Swarm nodes can mix local and cloud providers.
- 3.ScoreThe pinned scorer executes the build: HTTP checks, a headless-browser pass, response-time measurements. The recorded checks and score composition belong to that exact scorer version.
- 4.PublishThe app POSTs the structured result with screenshots and available graded video to the site's API. Accepted posts appear on the board immediately.