Benchmark run

3-node local fleet0.0154 on sb-7.0

qwen3.6-27b-fable-fusion-711 (3-node LM Studio fleet) · 3 nodes · scorer sb-7.0

Aug 21, 2026
Overall
0.0154
Excellent
no
Scorer
sb-7.0
Wall clock
180 min
Nodes
3
Prompt tok
4.9M
Gen tok
214.5k

Tier breakdown

A · structure & runtime0.38
B · behaviour0.00
C · vendor contract0.00
D · finesse0.13

Scoring detail

How this score was built

Per-check scorer output, as posted by the app.

The number, exactly

(0.88 × 0.0292 core + 0.12 × 0.00 gate × 0.00 excellence) × 0.6000 critical = 0.0154

Critical defects compound a multiplier on the whole score (pre-severity 0.0257):

server_runs 0.00 → ×0.60sync_completeness 0.00 → ×1.00b_money_rendered 0.00 → ×1.00b_buckets_dst 0.00 → ×1.00j_loads_data 0.00 → ×1.00j_workflow_journey 0.00 → ×1.00x_conservation_residual 0.00 → ×1.00x_no_lost_write 0.00 → ×1.00r_no_row_loss 0.00 → ×1.00r_no_dupe_effect 0.00 → ×1.00r_cache_truth 0.00 → ×1.00

The core (88% of the total) is the weighted mean of the ten measured tiers. The last 12% is the excellence slice: it unlocks in proportion to the perfection conditions below (0 of 16 met here), then pays out at the excellence tier's own measured mean. Core 0.0257 + excellence 0.0000.

j_first_use 0.00j_workflow_journey 0.00j_error_state 0.00j_empty_state 0.00console_clean nullv_responsive_375 0.00v_dates_readable 0.00t_scene_binding 0.00x_conservation_residual 0.00r_no_row_loss 0.00p_drag_frames 0.00p_idle_flatness 0.00p_stream_apply 0.00p_under_stream 0.00p_api_latency 0.00p_sync_wall 0.00
ABoot & deliverablesthe two services boot and bind, the spec-named files exist, the 150 KB asset budget holds, each service owns only its own database

a package layout

0.88

The submitted tree contains every deliverable the spec names by path — app/__main__.py, DECISIONS.md, and the four web files (index.html, styles.css, app.js, viz.js) — plus a bootable module for each of the two services.

5/6 named files, 2/2 bootable service modules

server runs

0.00

Both services boot as real processes and report healthy over HTTP within the spec's 10-second budget — the tool runs at all.

crash at boot (usage: meridian [-h] [--db-dir DB_DIR] [--ledger-port LEDGER_PORT] [--notifier-port )

a combined entrypoint

0.00

The documented single-command form — python -m app — really boots BOTH services, not just the two individual service commands the harness normally uses.

PROBE UNAVAILABLE: combined-entrypoint smoke never ran (harness phase skipped)

serves page

0.00

The backend itself serves the frontend: GET / returns the page and styles.css, app.js, viz.js come back with correct content types, as a real browser would need.

GET / -> None

a asset budget

1.00

The whole frontend fits the 150 KB source budget and ships zero external code — the budget exists so a vendored 3D library cannot, and the page must work fully offline.

38 KB of 150 KB (3/4 files), 0 external ref(s)

a db ownership

0.00

Each service owns exactly its own SQLite file under --db-dir — ledgerd's ledger.db and notifierd's notifier.db — the one-file-per-service contract.

ledger.db=False notifier.db=False

BWire behaviourwhat the API serves on the wire: sync completeness (12,288 rows), money and Berlin-DST bucketing, the append-only event ledger, error envelopes

sync completeness

0.00

After the self-driven full sync the app's local store holds every one of the vendor's 12,288 fixture payments — an incomplete sync silently loses money rows.

0/12288 payments after the self-driven full sync

b total field

0.00

GET /api/payments reports the correct collection total — a wrong total breaks every caller's paging math.

no full state to grade the total against

b row shape

0.00

A payment row on the wire carries exactly the 10 documented keys (id, amount_minor, currency, created_at, settled_at, status, version, note, counterparty_name, country) — the vendor's nested counterparty object flattened into the last two.

no rows

b chronological order

0.00

The default payments page is ordered by created_at as a parsed instant, not as a string — mixed UTC offsets make string ordering wrong.

too few rows

b summary shape

0.00

GET /api/summary carries the documented per-currency money blocks: by_currency and the reversals block, both sorted ascending by currency code — the summary is the money surface.

no summary

b money rendered

0.00

Rendered money is arithmetically correct per currency — right digits and right decimal exponent, with the zero-decimal JPY and three-decimal KWD traps weighted separately — and no cross-currency sum exists anywhere in the summary.

vacuous — nothing to lose because `sync_completeness` already failed

b buckets dst

0.00

GET /api/buckets — the 3D field's aggregate — buckets payments into Europe/Berlin calendar days per status, correct across the seeded DST transition (the data-correctness trap).

vacuous — nothing to lose because `sync_completeness` already failed

b viz records

0.00

GET /api/viz/records — the 3D field's sanctioned full fetch — serves the whole collection columnar with the server-computed Berlin day, so the frontend never recomputes days in UTC.

no /api/viz/records

b events log

0.00

The append-only event ledger is contiguous from seq 1 with the frozen type and source vocabularies — a seq gap is evidence of a lost write and is graded as one.

no events returned (status None)

b error envelope

0.00

Error responses carry the documented structured envelope — error.code and error.message plus field_errors[] with dot-and-[index] paths where validation detail is expected.

no error responses observed

b json shapes

0.00

Every sampled API response — success and error paths alike — is parseable JSON; an HTML error page mid-API breaks every client that trusted the contract.

no responses sampled

CSync disciplinehow the vendor is consumed: the 192-page walk, drop resume, Retry-After, the collection-generation rule, webhook and idempotency-key discipline

c paged walk

0.00

The first sync walks the vendor's server-fixed 64-per-page protocol to completion — all 192 pages, documented parameters only, no page fetched twice.

PROBE UNAVAILABLE: no vendor trace

c b1 drop resume

0.00

A connection dropped mid-page-stream during the first walk costs a documented resume — not a full restart of committed work, and not a hole in the collection.

PROBE UNAVAILABLE: no vendor trace

c b2 retry after

0.00

A seeded 500 with Retry-After during the walk costs exactly one documented retry after the advertised wait — never a fresh unconditional restart of committed work.

PROBE UNAVAILABLE: no vendor trace

c b5 generation 304

0.00

The collection-generation rule: a 304 whose X-Collection-Generation disagrees with the stored generation is a cache miss — drop the validator and refetch unconditionally exactly once, without looping and without serving stale data as fresh.

PROBE UNAVAILABLE: no vendor trace

c conditional resync

0.00

Later syncs are cheap: every re-sync request carries a validator (If-None-Match) except the documented unconditional cases — an unconditional full re-walk on every sync is the expensive-client defect.

PROBE UNAVAILABLE: no vendor trace

c webhook discipline

0.00

The app accepts the vendor's signed push traffic — every delivery acknowledged 2xx within budget — and bounces the one forged signature with 401, state untouched.

PROBE UNAVAILABLE: vendor trace carries no webhook delivery records (vendor_service_v3 surface missing)

c send idempotency

0.00

Approved drafts become real vendor payments through POST /v3/payments with a stored Idempotency-Key, and the kill-interrupted send's retry reuses that key — a fresh key per retry is the seeded duplicate-payment bug.

PROBE UNAVAILABLE: vendor trace carries no POST /v3/payments records

DValidation & docsinput validation, content types, client timeouts, behavior when the peer service is down, and the DECISIONS.md judgment corners

d content types

0.00

API responses declare a JSON content type and the SSE stream declares text/event-stream — a wrong content type breaks strict clients and kills EventSource.

api json 0.00, /api/stream ctype ''

d validation

0.00

Invalid input and wrong-role requests are rejected with the documented status codes — bad limit/offset/sort/status 400, unknown paths 404, missing tokens 401, wrong roles 403, self-approval 403 approval_forbidden.

matrix not exercised

d client timeouts

0.00

The app is provably resilient to an unresponsive vendor: it boots, binds and serves local data while the vendor refuses connections — one hung vendor call must never hang the tool.

no request timeout and no vendor-down evidence

d peer absence

0.00

Neither service crashes on the other's absence: with notifierd down the proxy answers 502 with the documented envelope code and ledgerd keeps running; with ledgerd killed notifierd keeps running.

PROBE UNAVAILABLE: peer-absence window never orchestrated (partition phase missing)

d decisions doc

0.50

The three deliberately unstated corners — D1 brush survival on streamed mutation, D2 rejected-draft terminality, D3 pre-first-sync table state — are decided, documented under the frozen headings in DECISIONS.md, and consistent with observed behavior.

D1=0.5, D2=0.5, D3=0.5

JJourneysreal user flows in a real browser: first use, the sync click, the maker/checker approval journey, notifications, error and empty states

j loads data

0.00

Whether the Meridian Payments Console renders any payment rows at all in a real browser, and whether the total it claims on the page matches the collection truth at probe time.

0 rows rendered, DOM claims None (want None)

j console clean

1.00

Whether normal use of the console — loading it and running a sync — produces JavaScript console errors or uncaught page errors.

0 console error(s) across load+sync

j first use

0.00

The first-visit experience as one property: real data appears quickly, the on-page total agrees with the collection truth, and the console stays clean.

ttfd=Nonems reconcile=0.0 consoleErrs=0

j sync journey

0.00

The headline interactive flow: a user clicks Sync now and the app visibly starts, indicates progress, finishes, and refreshes the view.

sync button never found

j workflow journey

0.00

The maker/checker approval workflow driven end to end through the UI alone: token in, draft created, submitted, approved, listed, notified, and the vendor-created payment landing in the payments table.

required app surface absent: workflow UI (role token / draft form / approve)

j workflow reject

0.00

The checker's other half of the workflow: rejecting a submitted draft completes through the UI, the rejected state shows in the draft list, and the rejection notification appears in the feed.

rejection never completed

j notifications feed

0.25

The notifications feed visibly degrades while notifierd is down and heals itself — without a reload — once it returns.

partition state=None, heal state=None, live in Nones

j error state

0.00

What a user sees when the backend is unreachable: a visible, actionable error state instead of a blank or silently broken page.

no error state

j empty state

0.00

What a fresh install looks like before the first sync completes: an honest empty-or-progress state rather than phantom data or a blank page.

blank page on empty db

VRendered truthwhat the page visibly shows: human dates, exponent-correct money, status badges at the frozen hexes, 375 px layout, deliberate styling

v dates readable

0.00

Whether the Date column shows humans a readable date instead of a raw machine timestamp.

no date cells rendered

v money presentation

0.00

Whether rendered amounts identify their currency — the presentation half of money; the exponent-and-digits truth is graded separately by the critical b_money_rendered.

no amount cells rendered

v status badges

0.00

Whether the four payment statuses are visually distinguishable at a glance and painted in the spec's frozen palette — the same four hexes the 3D field uses.

no status cells rendered

v responsive 375

0.00

Whether the page survives a phone-width viewport: no horizontal scrolling and real content still rendered at 375 px.

375px horizontal scroll: None

v styling

0.20

Whether the page is deliberately styled rather than browser-default: a real stylesheet, layered surface colors, a chosen font, and a branded header.

stylesheet=False, None backgrounds, font '', header=True

PPerformancethe spec's budgets, measured: frames under drag, idle flatness (demand rendering), stream apply, API latency under load, sync wall clock

p drag frames

0.00

The 3D field stays interactive under input: real frames rendered during the scripted 40-move budget drag at the full 12,288-instance count.

frames not measurable over the scripted drag

p idle flatness

0.00

Demand rendering, the frozen spec rule: at rest — no input, no coast, no pending stream batch — the scene draws nothing; a continuous rAF render loop fails by design.

no idle windows sampled

p stream apply

0.00

A live SSE batch becomes visible fast: the median time from receipt to applied — store, digest and pixels — against the spec's 250 ms budget.

no stream batch applied

p under stream

0.00

The API stays fast AND correct while the SSE stream burst is landing — read p95 under proven concurrent load.

PROBE UNAVAILABLE: under-stream measurement never ran (fire_burst surface missing)

p api latency

0.00

The read endpoints answer within their spec budgets when idle: the worst p95 across the API latency battery.

PROBE UNAVAILABLE: idle latency battery never ran

p sync wall

0.00

Wall-clock time for the self-driven first sync — the full 192-page walk, seeded faults and documented waits included — against the spec's 120 s budget.

sync #1 never completed inside the harness budget

T3D fieldthe instanced WebGL field: real context, scene math, GPU pick buffer, camera and coast physics, collision-culled labels, brush, streaming diffs

t context real

0.00

The #viz3d panel is a real, drawing WebGL surface with the pinned context attributes and a correctly sized backing store — not a styled div, an image, or an unused canvas.

ctx=0 type=None nonBg=None/None distinct=None backingOk=None

t layout basis

0.00

The locked layout basis vs7dbg.layout() reports — d0 (first day), D0 = 96 (the span), R0 (max in-day count at load) — matches the fixture truth and never moves when a streamed create arrives.

layout got={'d0': None, 'D0': None, 'R0': None} want={'d0': '2025-08-23', 'D0': 96, 'R0': 180} unmoved_after_stream=None

t scene binding

0.00

The rendered scene actually encodes the payment data: vs7dbg.sceneDigest()'s seven statistical moments over all 12,288 instanced columns match an independent recomputation from the fixture.

no data to bind

t height pixels

0.00

Column heights are true in rendered pixels — including the JPY (exponent 0) and KWD (exponent 3) instances whose heights expose a forgotten currency exponent.

no data to measure

t draw budget

0.00

The field renders 12,288 instances inside the draw budget — at most 8 default-framebuffer draw calls per rendered frame — forcing instanced draws or a merged buffer instead of per-column draws.

wrapper counted no draws over the budget window

t pick buffer

0.00

Picking is GPU truth: the offscreen pick buffer answers occlusion exactly as the depth buffer says, on click points constructed to kill CPU raycasts and last-drawn-wins shortcuts.

no instances to pick

t pick real pass

0.00

The pick buffer is real GPU work with the documented cost profile: a fresh offscreen pass after each scene invalidation, bounded at 4 offscreen draws, and never flashing ID colors on the visible canvas.

no pick-refresh counter windows

t click semantics

0.00

Canvas clicks follow the frozen §3.3 semantics: click an instance to toggle it into the brush, click it again to toggle it out, click background to clear the brush.

click semantics never exercised

t camera math

0.00

The orbit camera implements the documented math exactly: defaults yaw 30 / pitch 40 / distance 260, the 0.30 deg-per-px drag law, the exponential wheel law without page scroll, both clamps, double-click reset, and a projection that matches the printed formula.

camera math never exercised

t coast identity

0.00

Post-release inertia obeys the closed-form τ = 0.4 s decay law: at any coasting instant the remaining travel equals v(t)·τ, a slow release starts no coast, and the coast settles inside the printed budget.

no coast evidence at all

t coast reality

0.00

The coast is real motion on the canvas, not a camera() narrative: after a fast flick the scene provably keeps moving past release.

no flick-coast evidence

t labels culling

0.00

The 12 highest-amount records get screen-space labels that are collision-culled deterministically THROUGH the app's own pick buffer — exact set, exact geometry, zero overlap, never floating over an occluded instance.

no candidates to label

t brush link

0.00

One brush set links the table and the 3D field in both directions: row clicks and instance clicks toggle the same set, non-members dim to the exact 0.30 pixel rule, the count readout tracks, and background click restores full color.

no rows to brush

t stream diff

0.00

SSE batches apply as true diffs: only the changed instances upload, no buffer realloc, the digest moves by exactly the change, and the changed instance's pixels show it.

no SSE batch observed applying

t vs7dbg truth

0.00

The mandated window.vs7dbg instrumentation tells the truth: camera() agrees with the pixels, sceneDigest() with the recomputed data, frames() with the wrapper's counted draws, and pick() with pickPixel() with the analytic answer.

required app surface absent: window.vs7dbg

XConsistency ledgerthe live consistency contract: no invented states, per-key version order, monotonic reads, convergence, money conservation, dup/forgery handling

x l1 no invented states

0.00

Every (payment, version) the app ever applied or served exists in the vendor's committed history — an invented state means the app fabricated data.

nothing applied to check

x l2 per key order

0.00

Applied versions per payment strictly increase in event-log order — duplicate and stale webhook outcomes belong in the counters, never as events.

no event log to order-check

x l3 monotonic reads

0.00

The version served for a payment never decreases from one read to the next — a sync page landing after a webhook applied v+1 must not regress the row.

no read stream against a live sync

x l4 convergence

0.00

At quiescence every payment's version and status equals the vendor's final committed state, and the row count equals the vendor's — the mid-walk create present exactly once.

never converged because sync never completed

x l5 group atomicity

0.00

No read ever observes half a transaction group: the refunded payment and its reversal become visible together or not at all.

the group never applied

x m1 amount immutability

0.00

No served row ever shows an amount_minor different from the vendor-committed amount — v3 never mutates amounts, only status/note/version.

no amounts served

x m2 pair conservation

0.00

At every observed instant, per currency, the summary's reversal totals equal the refunded rows' amounts — both halves of each refund visible, or neither.

no summaries to conserve

x m3 terminal conservation

0.00

Terminal per-currency counts and totals — reversals included — equal vendor ground truth: fixture plus scripted mutations plus every payment the app created.

no terminal state

x m4 no cross currency

0.00

No field anywhere in the summary carries a cross-currency money sum — minor units are not a common denomination and summing them is wrong money.

no summary to scan

x conservation residual

0.00

CRITICAL — after every duplicate and loss is attributed, no minor units remain created or destroyed: the money conservation residual is zero in every currency.

vacuous — nothing to lose because `sync_completeness` already failed

x no lost write

0.00

CRITICAL — every mutation the app acknowledged with a 2xx is present in the final state; an acked-then-vanished write is silent data loss.

vacuous — nothing to lose because `sync_completeness` already failed

x ooo dup forged

0.00

The three webhook trust traps land correctly: the out-of-order pair keeps v+2 (the late v+1 never overwrites), the forged-signature event changes nothing, and no duplicate delivery applies twice.

the deliveries never applied

RResiliencethe seeded fault schedule: SIGKILL mid-sync, vendor-down boot, outbox atomicity, partition catch-up, exactly-once effects, workflow durability

r b3 sigkill resync

0.00

ledgerd SIGKILLed mid-sync restarts, converges, and duplicates nothing — a kill mid-walk costs a clean cursor restart, never dupes or holes.

PROBE UNAVAILABLE: B3 kill window never orchestrated

r b4 vendor down boot

0.00

The app boots with the vendor down: binds within 10 seconds anyway, serves local data, does not crash, and completes the first sync unprompted once the vendor returns.

{'armed': True, 'refusal_secs': 5, 'bound_in_10s': False, 'served_while_down': False, 'no_crash': False}

r b6 outbox atomic

0.00

A SIGKILL between an outbox commit and its delivery loses nothing and doubles nothing — the exact window where commit-then-POST and POST-then-commit both fail.

PROBE UNAVAILABLE: B6 window (kill between outbox commit and relay) never arranged

r b7 partition

0.00

A notifier partition degrades visibly and heals in order: writes never block, /api/outbox/status reports down with growing pending, the feed shows degraded, and catch-up is in seq order with the UI live again within 5 seconds of heal.

PROBE UNAVAILABLE: B7 partition never orchestrated

r notifier exactly once

0.00

The notifier's durable processed set proves exactly-once: every outbox-crossing event processed, no seq twice, surviving kills.

PROBE UNAVAILABLE: notifier processed set never read

r notification multiset

0.00

Selective materialization is exact: draft.submitted, draft.approved, draft.rejected and reversal.created each produce exactly one notification row; payment.sent produces none.

PROBE UNAVAILABLE: notifier notifications endpoint never read

r no row loss

0.00

CRITICAL — no committed row goes missing after any seeded kill: the row count after every SIGKILL+restart is at least the count before it.

vacuous — nothing to lose because `sync_completeness` already failed

r no dupe effect

0.00

CRITICAL — no ledger effect applies twice: one vendor payment per approved draft, no duplicate notification rows, no (type, payment, version) event effect repeated.

vacuous — nothing to lose because `sync_completeness` already failed

r cache truth

0.00

CRITICAL — the lying-304 trap: a 304 whose X-Collection-Generation disagrees with the stored generation is a cache miss, and serving the stale cache as fresh is graded data loss.

vacuous — nothing to lose because `sync_completeness` already failed

r workflow durability

0.00

CRITICAL — submitted and approved are durable the moment their 200 is written: a SIGKILL immediately after either, including mid-send, must find the state intact after restart.

PROBE UNAVAILABLE: workflow F3 exercise never ran

EExcellencethe last 12%: drag frames, stream-apply latency, latency under load, optimistic paint, and mastery of the T+X+R mechanisms

e frames under drag

0.00

Excellence-grade fluidity: frames rendered during the scripted 40-move drag at the full 12,288-instance count, with proof the frames actually drew.

frames not measurable or no draw calls

e stream apply latency

0.00

Excellence-grade streaming: the median SSE batch apply time against rungs 2.5× tighter than the P-tier budget.

no stream batch applied

e under load latency

0.00

The read API's p95 under the proven stream-burst load, graded again in the excellence slice.

PROBE UNAVAILABLE: under-stream measurement never ran

e optimistic paint

0.00

The workflow UI paints optimistically: the submitted state appears while the write is provably still on the wire, then really saves.

no optimistic-paint exercise in the flow emit

e mastery

0.00

Excellence includes the mechanisms, not only the surface: mastery of the entire 3D contract (T), consistency-and-money invariants (X) and resilience (R) tiers together.

T+X+R mean 0.000

Token rates

Measured by the engine itself, one record per completed model call: prefill rate is prompt tokens over time-to-first-token, generation rate is completion tokens over the decode window. Medians per node.

NodeCallsPrompt tokGen tokPrefill tok/sGen tok/s
gabee631,334,32948,6311414.28.7
mihai571,295,61654,9731172.510.4
workhorse1592,229,139110,880331.613.6
fleet2794,859,084214,484364.512.9

Run details

Model
qwen3.6-27b-fable-fusion-711 (3-node LM Studio fleet)

Fleet nodes

NodeModel
workhorseworkhorse-qwen3.6-27b-fable-fusion-711
mihaimihai-qwen3.6-27b-fable-fusion-711
gabeegabee-qwen3.6-27b-fable-fusion-711

Notes

The local fleet’s best sb-7 run so far (its 4th attempt), still an honest FLOOR: the app now owns every advertised boot entry and validates its inputs exactly as the spec documents, but neither service ever starts listening — the process runs and never binds — so every runtime tier stays at zero and the score reflects structure and effort, not served behaviour. Two engine defects this run exposed (missing package entries in earlier plans; the boot probe handing the app an empty tokens file and repairing a defect that did not exist) are fixed in the engine for the next attempt. Fleet: 3× qwen3.6-27b on LM Studio (workhorse/mihai/gabee). Token rates in this entry are engine-measured per call (median prefill 364.5 tok/s, decode 12.9 tok/s).