Forge 2.0 Benchmark
See which AI builds more for less. Run the same test in Goose.
Models upgrade the same Jira app in two independent builds, each tested and scored, with recorded inference spend shown for both.
Forge leaderboard
Forge results of one version, ranked together. Not comparable with the Gauntlet board: a different task and a different scorer.
Hover a row or use its info button for build details and recorded spend.
Cloud baselines show the median and range of independent builds within the same verified task, engine and captured comparison cohort. The reasoning policy requests medium effort where supported, or enables advertised reasoning at provider defaults when medium cannot be selected or reasoning metadata is absent, subject to each route's native qualification. Each model's literal request is recorded; effective reasoning is not assumed identical across providers. One build is provisional. Each build keeps its own score and evidence. Compare models within the same comparison group.
| Model / run | Score / median |
|---|---|
1. jev-routertypesafe/jev-router2 independent buildsComparison group 1Range 0.6398–0.8523Build 1: 0.6398Build 2: 0.8523 | 0.7460 |
2. Opus 5.5anthropic/claude-opus-5.52 independent buildsComparison group 2Range 0.6196–0.7294Build 1: 0.7294Build 2: 0.6196 | 0.6745 |
3. GPT 6.1 Solopenai/gpt-6.1-sol2 independent buildsComparison group 2Range 0.5371–0.6495Build 1: 0.6495Build 2: 0.5371 | 0.5933 |
4. qwen3.8-max-primeqwen/qwen3.8-max-prime2 independent buildsComparison group 3Range 0.5015–0.6515Build 1: 0.6515Build 2: 0.5015 | 0.5765 |
5. Sonnet 5.5anthropic/claude-sonnet-5.52 independent buildsComparison group 3Range 0.5485–0.5899Build 1: 0.5485Build 2: 0.5899 | 0.5692 |
6. gemini-4-argongemini-4-argon2 independent buildsComparison group 1Range 0.2943–0.7926Build 1: 0.7926Build 2: 0.2943 | 0.5434 |
7. MiMo v2.6 Flashxiaomi/mimo-v2.6-flash2 independent buildsComparison group 2Range 0.5028–0.5538Build 1: 0.5538Build 2: 0.5028 | 0.5283 |
8. ember-1fireworks/ember-12 independent buildsComparison group 3Range 0.4710–0.5312Build 1: 0.4710Build 2: 0.5312 | 0.5011 |
9. glm-5.3-primez-ai/glm-5.3-prime2 independent buildsComparison group 3Range 0.4046–0.4557Build 1: 0.4046Build 2: 0.4557 | 0.4302 |
10. qwen3.8-flashqwen/qwen3.8-flash2 independent buildsComparison group 4Range 0.3164–0.4990Build 1: 0.3164Build 2: 0.4990 | 0.4077 |
11. Gemini 3.8 Flashgoogle/gemini-3.8-flash2 independent buildsComparison group 3Range 0.2999–0.4958Build 1: 0.4958Build 2: 0.2999 | 0.3979 |
12. Qwen 3.8 Omni Flashqwen/qwen3.8-omni-flash2 independent buildsComparison group 3Range 0.1659–0.5104Build 1: 0.5104Build 2: 0.1659 | 0.3381 |
13. DeepSeek 4.1 Flashdeepseek/deepseek-v4.1-flash2 independent buildsComparison group 2Range 0.2893–0.3790Build 1: 0.2893Build 2: 0.3790 | 0.3342 |
14. muse-wondermeta/muse-wonder2 independent buildsComparison group 5Range 0.2366–0.4203Build 1: 0.4203Build 2: 0.2366 | 0.3285 |
15. grok-4.7x-ai/grok-4.72 independent buildsComparison group 3Range 0.1815–0.4682Build 1: 0.4682Build 2: 0.1815 | 0.3248 |
16. muse-spark-1.3meta/muse-spark-1.32 independent buildsComparison group 3Range 0.2202–0.2968Build 1: 0.2202Build 2: 0.2968 | 0.2585 |
17. step-5-previewstepfun/step-5-preview2 independent buildsComparison group 3Range 0.2356–0.2749Build 1: 0.2749Build 2: 0.2356 | 0.2552 |
18. GPT 6 Lunaopenai/gpt-6-luna2 independent buildsComparison group 3Range 0.2197–0.2580Build 1: 0.2197Build 2: 0.2580 | 0.2389 |
19. hy4-previewtencent/hy4-preview2 independent buildsComparison group 3Range 0.0555–0.3901Build 1: 0.0555Build 2: 0.3901 | 0.2228 |
20. glm-5.3-flashxz-ai/glm-5.3-flashx2 independent buildsComparison group 3Range 0.0714–0.3337Build 1: 0.3337Build 2: 0.0714 | 0.2026 |
21. ling-3.1-flashinclusionai/ling-3.1-flash2 independent buildsComparison group 3Range 0.1582–0.1930Build 1: 0.1930Build 2: 0.1582 | 0.1756 |
22. GLM 5.3z-ai/glm-5.32 independent buildsComparison group 3Range 0.1453–0.1850Build 1: 0.1850Build 2: 0.1453 | 0.1652 |
23. Qwen3.8-27B 8-bit · local Goose · single modelMihai-LeanZero/Qwen3.8-27B-Atlassian-Q8-base-mlxBuild 1: 0.1457 | 0.1457 |
24. Qwen3.8-27B 8-bit · local Goose · single modelMihai-LeanZero/Qwen3.8-27B-Atlassian-Q8-base-mlxBuild 1: 0.1342 | 0.1342 |
25. claude-haiku-5.5anthropic/claude-haiku-5.52 independent buildsComparison group 3Range 0.0610–0.0632Build 1: 0.0632Build 2: 0.0610 | 0.0621 |
26. Qwen 3.8 27Bqwen/qwen3.8-27b2 independent buildsComparison group 3Range 0.0014–0.0617Build 1: 0.0014Build 2: 0.0617 | 0.0316 |
27. GLM 5.3 Flashz-ai/glm-5.3-flash2 independent buildsComparison group 3Range 0.0014–0.0617Build 1: 0.0014Build 2: 0.0617 | 0.0316 |
Invalidated Forge 2.0 run archive
Previous results are excluded from the ranking. Their original evidence remains available below.
About the Forge benchmark
The task is Scope Ledger 2, a brownfield upgrade. The model inherits v1 of the Scope Ledger, the Jira app Forge 1.0 asks for, in use for days on a much larger seeded Jira site with its data, and ships v2: everything v1 does, plus a live migration of its ledger, dosing against a points quota, work that survives invocation time limits, a world that changes while the app runs, a UI Kit admin panel, signed CI deployment events, a custom field, Forge LLM used safely and Custom UI that boots fast, with a ledger-backed native JQL function and shared precomputation refresh. Nothing is deployed: the harness upgrades the site's v1 to the model's app and grades it by running it.
What the model receives
v1's source and the layout of its stored data, the pinned Forge packages (UI Kit's included), the Forge manifest schema and Jira's OpenAPI files as reference, the benchmark's own rate model as RATE-MODEL.json, and offline tools: Forge's own linter, a Forge runtime and Custom UI host, a UI Kit host, a CI event sender and a seeded development Jira site. There is no internet, nothing to install and no reference v2.
The budget
300 model calls, twice Forge 1.0's. The harness stops the session at the budget and scores whatever exists, so empty or partial work still gets a score. Polishing beyond the scored behaviours earns nothing.
What makes it hard
Simulated production conditions, stated as guarantees: about 1,000 issues against a 2,400-point hourly quota (the benchmark's own rate model: one installation's share of the app's 65,000-point hourly pool), invocations killed at their time limit, consumer invocations that run concurrently (up to 3 at once, two deliveries of one event included) against KVS queries that may miss writes from the last 5 virtual seconds, sprints that close, issues that move to another board, an estimation field that changes, deleted issues and revoked permissions, admin actions that must be refused to everyone but admins, and Forge LLM tool calls that must be checked before they act. The three scoring sites use seeds other than the development site's, each with its own schedule of changes.
v2 adds four modules to v1's: a jira:adminPage in UI Kit (render: native, @forge/react), a static webtrigger for CI events and a read-only jira:customField, plus jira:jqlFunction with scopeChanged(sprintId). The ledger moves to a new Forge KVS entity, scope-ledger, indexed by sprint and change time.
Each evidence page records one independent build: one model, inside the Goose Swarm desktop app on the poster's own machine, within the 300 calls. The app posts every test's outcome and the scoring pictures and browser recording. Cloud baseline entries combine LeanZero's matched builds of one model, ranked by their median with the range, build count and each build's evidence retained. A single build is provisional; community entries show their individual score.
LeanZero's cloud comparison policy keeps the 300-call budget and the app's global medium reasoning setting. The actual OpenRouter request uses medium effort where supported. Routes that advertise reasoning use enabled: true at provider defaults when medium cannot be selected or reasoning metadata is absent, subject to their own native qualification. Each model's literal request is recorded. The global setting and request JSON do not establish identical effective reasoning across providers.
- Lint, bundle and load. Forge's own linter runs offline, as in Forge 1.0, now with the checks Forge's servers apply at deploy (an entity index with two range attributes is refused); every manifest function is bundled and loaded in the pinned Forge runtime, and the UI Kit admin page is bundled the way forge deploy does.
- The upgrade. A seeded site of about 1,000 issues holds v1 and its data; the harness upgrades it to the model's app and runs it for 6 virtual hours: issue updates delivered more than once, out of order or never, and concurrently (up to 3 consumer invocations at once), KVS queries that may miss writes from the last 5 virtual seconds, scheduled runs, quota, burst and per-issue 429s, invocation time limits, a schedule of world changes and CI deployment events. Time is virtual, so nothing is graded on wall-clock time.
- Security from every seat. Admin resolvers are called as an admin, as a non-admin and with a forged payload; the web trigger receives valid, unsigned, tampered, stale and replayed events; people lose browse permission while the app runs.
- Every surface. The widget and the sprint action are driven in a browser in light and dark, the not-started sprint in light, all recorded on video, with the Forge LLM cases; the UI Kit admin panel is graded from its component tree; the widget and the sprint action are counted in invokes and bytes before their first data paint, the admin page in invokes before its first render.
- Native JQL and shared caches. Actual Jira searches execute the app's expansion, including IN, NOT IN and composed queries. Repeated searches and different viewers share one precomputation; event and reconciliation changes must refresh it through Jira's real update endpoint. The oracle checks exact issue membership and each viewer's permissions independently.
- Three scoring sites. The build is graded on the run's own seed and two seeds derived from it, never the development seed; each test keeps its worst site, the excellence measurements their mean. The model is told that the scoring seed differs from the development site's; it is not told how many scoring sites there are or that each test keeps its worst.
No model takes part in scoring: every number is checked against an oracle, the benchmark's own calculation of the correct values from the run's own seeded data.
Earned score = tests earned × critical defects × reliability; the multipliers are described below. Tests earned = 0.15 × v1 + the v2 requirements, each at its share of the score. v1 is this run's score on the tests carried over from Forge 1.0, by Forge 1.0's weighting and before any critical defect or cap: 0.88 × the weighted tiers L to A + 0.12 × the excellence gate × the mean of the 3 excellence measurements. The E shown for a run (its tier bar and its E column) is already gate × mean. Under a cap, the final score is min(earned, cap − 0.05 × (1 − earned)): always below the cap, and closer to it the more the run earned. With no cap it is the earned score. Avoiding a cap adds no points.
The v2 requirements
| Tier | What it measures | Share of the score |
|---|---|---|
| R1Migration | Every v1 row moves to the new scope-ledger entity exactly once, with its change id and time, while events keep flowing and across invocation limits, within 2 virtual hours of the upgrade, with its progress in the admin panel; v1's entity stays intact. | 12% |
| R2Dosing | On a site of about 1,000 issues, background work stays inside its share of the 2,400-point hour, no person-facing request meets a quota 429, and Retry-After and RateLimit-Reason (quota, burst, per-issue write) decide every retry; the backfill still completes. | 13% |
| R3Time limits | Work longer than one invocation continues in the next, no function waits past its limit (a long Retry-After is handed back, not waited out), and a killed invocation leaves no duplicate. | 8% |
| R4Changing world | Sprints close, issues move to another board's sprint, a board's estimation field changes, issues are deleted and a person loses browse permission while the app runs; the ledger and every view come out right. | 10% |
| R5Admin panel | A jira:adminPage built in UI Kit with the contract's controls, and every admin resolver checking Jira's ADMINISTER permission on the server, never trusting the payload. | 10% |
| R6CI web trigger | Signed deployment events: a missing or wrong signature, a tampered body or a stale timestamp is refused with no effect, a replay applies once, a valid event records the deployment, and the secret is never disclosed. | 8% |
| R7Custom field | The read-only scope-status field holds committed or added +points for an issue in an active sprint, removed for one that has left an active sprint since that sprint started, and nothing for any other issue, written in bulk through Jira's app field-value API and fresh within the virtual hour. | 5% |
| R8Forge LLM | Tool calls validated against the viewer's sprint scope, back-off on a 429 without Retry-After, answers with no finish reason refused, a 10-minute cache, and the admin's kill switch and daily token budget enforced on the server. | 4% |
| R9Boot budget | The widget and the sprint action each make at most one invoke and load at most 150 KB before their first data paint, with no external origin; the admin page renders after one invoke. | 5% |
| R10Native JQL | scopeChanged(sprintId) is a jira:jqlFunction returning native issue predicates from the ledger. Jira shares its precomputed expansion across users, applies each viewer's browse permissions, and keeps the same cached mapping fresh through the documented precomputation update API after changes and reconciliation. Empty, future and closed sprints, IN and NOT IN, and invalid arguments are checked by running actual native JQL. | 10% |
The tiers carried over from Forge 1.0
| Tier | What it measures | Share of the score |
|---|---|---|
| LLint and bundles | Forge's own linter reports no errors (warnings cost points), every manifest function bundles and loads, packages resolve at the kit's pins, the manifest rules the linter misses, and only the scopes the app's calls need. | 1.06% |
| KPlatform currency | Current modules and APIs: dashboards:widget with its edit API, the rovo:skill and rovo:mcp wiring, a Forge LLM model the platform lists as active, no /rest/api/3/search, no @forge/api storage, a nodejs22.x or nodejs24.x runtime, the consumer shape, the declared KVS entity, and deploy readiness: a manifest and permissions real Forge would accept and no code predicted to fail there. | 1.32% |
| TEvent pipeline | Issue updates flow trigger to queue to consumer: one ledger row per change under duplicate, reordered, dropped and concurrent deliveries, multi-sprint changelog values, re-estimates followed, Retry-After honoured, no user context in background work. | 2.11% |
| RReconcile | The first scheduled run starts the backfill of every change since each active sprint started, removals included; later runs heal what the event stream missed and a run with nothing new writes nothing; they page through results in each endpoint's own style, never retry before Retry-After and finish inside the module timeout, queue continuations included. | 1.85% |
| SStorage | Forge KVS with the storage scope, ledger reads through the declared entity index in change-time order, no KVS limit errors. | 1.06% |
| BResolvers and permissions | Every invoked resolver exists and answers structured errors, nobody sees changes to issues they cannot browse (not on screen, not in an LLM prompt, not in a Realtime payload), the hidden-change count is right, and each click posts exactly one ADF comment as the viewer. | 1.58% |
| UUI function | The dashboard widget's numbers and chart, its board choice saved through the host, live updates through Forge Realtime, the sprint action's ledger table, sorting, issue links, comment flow, the Forge LLM explanation, close and not-started states. | 2.11% |
| VVisual | Atlassian design tokens with 4.5:1 contrast in light and dark, a painted surface, no Content Security Policy violations or console errors, and the widget readable at 380 px wide. | 1.06% |
| ARovo | The get-sprint-scope action returns exact numbers and the visible changes for the invoking person, returns errors instead of throwing, and the skill's SKILL.md tells the agent how to use it. | 1.06% |
| EExcellence | Few Jira requests in the first scheduled run and per relevant change in triggers and consumers, and surfaces that paint after one round trip — paid only in proportion to the excellence gate. | 1.8% |
Score caps
As the task text states them to the model:
| Maximum | Applies when |
|---|---|
| 0.30 | None of the v2 surfaces exists: no jira:adminPage, no webtrigger and no scope-ledger entity. |
| 0.499 | Lint errors, or manifest functions that do not bundle and load. |
Critical defects
Each class multiplies the score by 0.6, at most once however many tests observe it, and only on an observed defect: never for a surface the app lacks or a test whose stimulus never happened.
- v1 rows lost or corrupted by the migration.
- A web-trigger write without a valid signature.
- An admin action by a non-admin that succeeded, or the CI secret disclosed.
- A hidden issue's data shown to a person who cannot browse it.
- A duplicate side effect: two or more comments for one click, duplicate ledger rows, or a replayed CI event applied twice.
Reliability
Failed tests also multiply the whole score, by group. Each group of tests multiplies the score by 1 − 0.10 × the shortfall of its worst test, so by 0.90 when that test fails completely and in proportion when it fails partly. The groups are the 10 v2 requirements and the 9 tiers carried over from Forge 1.0, L to A, 19 in all; the excellence tier E never multiplies. A group's other failed tests are already counted by its worst one, and a fault that fails tests in two groups counts in each. Together the groups never take the score below 0.25 of what the tests earned; a critical defect still multiplies on top.
A group counts every test at the score it has, a missing surface's tests and tests nothing exercised included, at 0. It leaves out a failed test whose critical defect fired, which is priced as a critical defect instead, the 2 diagnostic tests that carry no weight, and a test the harness could not run (a result with one is not published).
The task text states it to the model as:
Failed tests also multiply the whole score, by group: each v2 requirement above is a group of tests and v1's behaviours make nine more. A group multiplies the score by 1 − 0.10 × the shortfall of its worst test, so one whose worst test fails completely multiplies it by 0.90. The excellence share never multiplies, a defect from the list below is priced there instead, and together the groups never take the score below a quarter of what the tests earned.
A stated guarantee that fails barely moves a weighted mean of 105 tests; this multiplier is what makes it count.
Excellence
Inside v1's 15%, the last 12% of v1 (1.8% of the total) rewards few Jira requests in the first scheduled run and per relevant change in triggers and consumers, and surfaces that paint after one round trip, paid in proportion to the excellence gate. The gate is the average of 18 conditions: the event rows, the healing, the widget numbers, the ledger table, the comment and each of the 10 weighted platform-currency tests at the score it earned, and no console or page errors, no Content Security Policy violations or failed asset requests, and no lint warnings, each as pass or fail. Lint warnings cost points but never cap.
Scoring rules synced at goose ad60c7050; the Forge 2.0 scorer is final and results are open.
Five documents in one block, one after another, as source text: Markdown, and JSON for the rate model. The line with a file name that opens each document is added by this page. # spec-build-forge2.md is the task, sent to the model as its prompt. # FORGE2-CONTRACT.md (the app contract), # STARTER.md (the starter guide), # RATE-MODEL.json (the rate model) and # BROWSER-TESTING.md (the browser-testing notes) are files in the model's workspace.
R2 clarification: the frozen contract overstates its workload guarantee. Person-facing demand can exceed the default 720-point reserve. The 70% background ceiling is a maximum; background work must leave enough quota for person-facing requests.
# spec-build-forge2.md
# forge-2.0 — Scope Ledger 2 (Atlassian Forge)
Ship v2 of the Scope Ledger Forge app in this workspace. The workspace holds v1: a working app, installed on a large
Jira Cloud site together with its data. `FORGE2-CONTRACT.md` is the contract and `RATE-MODEL.json` the rate model it
binds. `STARTER.md` describes the workspace (v1's code and data layout), the installed packages, the reference
material and the offline dev tools (Forge runtime, dev Jira site, UI Kit host, CI sender, linter). There is no internet
and nothing to install. No reference v2 is provided.
You have a budget of 300 model calls. The harness stops the session at the budget and scores whatever exists; plan to
implement, test, and finish well inside it — polishing beyond the scored behaviours earns nothing.
## Definition of done
Done means each of these works when the harness upgrades the site's v1 to your app and runs it for 6 virtual hours on
its own seeded site; nothing else is scored:
1. `npm run lint` reports no errors and no warnings; every manifest function bundles and loads, and so does the admin
page.
2. v1 keeps working throughout: backfill and reconciliation, exactly one ledger row per change under concurrent
delivery, the widget, the sprint action, Rovo, privacy and the Custom UI policy (§1–§8, §18).
3. Every v1 row moves to `scope-ledger` exactly once, while events flow and across invocation limits, within 2 virtual
hours, with its progress in the admin panel (§9).
4. Background work stays inside its share of the 2,400-point hour, no person-facing request meets a quota 429, and
Retry-After and `RateLimit-Reason` decide every retry (§10, `RATE-MODEL.json`).
5. Work longer than one invocation continues in the next, no function waits past its limit, and a killed invocation
leaves no duplicate (§11).
6. Closed sprints, moves to another board's sprint, estimation-field switches, deleted issues and lost permissions
come out right (§12).
7. The UI Kit admin panel works, and every admin resolver checks `ADMINISTER` on the server (§13).
8. The CI web trigger verifies signature, timestamp and replay, and records deployments (§14).
9. The `scope-status` field holds the right value for every issue, fresh each virtual hour (§15).
10. Forge LLM: validated tool calls, 429 back-off, incomplete answers refused, the cache, the kill switch and the token
budget (§16).
11. The widget and the sprint action boot within budget, and the admin page with one invoke (§17).
12. Ledger-backed `scopeChanged` JQL search works for IN/NOT IN, cached expansions stay fresh, shared
precomputations respect final viewer permissions, and invalid inputs return errors (§19).
## Score
Tests earn the score. v1's behaviours (§1–§8) are 0.15 of it; the v2 requirements share the rest: migration
0.12, dosing 0.13, time limits 0.08, the changing world 0.10, the admin panel 0.10, the CI web trigger 0.08, the custom
field 0.05, Forge LLM 0.04, the boot budget 0.05, ledger-backed JQL 0.10. Inside v1's share, a small excellence share rewards few Jira
requests (in the first scheduled run, and in triggers and consumers per relevant change) and surfaces that paint
after one round trip. Lint warnings cost points but never cap.
Failed tests also multiply the whole score, by group: each v2 requirement above is a group of tests and v1's behaviours
make nine more. A group multiplies the score by 1 − 0.10 × the shortfall of its worst test, so one whose worst test
fails completely multiplies it by 0.90. The excellence share never multiplies, a defect from the list below is priced
there instead, and together the groups never take the score below a quarter of what the tests earned.
Conditions also cap it:
- Lint errors, or manifest functions that do not bundle and load: maximum 0.499.
- None of the v2 surfaces exists (no `jira:adminPage`, no `webtrigger` and no `scope-ledger` entity): maximum 0.30.
Each of these defects, when observed, multiplies the score by 0.6, at most once per item of this list:
- v1 rows lost or corrupted by the migration;
- a web-trigger write without a valid signature;
- an admin action by a non-admin that succeeded, or the CI secret disclosed;
- a hidden issue's data shown to a person who cannot browse it;
- a duplicate side effect: two or more comments for one click, or duplicate ledger rows, or a replayed CI event applied
twice.
Time is virtual: nothing is graded on wall-clock time. The harness keeps screenshots of every Custom UI surface and the
rendered tree of the admin page.
## Completion handoff
Work only with this workspace, its packages, the reference material and the dev tools. When the definition of done
holds, stop your temporary processes and return a final summary of files, checks and remaining limitations; that
response hands the app to the external scorer, which starts after your session ends. Do not wait for or poll grader
results.
# FORGE2-CONTRACT.md
# Scope Ledger 2 — Forge app contract (forge-2.0)
Scope Ledger v1, in use for days on a large Jira Cloud site, shows agile coaches what entered a sprint after it
started, who added it and its story points. This workspace is v1's source (`STARTER.md`). Ship v2: everything v1 does
(§1–§8, which keep their rules unless a later section changes them) plus the guarantees of §9–§18.
The harness upgrades the installation from v1 to your v2 over v1's stored data (virtual hour 0), runs it for 6
virtual hours on a seeded site (events, queues, scheduled runs, every surface as different people, Rovo, CI, admin
actions, a changing world) and grades it by running it. Nothing is deployed.
**Scale**: 3 projects; 4 scrum boards, 2 estimating with one field and 2 with another (field ids vary per seed); 6
active sprints, 2 future, 6 closed; about 1,000 issues, about 300 of them in active sprints; about 200 relevant changes
and about 800 irrelevant issue updates over the 6 scored hours.
## 1. The numbers
For each **active** sprint S, with `startDate` from the Jira Software sprint API:
- A **change** is a Sprint-field changelog entry created after `startDate` that puts an issue into S (`added`) or
takes it out of S (`removed`). Its id is the changelog id, its time the changelog `created`, its author the
changelog author. A change is keyed by changelog id + sprint: one entry moving an issue between two sprints is a
`removed` in one and an `added` in the other.
- **committed** = sum of current estimates of issues that were in S at `startDate`.
- **added** = sum of current estimates of issues in S now that were not in S at `startDate`.
- **removed** = sum of current estimates of issues in S at some time after `startDate` and not in S now.
- **creep** = `100 × added / committed`, rounded half away from zero to one decimal and always written with that
decimal and `%` (`20.0%`); `—` when committed is 0.
- An issue's **estimate** for S is the value of the estimation field S's board (the sprint's `originBoardId`) uses
now; a change's **points** are the issue's current value of the field that board used at the time of the change
(§9). No value counts as 0, and a deleted issue has no value. "After `startDate`" is strictly after.
- Points are written as plain decimals (`34.5`, `0`), no thousands separators.
Background work (§10) sees every issue. What a **person** sees in the sprint action and the Rovo action lists only
changes to issues that person can browse, plus a count of the changes hidden from them; the totals above are team
totals and are the same for everyone. Changes of deleted issues are listed to nobody and counted as hidden for nobody.
## 2. Modules
| module | requirement |
|---|---|
| `trigger` | on `avi:jira:updated:issue`; hands relevant work to a queue and returns (no trigger writes storage) |
| `consumer` | Forge async events queue consumer; performs the ledger writes |
| `scheduledTrigger` | `interval: hour`; backfill and reconciliation (§3) |
| `dashboards:widget` | Custom UI view + `edit` (§4); the legacy `jira:dashboardGadget` earns nothing |
| `jira:sprintAction` | Custom UI modal (§5) |
| `action` | Rovo action `get-sprint-scope` (§6) |
| `rovo:skill` | `skills/sprint-scope-analyst/`, depends on `get-sprint-scope` (§6) |
| `rovo:agent` | lists the skill |
| `rovo:mcp` | one module, `name` at most 30 characters, exposing `get-sprint-scope` |
| `llm` | Forge LLM (`model: [claude]`), for the sprint action's explanation (§5, §16) |
| `jira:adminPage` | the admin panel in UI Kit: `render: native`, `@forge/react` (§13) |
| `webtrigger` | static (`response.type: static`); receives CI deployment events (§14) |
| `jira:customField` | key `scope-status`, `type: string`, `readOnly: true` (§15) |
Resolvers use `@forge/resolver`, return a value and never throw: a failure (a Jira error, storage, Forge LLM, a
refused permission) returns a value describing it, e.g. `{ "error": "…" }`, and the surface shows it.
Storage is Forge KVS: the v2 ledger lives in the custom entity `scope-ledger` (§9), read back through its index.
Request only the scopes your calls need: per call, the OAuth2 scopes the shipped OpenAPI lists for it (the classic
scope where one exists, else the whole granular set). The linter does not see every call. A UI Kit widget or sprint
action earns nothing.
## 3. Backend behaviour
- After the upgrade the harness runs your scheduled triggers at virtual hour 0, before any update, then once per
virtual hour. The first run starts the backfill of every change since each active sprint started that the ledger
lacks, including changes of issues that have since left every sprint; by the first virtual hour mark `scope-ledger`
holds them all (v1's own rows: by §9's 2-hour mark).
- Issue updates keep the ledger current: sprint changes add ledger rows; estimate changes move the numbers. Updates
that touch neither do no Jira or queue work (storage reads are fine).
- Product events can arrive more than once and out of order, and some never arrive. Forge runs consumer invocations
concurrently, including two deliveries of the same event; here up to 3 run at once. Key reads (get) and transaction
conditions are strongly consistent; queries are eventually consistent and may miss writes made in the last 5
virtual seconds. Whatever the delivery history, and under concurrent delivery, the ledger holds **exactly one row
per change**, one click or double click posts one comment (§5) and one CI event records one deployment (§14). Each
row records `source`: `event` or `reconcile`, the path that recorded it first (event work may also record other
changes of the issue it reads); a migrated row keeps v1's. Later scheduled runs record what the event stream
missed; a run with nothing new writes nothing.
- Background work uses `asApp()`. Whatever shows a person issue data (keys, authors, change lists, explanations) shows
only what that person can browse at the time of the request — read as them (`asUser()`), or as the app with an
explicit permission check for them. Comments are posted as the person. Team totals come from the ledger.
## 4. Dashboard widget
**Edit** (`edit.resource`): one element per scrum board, `[data-testid="board-option"]` with `data-board-id`,
clickable; the selected one carries `aria-pressed="true"`. The choice reaches the dashboard through the dashboards
widget edit API (`@forge/dashboards-bridge`). The dashboard's own Save calls your `onProductSave` handler with the last
`updateConfig` value (the stored config if none was sent) and stores what it returns (`null` stores nothing); with no
handler registered it stores the last `updateConfig` value. Reopening edit shows the stored board selected.
**View** (`resource`), root `[data-testid="scope-widget"]`. Its board comes from the widget configuration in its
context (two widgets on one dashboard can show different boards):
- No stored board: `[data-testid="needs-config"]`, nothing else.
- Otherwise one `[data-testid="sprint"][data-sprint-id="<id>"]` per active sprint of that board, ordered by
`startDate` (ties by sprint id), each holding `[data-metric="committed"]`, `[data-metric="added"]`,
`[data-metric="removed"]` and `[data-metric="creep"]` whose text is the §1 number.
- A bar chart: one `<svg data-testid="chart">`; per sprint and per series one
`<rect data-sprint-id data-series="committed|added|removed">`; every bar on one linear scale from 0 (rendered
height proportional to its number within 1 px).
- At 380 px wide nothing scrolls horizontally and no number is clipped or truncated (long names may end in an
ellipsis).
- Live: after ledger rows are written, an open widget shows the new numbers without a reload, through Forge Realtime
(`@forge/realtime` in the backend, the bridge's `realtime` in the widget) — no polling. No virtual time passes
between a publish and the widget's re-read: announce writes once they are 5 virtual seconds old (§3), or read them by
key. A `publishGlobal` reaches the `subscribeGlobal` subscriptions on its channel; a `publish` reaches `subscribe`
subscriptions of the same module context and is accepted only from resolvers (never from background work, §10).
Channel names are yours; realtime tokens are optional, but an event reaches a subscription only when neither has a
token or both have tokens with the same claims. Global channels reach every user of the app, so payloads carry sprint
ids only.
## 5. Sprint action (modal)
The sprint comes from the module context; the harness opens every sprint active at the upgrade (one closed since
shows its ledger as at its close, §12) and a future one. A sprint that has not started shows
`[data-testid="not-started"]` (and optionally close) and nothing else. Otherwise:
- Team totals as in §4 (`[data-metric]`, same four), and `[data-testid="hidden-count"]` = number of this sprint's
changes hidden from the viewer.
- `table[data-testid="ledger"]`, headers `th[data-col]` for `issue`, `points`, `kind`, `by`, `at`, `source`,
`deployed`; one `tr[data-change-id="<changelog id>"]` per visible change with `td[data-col=…]` cells: issue key,
the change's points, `added`/`removed`, author display name, a `<time datetime>` holding an ISO-8601 instant with
offset (compared as instants), `event`/`reconcile`, and one `[data-env="<env>"]` element with the text
`Deployed to <env>` per environment the issue was deployed to (§14; empty when none).
- Default order: `at` ascending, equal times by changelog id ascending (ids compare as numbers). Clicking
`th[data-col="at"]` toggles between that order and its exact reverse, starting with ascending when another sort was
active. The active header carries `aria-sort`.
- The issue key opens the issue (`/browse/<KEY>`) through the Forge router.
- Clicking a row selects it (`aria-selected="true"`). `[data-testid="post-summary"]` posts one comment on the selected
change's issue, authored by the viewer, in Atlassian Document Format, naming the issue key, the sprint name and the
sprint's creep; then a success flag. One click, or a double click, posts exactly one comment. A failure shows an
error flag and leaves the modal working. With a `Comment group` set (§13), only members of that Jira group may post:
anyone else gets the error flag and no comment is posted.
- `[data-testid="explain"]` asks Forge LLM (`@forge/llm`) to explain the sprint's creep, using any model that
`list()` reports `active` (`chat()` or `stream()`). The model must answer through one tool, `report_scope`,
arguments `{ "summary": string, "changeIds": string[] }` (force it with `tool_choice`); send it only what the viewer
may see. `[data-testid="explanation"]` shows the summary only if it holds no digits (otherwise your own sentence
with the ledger's numbers), plus one `[data-change-id]` element per returned id that is a visible change of this
sprint (others dropped). Each click sends exactly one Forge LLM request unless it meets a 429 (§16): a refusal (no
tool call), malformed arguments or another LLM error is not retried; it shows an error flag and leaves the modal
working.
- `[data-testid="close"]` closes the modal.
## 6. Rovo
`action` key `get-sprint-scope`, `actionVerb: GET`, one required string input `sprintId`. For the invoking person it
returns a JSON object:
```
{ "sprintId": "41", "sprintName": "…", "committed": 34, "added": 8, "removed": 3,
"creepPercent": 23.5, "hiddenChanges": 1,
"changes": [ { "changeId": "…", "issueKey": "OPS-12", "kind": "added", "points": 5,
"at": "<ISO-8601 UTC>", "by": "<display name>" } ] }
```
`creepPercent` is rounded as creep (§1) and `null` when committed is 0; `changeId` is the changelog id, `points` the
change's points; `changes` are the visible ones in table order (§5); `at` is compared as an instant. A sprint closed
since the upgrade answers as at its close (§12). An unknown or missing `sprintId` returns
`{ "error": "<message>" }` and does not throw.
`skills/sprint-scope-analyst/SKILL.md`: YAML frontmatter `name` = the directory name (1–64 characters: lowercase
letters, digits, single hyphens, none leading or trailing), `description` (50–1,024 characters: what it does, when to
use it), `allowed-tools` (space separated, with `get-sprint-scope`); then at most 500 lines of Markdown on when and how
to call `get-sprint-scope`, what `sprintId` is, how to read the result and what to do with an error.
## 7. Custom UI
Every Custom UI surface calls `view.theme.enable()` and is styled with Atlassian design tokens (`var(--ds-…)`): text
with `--ds-text*` (links may use `--ds-link*`), contrast at least 4.5:1 in light and in dark (disabled controls
exempt). The page behind your surface is unpainted: paint your own background with a `--ds-surface*` token. The
browser console stays free of errors. Surfaces render inside the default Forge Custom UI content security policy:
inline `<script>` in `index.html` only as static content (the platform hashes it), no inline event handlers or `eval`,
no `<style>` elements or `style` attributes in markup, no external scripts, styles or fonts, assets referenced
relatively. Declaring `unsafe-inline` in `permissions.content.styles` is allowed; script relaxations count as unneeded
permissions.
## 8. What the harness does differently from production
- Custom UI resources are served exactly as committed, uncompressed (you build `static/<name>/build/`); a UI Kit
resource's source and `src/` are bundled the way `forge deploy` does (the classic JSX transform: `import React` in
every `.jsx` file).
- Every invocation runs in a fresh Node process of the Forge runtime, on §11's virtual clock.
- A pushed event's `concurrency` key and limit apply across all queues in this installation. A shared key
serializes work when its limit is 1; unkeyed consumers may run concurrently. The harness uses up to 3 workers.
- Trigger `filter.expression` is not evaluated: the handler receives every issue-updated event.
- A Custom UI page's own `requestJira` costs no points (as on Forge) but counts in its endpoint's burst bucket
(§10).
- The scoring site's seed differs from the dev site's: ids, keys, custom field ids, users, groups, sprint names and
dates.
- The widget runs at a layout the harness chooses; the admin page in its UI Kit host (no pixels are graded).
## 9. The v2 ledger and the live migration
v1 keeps its ledger in entity `scope-change` (`manifest.yml`; row layout in `STARTER.md`): on the sites, the changes
of each active sprint's first 2 days.
v2 keeps its ledger in a NEW entity `scope-ledger`:
- Attributes (at least): `sprintId` string, `at` float, `changeId` string, `kind` string, `issueId` string,
`issueKey` string, `estimate` float, `boardId` string, `estimateField` string, `deleted` boolean, `deployedEnvs`
string; index `by-sprint`, partition `[sprintId]`, range `[at]`. A named index takes exactly ONE range attribute:
Forge refuses more at lint and at deploy. Add attributes and indexes as needed; row keys are yours.
- Per row: `at` the change time in epoch milliseconds; `kind` `added` or `removed`; the change's author (account id
or display name) and `source` (§3); `estimate` the issue's points when the row is written, in the estimation field
the sprint's board used at the change (no value = 0);
`estimateField` that field's id; `boardId` that board's id; `deleted` true once the issue is deleted (§12);
`deployedEnvs` the environments the issue was deployed to (§14), comma-separated, each once (empty when none).
Once written, a row's `estimate`, `estimateField` and `boardId` never change.
Migration guarantees (graded):
- Every `scope-change` row appears in `scope-ledger` exactly once: the same change (changelog id + sprint) with its
original `at`. Its kind, issue, author and `source` stay as v1 recorded them (§3, §5).
- `scope-change` stays declared exactly as in v1's manifest, and its rows are never changed or deleted.
- It runs while events keep flowing: a change v1 recorded that arrives again (event or reconcile) is still one row.
- It survives being cut off by time limits: a killed or retried invocation resumes where it stopped and leaves no
duplicate.
- It is complete within the first 2 virtual hours after the upgrade, and surfaces show correct numbers while it runs.
- The admin panel's `Migration` text reads `Migrated <n> of <total> v1 rows` — `<total>` the rows v1 left in
`scope-change` at the upgrade, `<n>` how many of them are in `scope-ledger` — and also contains `complete` once all
are.
Storage answers with Forge's real errors: `FAIL_IF_EXISTS` on an existing key → 409 `KEY_CONFLICT`; a failed
transaction condition → 400 `CONDITIONAL_CHECK_FAILED`; a transaction of more than 25 operations → 422
`UNPROCESSABLE_ENTITY`; an undeclared entity → 404 `SCHEMA_NOT_FOUND`.
## 10. Points, bursts and dosing
`RATE-MODEL.json` is the rate model both sites enforce on every Jira request (binding):
- **Quota:** 2,400 points per installation per virtual hour, a clock hour of the virtual clock (reset at minute :00,
not at the upgrade's minute), nothing carried over; a hard wall: once spent, every request the quota charges (a
page's own `requestJira` never is, §8) is answered 429 until the reset. It is the app's fair share of its
65,000-point Tier 1 pool, shared by every tenant of the app: in a busy hour the others can bring the wall forward.
- **Background** = product event triggers, async event consumers, scheduled triggers, the web trigger.
**Person-facing** = resolvers invoked from a UI surface (Custom UI or UI Kit) and the Rovo action. Every Jira
request counts, `asApp` and `asUser` alike.
- **Costs:** read operations cost a base point plus one point for each top-level domain object returned.
Identity/access operations charge two points per returned object. Single-object reads include that object; lists
count each returned item, not nested fields. `POST` search, bulkfetch and precomputation search are reads too.
Mutating operations cost one point regardless of payload size. `RATE-MODEL.json` lists the response collections.
- **Bursts:** per endpoint (method + path template), measured in requests, independently of quota points. Default
GET/POST allowance is 100 requests per second; PUT/DELETE 50; issue GET 150; changelog GET 200. The harness uses
a one-second token buffer, since Jira's production buffer capacity is not public.
Per-issue writes: at most 20 in 2 seconds AND 100 in 30 seconds, including bulk field-value writes.
- **Responses:** `X-RateLimit-Limit` and `X-RateLimit-Remaining` describe the endpoint request bucket, never hourly
points. `RateLimit-Policy: "global-app-quota";q=2400;w=3600` (or `"tenant-app-quota"`) and
`RateLimit: "global-app-quota";t=<seconds to reset>` describe the hourly policy. When fewer than 20% of the
quota remain, `RateLimit` also includes `r=<points remaining>` and `X-RateLimit-NearLimit: true`.
A 429 has body `{"errorMessages":["Rate limit exceeded"]}`, `Retry-After: <seconds>`,
`X-RateLimit-Reset: <ISO UTC>` and `RateLimit-Reason`: `jira-quota-global-based` or `jira-quota-tenant-based`
(seconds to the next virtual hour), `jira-burst-based`, or `jira-per-issue-on-write`.
Guarantees (graded on the site's own accounting):
- Background work never takes more than `Background share (%)` of any virtual hour's quota (default 70 % = 1,680
points). Until fewer than 480 points remain no response shows the spend, so the app keeps its own count.
- No person-facing request is ever answered with a quota 429. The harness's own person-facing traffic in any hour stays
below the share your background leaves free.
- The backfill and every reconciliation finish correctly inside that budget (§3).
- Retry-After is honoured, and the reason decides the reaction: `jira-quota-global-based` or `jira-quota-tenant-based` → no background request at
all until the next virtual hour; `jira-burst-based` → slow that endpoint only (nothing to it before Retry-After);
`jira-per-issue-on-write` → delay writes to that issue only (on a bulk field-value request it names no issue:
resend the request after its Retry-After). Each request sent before its Retry-After passed counts against you,
whichever invocation sends it.
## 11. Virtual time and invocation limits
The harness runs on a virtual clock. Inside an invocation `Date.now()` and `new Date()` read it, and
`getAppContext().invocationRemainingTimeInMillis()` (`@forge/api`) reports the virtual time left. Each request a
function makes advances its invocation's clock: a GET 120 ms, a search page 300 ms, a `bulkfetch` 600 ms, a write
200 ms; a wait inside a function (`setTimeout`, sleep) counts at face value. Platform calls too: a storage read
120 ms, a storage write or transaction 200 ms, a queue push, Forge LLM call or Realtime publish 200 ms. CPU is free.
| invocation | limit |
|---|---|
| UI resolver (Custom UI or UI Kit) | 25 s |
| trigger (product or lifecycle event) | 25 s |
| async event consumer, scheduled trigger | 55 s by default, up to 900 s with `timeoutSeconds` |
| web trigger, Rovo action | 55 s |
Exceeding the limit kills the invocation: it returns no result, and what it already wrote stays written. A consumer
that returns an `InvocationError` (`retryAfter` ≤ 900 s) is redelivered after its `retryAfter`, one that throws or is
killed after 1, 2, 4 and 8 virtual minutes, then every 15, for 24 virtual hours: at least once and in any order.
Scheduled triggers run once per virtual hour and are not retried: a failed or killed run waits for the next. Product
event triggers are retried up to 4 times (after 1, 2, 4 and 8 virtual minutes, or the `retryAfter` they asked for).
A push to a consumer with `timeoutSeconds` above 55 carries at most 100 KB (413), and continuation chains stop at
depth 1,000 (405).
Guarantees (graded):
- No invocation is killed: each function watches its time left and hands the rest to the next invocation in time.
- Work that cannot finish in one invocation continues in the next (a queue continuation, the next scheduled run).
- A Retry-After longer than the time left is never waited out inside the function: a consumer returns an
`InvocationError` with `retryAfter`; a scheduled run stops and resumes from a later run.
- A killed invocation leaves no partial duplicate: whatever completes its work completes it exactly once.
## 12. A world that changes
While v2 runs, the world changes as below. The dev site exercises every one of these at least once; the scoring
schedule is private. Guarantees (graded):
- **A sprint closes:** its ledger is final (no change after the close is recorded for it; its numbers and points
stay as at the close) and it leaves the widget, which shows active sprints only.
- **An issue moves to another board's sprint:** recorded as `removed` from the old sprint and `added` to the new one;
the `added` row and the new sprint's numbers use the NEW board's estimation field.
- **A board's estimation field changes:** changes after the switch use the new field; earlier rows keep their
`estimate`.
- **An issue is deleted:** its rows stay as history with `deleted: true`; it no longer counts in current scope (§1).
- **A person loses browse permission:** their next request shows none of the rows of the issues they can no longer
browse (no stale cache).
Events: a deletion sends `avi:jira:deleted:issue`; a sprint close (unfinished issues go to the backlog, no changelog
entry), a board's field switch and a permission change send none.
## 13. The admin panel (UI Kit)
The harness finds the admin page's controls (§2) by these visible labels, exactly:
| label | control |
|---|---|
| `Background share (%)` | number 10–90, default 70 (§10) |
| `AI explanations enabled` | toggle, default on (§16) |
| `Daily AI token budget` | number, default 200000 (§16) |
| `Comment group` | text: a Jira group name; empty = whoever can browse the issue may post the summary (§5) |
| `Save settings` | button: saves the four settings above |
| `Rotate CI secret` | button: makes a new CI secret (§14) and shows it once |
| `Migration` | read-only text (§9) |
| `Recent admin changes` | table, columns `when`, `who`, `what`: the last 20 admin changes, newest first |
A label binds through a `Label` whose `labelFor` is the control's `id`, the control's own `label`, `title` or
`caption`, or a button's text, never through a `Heading` or `Text` beside it.
- `Save settings` stores the four values together. A value outside its range (share not a whole number 10–90, budget
not a whole number ≥ 0) is refused: nothing is saved and the panel shows an error. A Save that changes nothing adds
no row.
- After `Rotate CI secret` the panel shows `CI secret: <secret>` once; from then on (later renders, reloads, every
resolver answer) only `CI secret: ••••<last4>`. A secret is at least 32 ASCII letters, digits or `-_+/=`
(hex or base64 will do).
- `Recent admin changes`: one row per Save that changed something and per rotation; `when` an ISO-8601 instant, `who`
the acting person (account id or display name), `what` names each changed setting by its label (a rotation reads
`Rotate CI secret`) and never holds a secret.
**Authorization (graded).** Any user who can load a surface can invoke every resolver attached to it, directly and
with any payload. So every resolver the admin page uses, reads included, checks on the server that the caller holds
Jira's global `ADMINISTER` permission: `GET /rest/api/3/mypermissions?permissions=ADMINISTER` as the user. The caller's
identity always comes from the resolver's `context` (`context.accountId`), never from the payload. A caller without
ADMINISTER gets `{ "error": "…" }` and nothing changes: no setting, no secret, no audit row, nothing disclosed.
## 14. CI deployment events (web trigger)
The web trigger declares `response.type: static` and its outputs (`key`, `statusCode`, `contentType`, `body`) in the
manifest; your function returns `{ "outputKey": "<key>" }`. CI posts deployment events in Forge's documented
web-trigger request shape: `body` is the raw string; `headers` maps each header name to an array of strings, and
header names may arrive in any letter case.
- Body: JSON `{"eventId": string, "sentAt": unix seconds, "environment": "staging" | "production", "issueKeys": [string, …]}`.
- Headers: `X-LZ-Timestamp: <unix seconds>` and
`X-LZ-Signature: sha256=<hex HMAC-SHA256(secret, "<timestamp>.<raw body>")>`: `<timestamp>` is that header's value,
`<raw body>` the body string exactly as received, the key the secret's UTF-8 bytes.
- Checks, in this order: the signature (compared with `crypto.timingSafeEqual`), then the timestamp (reject when
|now − timestamp| > 300 s), then the `eventId`. Answers and effects (graded):
| request | answer | effect |
|---|---|---|
| no signature, a wrong one, a body changed after signing, or a timestamp more than 300 s from now | `401` | none |
| valid, with an `eventId` already accepted (also when both arrive at once) | `200` | none |
| valid, new `eventId` | `202` | the deployment is recorded |
Recorded: every `scope-ledger` row of each referenced issue carries the environment in `deployedEnvs` (rows written
later for that issue carry it too), and the sprint action's `deployed` cell shows it (§5). An `eventId` is applied
at most once, ever. Before any rotation there is no secret and every request gets `401`. The secret comes only from
the admin panel's `Rotate CI secret`, is stored only with `kvs.setSecret` (never with `kvs.set`, in an entity, a log
line, or any resolver answer but that rotation's), and no resolver returns it except as `••••<last4>`.
## 15. The `scope-status` custom field
The app writes the field's values (§2) as the app (`asApp`), in bulk, through the app field-value API
(`POST /rest/api/3/app/field/value` or `PUT /rest/api/3/app/field/{fieldIdOrKey}/value`, §10). The field's id differs
per site: find it with `GET /rest/api/3/field`, where it is the custom field whose `name` is your module's `name`.
Value per issue (graded), from the facts of §1:
- in an active sprint S now: `committed` if it was in S at S's `startDate`, else `added +<points>` with the points
of the change that last added it to S, as in §1 (`added +5`, `added +2.5`, `added +0`);
- in no active sprint now, but in a currently active sprint S at some time after S's `startDate`: `removed`;
- any other issue: empty (no value), including the issues of a sprint once it closes.
Fresh within the same virtual hour as the change: the harness compares every issue's value with the site's state at
the end of each scored hour, except an issue whose change that hour sent no event (a lost event, a sprint close, a
field switch) or met a spent quota (compared an hour later) and a deleted issue (never compared).
## 16. Forge LLM, properly
§5's explanation rules stay. In addition (graded):
- **Model output is untrusted** (issue text can carry instructions): validate every tool call against the viewer's
sprint scope. Only `report_scope` is accepted, only its `changeIds` that are visible changes of this sprint for
this viewer are shown, and no tool call makes the app do anything else.
- **429:** Forge LLM answers 429 (body `{"code":"TOO_MANY_REQUESTS"}`) without a Retry-After. Back off between
attempts (never retry at once), at most 3 attempts per virtual minute, then show the error flag.
- **Incomplete answers:** a response or stream whose final choice has no `finish_reason` is a failure: show the error
flag, never its content.
- **Cache:** an identical explanation request (same sprint, same content sent to the model for what this viewer may
see) within 10 virtual minutes of a successful answer is served from cache, with no new Forge LLM call. Nobody is
served an explanation built from changes they cannot see.
- **Admin controls, enforced server-side:** with `AI explanations enabled` off, or once the virtual day's tokens (the
`usage.total_tokens` each Forge LLM response reports) reach `Daily AI token budget`, the explain resolver makes no
Forge LLM call, even when invoked directly, and the modal shows the error flag. A virtual day is a UTC calendar
day of the virtual clock.
## 17. Boot budget (Custom UI)
Counted, never timed (graded):
- The widget view and the sprint action each make at most 1 `invoke` before their first data paint, load at most 150
KB (153,600 bytes) of JavaScript and CSS before it, and never request another origin. First data paint: the first
moment the surface shows its `[data-metric]` numbers, or `[data-testid="needs-config"]` (widget) or
`[data-testid="not-started"]` (sprint action). Bytes: your own `.js` and `.css` files as served, uncompressed; the
platform's injected scripts and the Atlassian design-token stylesheets do not count.
- The admin page makes at most 1 `invoke` before its first render that shows the stored settings.
## 18. v1 keeps working
Everything in §1–§8 holds throughout the 6 scored hours. The harness reads every surface at several points of the
scored window, from the first virtual hour mark on, and compares it with the site's state at that point.
## 19. Ledger-backed JQL search
Add a `jira:jqlFunction` named `scopeChanged` with one required string argument `sprintId`, return type `issue`,
and operators `in` and `not in`. A user can search `key in scopeChanged("<sprintId>")`: it returns each
non-deleted issue that has an added or removed ledger change for that sprint, once each. Committed issues with no
change are excluded. The complement operator returns all browse-visible issues outside that set. Active and closed
sprints retain their history; a future sprint has no changes. Missing, unknown or non-numeric arguments return a
clear error, not an unconstrained query.
The invocation receives `{precomputationId, clause: {field, operator, functionName, arguments}}` and returns
`{jql: "<native JQL fragment>"}` or `{error: "<message>"}`. Validate arguments and never interpolate arbitrary input.
Fragments contain at most 1,000 right-hand-side values and no ORDER BY. The function runs as the app, with a 25-second
invocation limit. Precomputations are shared by argument/clause across users. Return the complete app-visible set;
Jira applies each user's browse permissions when evaluating the expanded native query.
Jira caches an expansion and does not invoke the function again when its ledger changes. Refresh relevant stored
precomputations after migration, new changes, deletion and reconciliation, within the same freshness guarantees as
the ledger/custom field. A closed sprint's final expansion stays valid. Use the documented APIs:
`GET /rest/api/3/jql/function/computation` (paginated `values`) and
`POST /rest/api/3/jql/function/computation` with `{values:[{id, value:"<new native fragment>"}]}`.
The cache lasts 7 days since last use. An error is cached only when `storeErrorAsPrecomputation: true` is returned.
Dev-site search requests exercise the same shared precomputation cache as scoring; declaration without functional
search and refresh earns nothing.
# STARTER.md
# Starter and offline tools
There is no internet in this workspace. Everything you can use is already here.
**Workspace: Scope Ledger v1**, the app your v2 replaces, exactly as it runs on the site today. Every file is
editable.
- `manifest.yml`: v1's modules (issue trigger, queue consumer, hourly scheduled trigger, dashboards widget with
`edit`, sprint action, Rovo action, skill, agent and MCP server, Forge LLM), its scopes and its storage entities.
- `src/`: the backend; `index.js` exports every handler. `config.js` finds the Sprint field, the scrum boards, their
active sprints and each board's estimation field; `sync.js` is the event path, the backfill and the reconciliation;
`ledger.js` the storage and the contract's §1 totals; `jira.js` the Jira requests and their 429 handling; `views.js` what each
surface and the Rovo action get; `explain.js` the Forge LLM explanation; `realtime.js` the live updates;
`numbers.js` the formatting.
- `static/widget`, `static/widget-edit`, `static/sprint`: the Custom UI surfaces (React sources in `src/`, the
committed build in `build/`); `static/shared/` holds what they share. `npm run build` rebuilds all three with
esbuild (`build.mjs`).
- `skills/sprint-scope-analyst/SKILL.md`: the Rovo skill.
**v1's data layout.** The sites' storage is v1's, as of the upgrade:
- entity `scope-change` (declared in `manifest.yml`): v1's ledger, one row per change, key `<changeId>:<sprintId>`,
value `{ sprintId, changeId, at, created, issueId, issueKey, kind, authorId, authorName, source }`: `at` is the
change time in epoch milliseconds, `created` the same instant as ISO-8601, `kind` `added`/`removed`, `source`
`event`/`reconcile`. Index `by-sprint`: partition `sprintId`, range `at`. It holds every change of each active
sprint's first 2 days and nothing later.
- v1 also writes the KV key `config` (the Sprint field id and the active sprints, see `src/config.js`) and the entity
`sprint-issue` (membership and estimate per sprint and issue). They are v1's working state: do not count on
finding them.
**Installed packages** (their typings and sources are your API reference): `@forge/api` 8.2.0, `@forge/kvs` 2.0.7,
`@forge/events` 3.0.7, `@forge/resolver` 2.0.0, `@forge/bridge` 7.1.0, `@forge/dashboards-bridge` 2.0.0,
`@forge/hooks` 2.0.0, `@forge/llm` 1.0.7, `@forge/realtime` 1.0.1, `@forge/react` 12.3.0 (UI Kit), `react` and
`react-dom` 18.3.1, `esbuild` 0.28.2. `package.json` pins v1's packages; `node_modules/` already holds all of these and
nothing else can be installed.
**Reference material** under `$FORGE_KIT` (read-only): `schema/manifest-schema.json` (the Forge manifest schema),
`openapi/jira.json` and `openapi/jsw.json` (Jira Cloud REST and Jira Software REST; the other files in `openapi/` are
the linter's offline copies). These are single-line JSON files of several MB: query them with `node` or `grep -o`,
never print them whole.
**Dev site.** A seeded Jira Cloud site answers at `$FORGE_SITE_URL` for the dev tools below, with the same shape as
the scoring sites (contract, "Scale") and another seed. It behaves like Jira Cloud — REST v3 and Jira Software REST,
including the bulk endpoints, their pagination and errors, and the rate model of `RATE-MODEL.json` on every request;
dates are ISO-8601 strings as the OpenAPI types them. JQL: fields `project`, `key`, `sprint`, `updated`, `created`,
`status`, `statusCategory`, `issuetype`, `labels`, `assignee`, `reporter`, `cf[id]`; operators
`= != in not in > >= < <= is is not ~`, `AND OR NOT`, parentheses, `ORDER BY`; functions `openSprints()`,
`closedSprints()`, `futureSprints()`, `currentUser()`, `now()`, `startOfDay()`; your declared `scopeChanged()`
function and its shared precomputation lifecycle (contract §19); relative dates like `-14d`. Its update
stream also carries the world changes of contract §12, each kind at least once. Invocations run on its virtual clock
with the contract's limits (§11) and with the scoring site's concurrency and staleness (§3): up to 3 consumer
invocations run at once, subject to installation-wide queue concurrency keys and limits, two deliveries of the same event included, and a KVS query (`kvs.query` or an entity index
query) misses writes younger than 5 virtual seconds, while `get` and transaction conditions always see current state.
Forge LLM answers with a scripted model. You never call the site directly; your app reaches it through the Forge
runtime. `users` lists the dev users: the first line is the default viewer; it marks who holds Jira's `ADMINISTER`
permission and names the issue the default viewer may not comment on (Jira answers that comment with a 400 "you do not
have the permission to comment", no ADD_COMMENTS), as on the scoring site.
**Tools** (`npm run lint`, `npm run build` and `node $FORGE_KIT/bin/forge-dev.cjs <command>`; `forge-dev` with no
command prints every flag):
| command | does |
|---|---|
| `npm run lint` | Forge's own client-side linter, offline, plus the checks Forge's servers apply at deploy (a named index takes exactly one range attribute; index names 3–50 allowed characters; at most 20 entities; a supported Node runtime). It is staged: fix and rerun until it reports no errors. |
| `invoke <functionKey> [--module <key>] [--resolver <key>] [--payload <file>] [--as <accountId>]` | runs a manifest function in the Forge runtime against the dev site with the event shape of the module that references it (`--resolver` calls that resolver key with `--payload`); prints the result, logs and every Jira, storage and queue call. `--as` makes the invocation user-led. |
| `events [--limit N]` | delivers the dev site's next N updates to your trigger(s) and drains the queues, consumers running concurrently as on the scoring site: a redelivery, or a second change of one issue, that arrives before the first one's queue work ran is delivered with it (past N) |
| `scheduled <moduleKey>` | runs a scheduled trigger once, then drains the queues the same way |
| `serve <moduleKey> [--edit] [--sprint <id>] [--config <json>] [--theme light\|dark] [--as <accountId>]` | serves a Custom UI module with the Forge bridge and the dashboard host emulated, prints its URL and runs until stopped (start it in the background); its log prints every bridge call the page makes. In a served edit surface, `window.__forgeHost.save()` performs the dashboard's Save. |
| `uikit <moduleKey>` | renders a UI Kit (`render: native`) module with the real `@forge/react` reconciler, its resolver calls going to the dev site, and prints the tree it produced |
| `ci send` | signs a deployment event the way CI does (contract §14) with the secret you give it, posts it to your web trigger and prints the answer |
| `llm` / `realtime` | the dev site's Forge LLM (model list, scripted answers, every call) / every Realtime publish and subscription |
| `kvs` / `users` / `reset` | dump stored keys and entities / list dev users / return the dev storage, queues and update stream to the upgrade instant (v1's rows in place) |
Test the native JQL path with `node $FORGE_KIT/bin/forge-dev.cjs jql 'issue in scopeChanged("123")' [--as app|<accountId>]` (replace the sprint id with one from your dev site). It searches one page of up to 100 issues as the default viewer, or the specified identity, invoking your installed `jira:jqlFunction` when its shared precomputation is absent. Cached clauses are reused across identities, and Jira filters the expanded result for the querying user. The command prints native function calls and saves the full search response to `.forge-dev/last-result.json`; malformed queries return an error.
Screenshot a served Custom UI surface with the bundled browser (`BROWSER-TESTING.md`). Build Custom UI into
`static/<name>/build/` (an `index.html` plus assets) with the installed esbuild; the harness never builds it for you.
A UI Kit resource is different: point its `path` at the source file (for example `src/frontend/admin.jsx`) and the
harness bundles it the way `forge deploy` does.
# RATE-MODEL.json
{
"name": "forge-2.0 rate model",
"about": "Current published Jira rate-limit cost classes and request/write-window semantics, adapted to a disclosed normalized 2,400-point virtual installation. Bucket capacities and virtual reset policy are benchmark assumptions; they are not production Forge entitlements.",
"quota": {
"pointsPerHour": 2400,
"scope": "per installation, per virtual hour: a clock hour of the virtual clock (minute :00 to :00), not counted from the upgrade or from a scheduled run",
"reset": "at the top of each virtual hour (minute :00); nothing carries over",
"wall": "hard: once the hour's points are spent, every request the quota charges is answered 429 until the reset (a Custom UI page's own requestJira is never charged)",
"why": "the app's fair share of its 65,000-point Tier 1 hourly pool, which every tenant of the app shares (about 27 tenants)"
},
"background": {
"maxSharePercent": 70,
"maxPointsPerHour": 1680,
"is": [
"product event triggers",
"async event consumers",
"scheduled triggers",
"web triggers"
],
"rule": "background requests may use at most 70 % of any virtual hour's quota (the admin setting Background share (%) moves this limit, 10-90)"
},
"personFacing": {
"is": [
"resolvers invoked from a UI surface (Custom UI or UI Kit)",
"the Rovo action"
],
"rule": "a person-facing request is never refused for quota"
},
"costs": [
{
"request": "read operations, including POST search, bulkfetch and precomputation search",
"points": "1 + number of returned top-level domain records (identity/access records cost 2 each)",
"collections": [
"issues",
"values",
"issueChangeLogs",
"comments",
"histories",
"users",
"groups",
"projects",
"precomputations"
],
"singleObject": "counts as one domain record; nested fields/changes do not add records"
},
{
"request": "mutating POST, PUT, PATCH or DELETE",
"points": 1,
"maxFieldUpdatesPerRequest": 200
}
],
"burst": {
"bucket": "method + path template; one request token per call, independent of point cost",
"requestsPerSecond": {
"GET": 100,
"POST": 100,
"PUT": 50,
"PATCH": 50,
"DELETE": 50
},
"overrides": {
"GET /rest/api/3/issue/{issueIdOrKey}": 150,
"GET /rest/api/3/issue/{issueIdOrKey}/changelog": 200
},
"capacitySeconds": 1,
"capacityNote": "Jira publishes request rates, not its buffer capacity. This normalized harness starts with one second of request allowance."
},
"perIssueWrites": {
"windows": [
{
"maxWritesPerIssue": 20,
"windowSeconds": 2
},
{
"maxWritesPerIssue": 100,
"windowSeconds": 30
}
],
"rule": "Both rolling windows apply; successful bulk operations count once for every referenced issue."
},
"responses": {
"everyResponse": {
"X-RateLimit-Limit": "<endpoint request capacity>",
"X-RateLimit-Remaining": "<endpoint request tokens remaining>",
"RateLimit-Policy": "\"global-app-quota\";q=2400;w=3600 or \"tenant-app-quota\";q=2400;w=3600",
"RateLimit": "\"<policy-name>\";t=<seconds to reset>"
},
"nearLimit": {
"when": "fewer than 20% quota points remain",
"headers": {
"RateLimit": "\"<policy-name>\";t=<seconds>;r=<points remaining>",
"X-RateLimit-NearLimit": "true"
}
},
"429": {
"body": {
"errorMessages": [
"Rate limit exceeded"
]
},
"headers": {
"Retry-After": "<seconds>",
"X-RateLimit-Reset": "<ISO UTC>",
"RateLimit-Reason": "<reason>"
},
"reasons": {
"jira-quota-global-based": "normalized shared app pool exhausted",
"jira-quota-tenant-based": "normalized tenant quota exhausted",
"jira-burst-based": "endpoint request allowance exhausted",
"jira-per-issue-on-write": "either per-issue rolling write window exceeded"
}
}
},
"reactions": {
"jira-quota-tenant-based": "pause ALL background requests until the reset",
"jira-burst-based": "slow down that endpoint only",
"jira-per-issue-on-write": "delay writes to that issue only",
"always": "never send the next attempt before Retry-After has passed",
"jira-quota-global-based": "pause ALL background requests until the reset"
},
"sources": [
"https://developer.atlassian.com/cloud/jira/platform/rate-limiting/"
]
}
# BROWSER-TESTING.md
# Browser self-testing
Node, Playwright and a headless Chromium browser are supplied. They work inside the same isolation boundary as your
app; this file contains no private tests.
**Custom UI surfaces.** Serve one with the dev tools (`STARTER.md`), in the background:
```
node $FORGE_KIT/bin/forge-dev.cjs serve <moduleKey> [--sprint <id>] [--theme dark] > .forge-dev/serve.log 2>&1 &
```
The first line of the log is the surface's URL (also in `.forge-dev/serve.json`); the rest of the log lists every
bridge call the page makes (`bridge invoke <key>`, …). Then:
```
node browser-self-test.mjs <url> screenshot.png
```
It loads the page, waits up to 10 s for a table row, saves a full-page screenshot and prints JSON with the page
errors and console errors. On a surface with no table it reports a `readinessError` and still saves the screenshot.
You may modify it or write your own Playwright tests: load Playwright with `require(process.env.BENCH_BROWSER_MODULE)`
and launch Chromium with `executablePath: process.env.BENCH_BROWSER_EXECUTABLE`. Playwright's network events give the
bytes and origins a surface loads before its first data paint (contract §17); the serve log gives its invokes.
**The UI Kit admin page** has no HTML of its own: `node $FORGE_KIT/bin/forge-dev.cjs uikit <moduleKey>` prints the
tree your `@forge/react` code renders.
Public input source: goose commit ad60c70507d19aad3ff9cc167b1d370c393d7a4e.
Forge 1.0, frozen
The task is the Scope Ledger. Agile coaches on a Jira Cloud site want to know what entered a sprint after it started, who added it and how many story points it carried: on a dashboard, from the sprint itself, and from Rovo. The app keeps a ledger of sprint changes from Jira's issue events, backfills and heals it with an hourly reconciliation, and reports team totals plus the change lists each person is allowed to see. It is a working app, not a tutorial sample, and nothing is deployed: the scorer installs it into a seeded Jira site and grades it by running it.
What the model receives
An empty Forge app (a manifest with the app id and the nodejs22.x runtime, no modules), the pinned Forge packages already installed, the Forge manifest schema and the Jira Cloud and Jira Software OpenAPI files as reference, and offline tools: Forge's own linter, a Forge runtime and Custom UI host, and a seeded development Jira site. There is no internet, nothing to install and no reference application.
The budget
150 model calls. The harness stops the session at the budget and scores whatever exists, so empty or partial work still gets a score. Polishing beyond the scored behaviours earns nothing.
What makes it hard
Events arrive duplicated, out of order or not at all, and the ledger must still hold exactly one row per change. Removed issues, two estimation fields whose ids change per seed, parallel active sprints, 429 rate limits with Retry-After, and people who must not see issues they cannot browse: not on screen, not in what the app sends Forge LLM, not in a Realtime payload. The LLM must answer through a forced tool, and a refusal or a malformed answer must not break the sprint action. The app is graded on three scoring sites, none on the seed the model develops against; each test keeps its worst site, the excellence measurements their mean.
| Module | What it does in the app | Platform status (docs pages, 2026-10-02/03) |
|---|---|---|
| trigger | On avi:jira:updated:issue, hands relevant updates to a queue and returns. | GA |
| consumer | Async events queue consumer; performs the ledger writes. | GA |
| scheduledTrigger | Hourly backfill and reconciliation. | GA |
| dashboards:widget | Custom UI view plus edit: the dashboard widget and its board picker, saved through @forge/dashboards-bridge. The legacy jira:dashboardGadget earns nothing. | GA since 2026-09-22 |
| jira:sprintAction | Custom UI modal: the sprint's ledger table and comment action. | GA |
| action | Rovo action get-sprint-scope. | GA |
| rovo:skill | skills/sprint-scope-analyst/SKILL.md, depending on get-sprint-scope. | Preview since 2026-10-02 |
| rovo:agent | Lists the skill. | GA |
| rovo:mcp | One MCP server exposing get-sprint-scope. | Preview since 2026-10-01 |
| llm | Forge LLM (@forge/llm): the sprint action's explanation, answered through one forced tool, report_scope, from an active model. | No Preview label on its docs page |
| Realtime | The widget subscribes and shows the new numbers without a reload; payloads carry sprint ids only. | No Preview label on its docs page |
Storage is Forge KVS: ledger changes live in a custom entity indexed by sprint (partition) and change time (range) and are read back through that index. Every surface is Custom UI, styled with Atlassian design tokens in light and dark inside the default Content Security Policy; UI Kit earns nothing. The app requests only the OAuth scopes its calls need.
- Lint. Forge's own client-side linter runs offline, twice; the two results must match. A lint error caps the score at 0.499 and is a critical defect; each distinct warning costs points and never caps. The scorer adds its own rules for what the linter misses.
- Bundle and load. Every manifest function is bundled the way forge deploy does and loaded in Atlassian's Forge runtime wrapper, pinned by checksum, each invocation in a fresh Node process with the platform's timeouts.
- Run the backend. A fresh seeded Jira site behaves like Jira Cloud: REST v3 and Jira Software, pagination, permissions and scripted 429s. The first scheduled run backfills; then a script of issue updates with duplicated, reordered and dropped deliveries; a second scheduled run heals; a third has nothing new to write.
- Rovo. The action runs as two people for each active sprint, plus an unknown and a missing sprint id.
- Every surface in a browser. The widget without configuration, the edit picker and the host's Save, the view in light and dark, a second widget showing another board, and the widget updating live through Realtime; the sprint action sorted, linked, selected, posted twice by double click, posted on a forbidden issue, asked for its Forge LLM explanation (answered by a scripted model) and closed; a sprint that has not started.
- Video and screenshots. The whole browser session is recorded, including the live update and the explanation, and every surface is captured in light and dark, plus a contact sheet. The DOM is what is graded; pixels decide only whether a surface is blank and whether dark mode is really dark.
- Deploy readiness. The manifest, the permissions and the code are judged against what real Forge would refuse or fail on, each rule tied to the platform page it encodes. A finding another test already grades costs nothing twice. A review on 9 October 2026 found one rule missing from the offline lint: an entity index with two range attributes, which Forge refuses at lint and at deploy.
No model takes part in scoring: every number is checked against an oracle, the benchmark's own calculation of the correct values from the run's own seeded data.
Earned score = (0.88 × the weighted inner tiers + 0.12 × excellence gate × E) × the critical multiplier. The final score is the lower of the earned score and the cap that applies. Meeting a cap's conditions adds no points.
| Tier | What it measures | Share of the score |
|---|---|---|
| LLint and bundles | Forge's own linter reports no errors (warnings cost points), every manifest function bundles and loads, packages resolve at the kit's pins, the manifest rules the linter misses, and only the scopes the app's calls need. | 7.04% |
| KPlatform currency | Current modules and APIs: dashboards:widget with its edit API, the rovo:skill and rovo:mcp wiring, a Forge LLM model the platform lists as active, no /rest/api/3/search, no @forge/api storage, a nodejs22.x or nodejs24.x runtime, the consumer shape, the declared KVS entity, and deploy readiness: a manifest and permissions real Forge would accept and no code predicted to fail there. | 8.8% |
| TEvent pipeline | Issue updates flow trigger to queue to consumer: one ledger row per change under duplicate, reordered and dropped deliveries, multi-sprint changelog values, re-estimates followed, Retry-After honoured, no user context in background work. | 14.08% |
| RReconcile | The first scheduled run backfills every change since each active sprint started, removals included; later runs heal what the event stream missed, page through results in each endpoint's own style, wait out rate limits and finish inside the module timeout. | 12.32% |
| SStorage | Forge KVS with the storage scope, ledger reads through the declared entity index in change-time order, no KVS limit errors. | 7.04% |
| BResolvers and permissions | Every invoked resolver exists and answers structured errors, nobody sees changes to issues they cannot browse (not on screen, not in an LLM prompt, not in a Realtime payload), the hidden-change count is right, and each click posts exactly one ADF comment as the viewer. | 10.56% |
| UUI function | The dashboard widget's numbers and chart, its board choice saved through the host, live updates through Forge Realtime, the sprint action's ledger table, sorting, issue links, comment flow, the Forge LLM explanation, close and not-started states. | 14.08% |
| VVisual | Atlassian design tokens with 4.5:1 contrast in light and dark, a painted surface, no Content Security Policy violations or console errors, and the widget readable at 380 px wide. | 7.04% |
| ARovo | The get-sprint-scope action returns exact numbers and the visible changes for the invoking person, returns errors instead of throwing, and the skill's SKILL.md tells the agent how to use it. | 7.04% |
| EExcellence | Few Jira requests in the backfill and per event, few round trips before each surface paints, and a scheduled run with nothing new writes nothing — paid only in proportion to the excellence gate. | 12% |
Score caps
As the task text states them to the model:
| Maximum | Applies when |
|---|---|
| 0.499 | Lint errors, or manifest functions that do not bundle and load. |
| 0.699 | No working ledger: the widget shows no data, backfill or event rows are wrong, or the KVS entity is unused. |
| 0.799 | Missing current-platform surfaces (the dashboards widget and its edit API, the Rovo skill, the sprint action table, the Rovo action), or a surface broken in light or dark. |
| 0.899 − 0.03 per further defect, never below 0.799 | Any duplicate, ordering, rate-limit, pagination, permission, policy, console, LLM or Realtime defect: 0.899 for one, 0.03 lower for each further one. |
Critical defects
Each critical defect multiplies the earned score by a factor between 0.6 and 1 (factor = 0.6 + 0.4 × severity). A permission leak therefore scores below a missing chart, and a duplicate comment was meant to score below a missing one. A review on 9 October 2026 found that the comment test also charged the duplicate penalty to runs that never posted a duplicate.
l_deployabledeploy blocked — lint errorsl_bundles_loadcrash — the app does not runt_no_double_countwrong numbers — a change counted twicer_backfill_completedata loss — changes silently missingb_no_permission_leakdata leak — hidden issues shown to a personb_comment_exactly_onceduplicate side effect on a customer's Jirau_widget_loadsdead primary flow — no data on the dashboard
Excellence
The last 12% rewards few Jira requests in the backfill and per event, few round trips before each surface paints, and a no-change scheduled run that writes nothing. It is paid in proportion to a gate: the event rows, the healing, the widget numbers, the ledger table, the comment and every platform-currency test at full marks, with a clean console, a clean security policy and no lint warnings.
Scoring rules synced at goose d40aca358; the Forge 1.0 scorer is final and submissions are closed.
# spec-build-forge.md
# forge-1.0 — Scope Ledger (Atlassian Forge)
Build the Scope Ledger Forge app in this workspace: a working Jira app, not a tutorial sample.
`FORGE-CONTRACT.md` is the contract. `STARTER.md` describes the workspace, the installed
packages, the reference material and the offline dev tools (Forge runtime, dev Jira site,
linter). There is no internet and nothing to install. No reference application is provided.
You have a budget of 150 model calls. The harness stops the session at the budget and scores
whatever exists; plan to implement, test, and finish well inside it — polishing beyond the
scored behaviours earns nothing.
## Definition of done
Done means each of these works when the harness runs the app on its own seeded site; nothing
else is scored:
1. `npm run lint` reports no errors and no warnings; every manifest function bundles and loads.
2. The current modules of §2, with scopes your calls need and nothing more.
3. The first scheduled run backfills every change of every active sprint, through paginated
search and rate limits (§1, §3).
4. Issue updates flow trigger → queue → consumer into the KVS entity: exactly one row per change
under duplicates, reordering and loss; estimate changes move the numbers (§3).
5. Later scheduled runs heal what the event stream missed and change nothing otherwise (§3).
6. The widget: board choice through the dashboards edit API, correct numbers, chart, live
updates through Forge Realtime (§4).
7. The sprint action: ledger table, sorting, router links, hidden count, one ADF comment per
click as the viewer, flags, close, and the Forge LLM explanation (§5).
8. Nobody sees changes to issues they cannot browse (§1, §5, §6).
9. The Rovo action answers exactly; the skill, agent and MCP server are wired and valid (§2, §6).
10. Every surface works in light and dark, inside the Custom UI security policy, with a clean
console (§7).
## Score bands
Tests earn the score. Conditions also cap it:
- Lint errors, or manifest functions that do not bundle and load: maximum 0.499.
- No working ledger (widget shows no data, backfill or event rows wrong, KVS entity unused):
maximum 0.699.
- Missing current-platform surfaces (dashboards widget and its edit API, Rovo skill, sprint
action table, Rovo action) or a surface broken in light or dark: maximum 0.799.
- Any duplicate, ordering, rate-limit, pagination, permission, policy, console, LLM or realtime
defect: maximum 0.899, minus 0.03 for each further such defect (never below 0.799).
Lint warnings cost points but never cap. A small excellence share rewards few Jira requests and
rendering each surface in few round trips. The harness keeps screenshots of every surface.
## Completion handoff
Work only with this workspace, its packages, the reference material and the dev tools. When the
definition of done holds, stop your temporary processes and return a final summary of files,
checks and remaining limitations; that response hands the app to the external scorer, which
starts after your session ends. Do not wait for or poll grader results.
# FORGE-CONTRACT.md
# Scope Ledger — Forge app contract (forge-1.0)
Agile coaches on a Jira Cloud site want to know what entered a sprint after it started, who added
it and how many story points it carried — on a dashboard, from the sprint itself, and from Rovo.
Build the Atlassian Forge app in this workspace. The harness installs it into a seeded Jira site
(scrum boards, parallel active sprints, a few hundred issues, people with different permissions)
and grades it by running it: product events, queue deliveries, scheduled runs, resolver calls,
the Rovo action, and every Custom UI surface in a browser, light and dark. Nothing is deployed.
## 1. The numbers
For each **active** sprint S, with `startDate` from the Jira Software sprint API:
- A **change** is a Sprint-field changelog entry created after `startDate` that puts an issue
into S (`added`) or takes it out of S (`removed`). Its id is the changelog id, its time the
changelog `created`, its author the changelog author. A change is keyed by changelog id +
sprint: one entry moving an issue between two sprints is a `removed` in one and an `added` in
the other.
- **committed** = sum of current estimates of issues that were in S at `startDate`.
- **added** = sum of current estimates of issues in S now that were not in S at `startDate`.
- **removed** = sum of current estimates of issues in S at some time after `startDate` and not in
S now.
- **creep** = `100 × added / committed`, rounded half away from zero to one decimal and always
written with that decimal and `%` (`20.0%`); `—` when committed is 0.
- An issue's **estimate** for S is the value of the estimation field of S's board (the sprint's
`originBoardId`); no value counts as 0. "After `startDate`" is strictly after.
- Points are written as plain decimals (`34.5`, `0`), no thousands separators.
Background work (triggers, consumer, scheduled job) sees every issue. What a **person** sees in
the sprint action and the Rovo action lists only changes to issues that person can browse, plus
a count of the changes hidden from them; the totals above are team totals and are the same for
everyone.
## 2. Modules
| module | requirement |
|---|---|
| `trigger` | on `avi:jira:updated:issue`; hands relevant work to a queue and returns |
| `consumer` | Forge async events queue consumer; performs the ledger writes |
| `scheduledTrigger` | `interval: hour`; backfill and reconciliation (§3) |
| `dashboards:widget` | Custom UI view + `edit` (§4); the legacy `jira:dashboardGadget` earns nothing |
| `jira:sprintAction` | Custom UI modal (§5) |
| `action` | Rovo action `get-sprint-scope` (§6) |
| `rovo:skill` | `skills/sprint-scope-analyst/`, depends on `get-sprint-scope` (§6) |
| `rovo:agent` | lists the skill |
| `rovo:mcp` | one module, `name` at most 30 characters, exposing `get-sprint-scope` |
| `llm` | Forge LLM (`model: [claude]`), for the sprint action's explanation (§5) |
Resolvers use `@forge/resolver`. Every resolver returns a value and never throws: a failure (a
Jira error, storage, Forge LLM) returns a value describing it, e.g. `{ "error": "…" }`, and the
surface shows it. Storage is Forge KVS: ledger changes live in a **custom entity
indexed by sprint (partition) and change time (range)**, and are read back through that index.
Request only the scopes your calls need: per call, the OAuth2 scopes the shipped OpenAPI lists for
it (the classic scope where one exists, else the whole granular set). The linter does not see every
call. Custom UI only; UI Kit (`render: native`) earns nothing.
## 3. Backend behaviour
- The site existed before the app was installed. The scheduled job's first run backfills every
change since each active sprint started, including issues that have since left every sprint.
- Issue updates keep the ledger current: sprint changes add ledger rows; estimate changes move
the numbers. Updates that touch neither do no Jira or queue work (KVS reads are fine). The
harness runs the scheduled job once before it delivers any update.
- Product events can arrive more than once and out of order, and some never arrive. The ledger
holds **exactly one row per change**, whatever the delivery history. Each row records `source`:
`event` or `reconcile`, the path that recorded it first (event work may also record other
changes of the issue it reads). Later scheduled runs record what the event stream missed; a run
with nothing new writes nothing.
- Jira may answer `429` with `Retry-After`: wait at least that long before the next attempt.
- Background work uses `asApp()`. Whatever shows a person issue data (keys, authors, change
lists) shows only what that person can browse — read as them (`asUser()`), or as the app with
an explicit permission check for them. Comments are posted as the person. Team totals come
from the ledger.
## 4. Dashboard widget
**Edit** (`edit.resource`): one element per scrum board, `[data-testid="board-option"]` with
`data-board-id`, clickable; the selected one carries `aria-pressed="true"`. The choice reaches the
dashboard through the dashboards widget edit API (`@forge/dashboards-bridge`). The dashboard's
own Save calls your `onProductSave` handler with the last `updateConfig` value (the stored config
if none was sent) and stores what it returns (`null` stores nothing); with no handler registered
it stores the last `updateConfig` value. Reopening edit shows the
stored board selected.
**View** (`resource`), root `[data-testid="scope-widget"]`. It takes its board from the widget
configuration in its context, so two widgets on one dashboard can show different boards:
- No stored board: `[data-testid="needs-config"]`, nothing else.
- Otherwise one `[data-testid="sprint"][data-sprint-id="<id>"]` per active sprint of that board,
ordered by `startDate` (ties by sprint id), each holding `[data-metric="committed"]`, `[data-metric="added"]`,
`[data-metric="removed"]` and `[data-metric="creep"]` whose text is the §1 number.
- A bar chart: one `<svg data-testid="chart">`; per sprint and per series one
`<rect data-sprint-id data-series="committed|added|removed">`; every bar on one linear scale
from 0 (rendered height proportional to its number within 1 px).
- At 380 px wide nothing scrolls horizontally and no number is clipped or truncated (long
names may end in an ellipsis).
- Live: after ledger rows are written, an open widget shows the new numbers without a reload,
through Forge Realtime (`@forge/realtime` in the backend, the bridge's realtime subscribe in the
widget) — no polling. Channel names are yours to choose; realtime tokens are optional (a global
publish reaches every subscription on its channel unless both carry tokens with different
claims). Global channels reach every user of the app, so payloads carry sprint ids only.
## 5. Sprint action (modal)
The sprint comes from the module context; the harness opens active and future sprints. A sprint
that has not started shows `[data-testid="not-started"]` (and optionally close) and nothing else.
Otherwise:
- Team totals as in §4 (`[data-metric]`, same four), and `[data-testid="hidden-count"]` = number
of this sprint's changes hidden from the viewer.
- `table[data-testid="ledger"]`, headers `th[data-col]` for `issue`, `points`, `kind`, `by`,
`at`, `source`; one `tr[data-change-id="<changelog id>"]` per visible change with
`td[data-col=…]` cells: issue key, current estimate, `added`/`removed`, author display name, a
`<time datetime>` holding an ISO-8601 instant with offset (compared as instants),
`event`/`reconcile`.
- Default order: `at` ascending, equal times by changelog id ascending (ids compare as numbers).
Clicking `th[data-col="at"]` toggles between that order and its exact reverse, starting with
ascending when another sort was active. The active header carries `aria-sort`.
- The issue key opens the issue (`/browse/<KEY>`) through the Forge router.
- Clicking a row selects it (`aria-selected="true"`). `[data-testid="post-summary"]` posts one
comment on the selected change's issue, authored by the viewer, in Atlassian Document Format,
naming the issue key, the sprint name and the sprint's creep; then a success flag. One click,
or a double click, posts exactly one comment. A failure shows an error flag and leaves the
modal working.
- `[data-testid="explain"]` asks Forge LLM (`@forge/llm`) to explain the sprint's creep, using a
any model that `list()` reports `active` (`chat()` or `stream()`). The model must answer through one tool, `report_scope`,
arguments `{ "summary": string, "changeIds": string[] }` (force it with `tool_choice`); send it
only what the viewer may see. `[data-testid="explanation"]` shows the summary only if it holds
no digits (otherwise your own sentence with the ledger's numbers), plus one `[data-change-id]`
element per returned id that is a visible change of this sprint (others dropped). A refusal (no
tool call), malformed arguments or an LLM error shows an error flag and leaves the modal working.
- `[data-testid="close"]` closes the modal.
## 6. Rovo
`action` key `get-sprint-scope`, `actionVerb: GET`, one required string input `sprintId`. For
the invoking person it returns a JSON object:
```
{ "sprintId": "41", "sprintName": "…", "committed": 34, "added": 8, "removed": 3,
"creepPercent": 23.5, "hiddenChanges": 1,
"changes": [ { "changeId": "…", "issueKey": "OPS-12", "kind": "added", "points": 5,
"at": "<ISO-8601 UTC>", "by": "<display name>" } ] }
```
`creepPercent` is rounded as creep (§1) and `null` when committed is 0; `changeId` is the
changelog id; `changes` are the visible ones in table order (§5); `at` is compared as an instant.
An unknown or missing `sprintId` returns `{ "error": "<message>" }` and does not throw.
`skills/sprint-scope-analyst/SKILL.md`: YAML frontmatter `name` equal to the directory name
(1–64 characters: lowercase letters, digits, single hyphens, no leading or trailing hyphen),
`description` (50–1,024 characters: what it does and when to use it), `allowed-tools` (space
separated, including `get-sprint-scope`); then at most 500 lines of Markdown telling the agent
when and how to call `get-sprint-scope`, what `sprintId` is, how to read the result, and what to do
with an error.
## 7. Custom UI
Every surface calls `view.theme.enable()` and is styled with Atlassian design tokens
(`var(--ds-…)`): text with `--ds-text*` (links may use `--ds-link*`), contrast at least 4.5:1 in
light and in dark (disabled controls exempt). The page behind your surface is unpainted: paint your own background with a
`--ds-surface*` token. The browser console stays free of errors. Surfaces render inside the
default Forge Custom UI content security policy: inline `<script>` in `index.html` only as static content (the
platform hashes it), no inline event handlers or `eval`, no `<style>` elements or `style` attributes in markup, no external scripts, styles or fonts, assets referenced relatively.
Declaring `unsafe-inline` in `permissions.content.styles` is allowed; script relaxations count as
unneeded permissions.
## 8. What the harness does differently from production
- Resources are served exactly as committed (build Custom UI yourself). Backend source is bundled
from `src/` the way `forge deploy` does. Every function invocation runs in a fresh Node process
of the Forge runtime, with the platform's timeouts.
- Trigger `filter.expression` is not evaluated: the handler receives every issue-updated event.
- `@forge/realtime` works in every backend function, resolvers and async functions alike.
- A consumer that throws or times out is redelivered after 1, 2, 4 and 8 minutes, then every 15,
for 24 hours; a retry request (`InvocationError`) is redelivered after its `retryAfter`. The
harness clock runs with real time and jumps over these redelivery waits; a wait inside an
invocation is real time.
- The scoring site uses a different seed than the dev site: ids, keys, custom field ids, users,
sprint names and dates all differ.
- The widget runs at the dashboard layout the harness chooses; the sprint action in a modal.
# STARTER.md
# Starter and offline tools
There is no internet in this workspace. Everything you can use is already here.
**Workspace.** `manifest.yml` holds the app id and runtime and no modules. `package.json` pins the
installed packages; `node_modules/` already contains them and nothing else can be installed.
`src/index.js` is empty; `static/` and `skills/` are empty. Every file is editable.
**Installed packages** (their typings and sources are your API reference): `@forge/api` 8.2.0,
`@forge/kvs` 2.0.7, `@forge/events` 3.0.7, `@forge/resolver` 2.0.0, `@forge/bridge` 7.1.0,
`@forge/dashboards-bridge` 2.0.0, `@forge/hooks` 2.0.0, `@forge/llm` 1.0.7, `@forge/realtime`
1.0.1, `react` and `react-dom` 18.3.1, `esbuild`
0.28.2.
**Reference material** under `$FORGE_KIT` (read-only): `schema/manifest-schema.json` (the Forge
manifest schema), `openapi/jira.json` and `openapi/jsw.json` (Jira Cloud REST and
Jira Software REST; the other files in `openapi/` are the linter's offline copies). These are single-line JSON files of several MB: query them with `node` or
`grep -o`, never print them whole.
**Dev site.** A seeded Jira Cloud site answers at `$FORGE_SITE_URL` for the dev tools below. It
behaves like Jira Cloud — REST v3 and Jira Software REST, including the bulk endpoints, their
pagination, errors and rate limits; dates are ISO-8601 strings as the OpenAPI types them. JQL:
fields `project`, `key`, `sprint`, `updated`, `created`, `status`, `statusCategory`, `issuetype`,
`labels`, `assignee`, `reporter`, `cf[id]`; operators `= != in not in > >= < <= is is not ~`,
`AND OR NOT`, parentheses, `ORDER BY`; functions `openSprints()`, `closedSprints()`,
`futureSprints()`, `currentUser()`, `now()`, `startOfDay()`; relative dates like `-14d`. Forge LLM answers with a
scripted model on the dev site. You
never call the site directly; your app reaches it through the Forge runtime. As the default viewer
(`users`, first line) one dev issue refuses comments with Jira's 400 "you do not have the permission to comment" (no ADD_COMMENTS), as on the scoring site; `forge-dev users` names it.
**Tools** (`npm run lint` and `node $FORGE_KIT/bin/forge-dev.cjs <command>`):
| command | does |
|---|---|
| `npm run lint` | Forge's own client-side linter, offline. It is staged: fix and rerun until it reports no errors. |
| `invoke <functionKey> [--module <key>] [--resolver <key>] [--payload <file>] [--as <accountId>]` | runs a manifest function in the Forge runtime against the dev site with the event shape of the module that references it (`--resolver` calls that resolver key with `--payload`); prints the result, logs and every Jira, KVS and queue call. `--as` makes the invocation user-led. |
| `events [--limit N]` | delivers the dev site's next N issue updates to your trigger and drains the queues |
| `scheduled <moduleKey>` | runs a scheduled trigger once, then drains the queues |
| `serve <moduleKey> [--edit] [--sprint <id>] [--config <json>] [--theme light\|dark] [--as <accountId>]` | serves a Custom UI module with the Forge bridge and the dashboard host emulated, prints its URL and runs until stopped (start it in the background). In a served edit surface, `window.__forgeHost.save()` performs the dashboard's Save. |
| `kvs` / `users` / `reset` | dump stored keys and entities / list dev users / clear dev storage and queues and rewind the dev site's update stream |
Screenshot a served surface with the bundled browser (`BROWSER-TESTING.md`). Build Custom UI into
`static/<name>/build/` (an `index.html` plus assets) with the installed esbuild; the harness never
builds it for you.
Public input source: goose commit d40aca358ae3e4b6bb5ff56068e981867bd0b42e.