One on-disk profile can't be shared by parallel workers, so the suite runs on a single worker.
Every Forge developer I know has shipped an app whose unit tests were green and which still fell over the first time a real user opened it in Jira. The resolver was fine. What broke was everything around it: the app renders inside a sandboxed iframe, on a page you reach through three menu clicks, behind a login that wants a second factor. None of that gets exercised by a Node test that imports your resolver and calls it directly. So you find out it's broken when someone tells you it's broken.
I got tired of that and built a harness that opens the real product in a real browser, navigates to the deployed app, and asserts on what actually renders. It drives five of our Forge apps across Jira and Confluence now, and it has caught fifteen bugs that every offline test waved through. Here's how it works, including the two things that cost me an afternoon each so you can skip them.
Note
Prerequisites — a Forge app deployed to a dev environment you can log into, Node 18+, and Playwright (npm i -D @playwright/test && npx playwright install chromium). You'll need the app's UUID (from your manifest's app.id, dropping the ari:cloud:ecosystem::app/ prefix, or from the Developer Console) and the environment id (from the Developer Console, or the forge deploy --verbose output — forge environments list shows environment names, not the id you need). This is written for Custom UI. Modern UI Kit (@forge/react) also renders through a hosted iframe, so most of it applies — but dump the frames first (there's a snippet below) and confirm where your app actually lives before you assume anything.
Why can't I just point Playwright at the app?
Auth is the first wall, and the token you already have is useless against it. A Forge REST API token talks to the REST API; it can't mint a browser session, so there's no header that makes you "logged in". You have to log in through the browser for real. And the moment you try to do that headlessly and replay a saved storageState, Atlassian treats the run as a brand-new device and asks for a 2FA code your script can't produce.
Caution
The token rabbit hole. I spent a while trying to make a headless login stick: export storageState, replay it, get challenged for a code, repeat. The fix wasn't a smarter token. It was to stop throwing away the login between runs — keep the whole browser profile on disk and Atlassian stops treating you as a stranger.
The second wall is the iframe. Forge Custom UI renders your React app inside an iframe for isolation, which is documented and deliberate. Your page.getByRole(...) calls run against the host page, and the host page doesn't contain your app, so they match nothing while you stare at a button you can plainly see and wonder why it isn't found. Everything for your app's UI has to be scoped through the frame.
A persistent profile that captures its own evidence
Instead of a fresh context per run, launch a persistent context pointed at a directory you keep around, and wrap it in a fixture. Log in by hand the first time; every run after reuses the cookies and the remembered-device token Atlassian stored in that profile, so the second factor stays quiet.
The fixture is also where you turn on video and tracing, and this is the part that bit me: Playwright's use: { video, trace } config only applies to the built-in page/context fixtures. A context you launch yourself ignores it completely. So you record evidence in the launch, or you get none.
tests/fixtures.ts
ts
1import{ test as base, chromium,typeBrowserContext}from"@playwright/test";2import path from"node:path";34// Gitignore this — it holds your session cookies and the remembered-device token.5constPROFILE_DIR= path.resolve(".auth/profile");67exportconst test = base.extend<{ site: BrowserContext }>({8site:async({}, use, testInfo)=>{9// headless:false is load-bearing. A headless profile trips the "new device" 2FA check.10// recordVideo lives here because config `use: { video }` never reaches a context you launch.11const ctx =await chromium.launchPersistentContext(PROFILE_DIR,{12 headless:false,13 viewport:{ width:1440, height:900},14 recordVideo:{ dir: testInfo.outputDir },15});16await ctx.tracing.start({ screenshots:true, snapshots:true});17try{18awaituse(ctx);19}finally{20// Runs even when an assertion throws, so the profile lock is always released.21// pages()[0] is the bootstrap about:blank a persistent context opens with, so22// screenshot the LAST page — the one the test actually drove.23if(testInfo.status !== testInfo.expectedStatus){24await ctx.pages().at(-1)?.screenshot({ path: path.join(testInfo.outputDir,"failure.png")});25}26await ctx.tracing.stop({ path: path.join(testInfo.outputDir,"trace.zip")});27await ctx.close();28}29},30});3132export{ expect }from"@playwright/test";
1
Warm the profile once
a throwaway script that launches the same PROFILE_DIR headed, opens your site, and sits on page.pause() while you complete SSO and the 2FA prompt by hand.
2
Reuse it everywhere
every spec imports this fixture, so it's already authenticated and already recording.
3
Fail fast when it expires
the session idles out after weeks; if a run lands on id.atlassian.com, stop and re-warm the profile instead of retrying blindly.
Deep-link straight to the app
Don't click through the product nav to reach your app; it's slow and it breaks. Forge app pages have stable deep links built from the app UUID and the environment id. Use the bare app UUID, not the full ARI. One asymmetry to know: a Confluence global page carries the module's route in the path, while a Jira global page doesn't.
tests/deeplink.ts
ts
1constSITE= process.env.SITE_URL!;// https://your-dev.atlassian.net2constAPP= process.env.FORGE_APP_ID!;// bare UUID from manifest app.id3constENV= process.env.FORGE_ENV_ID!;// env UUID from the Developer Console / `forge deploy --verbose`4constROUTE= process.env.FORGE_ROUTE!;// the `route` on your confluence:globalPage module56// jira:globalPage — no route in the path7exportconst jiraGlobalPage =`${SITE}/jira/apps/${APP}/${ENV}`;8// confluence:globalPage — Confluence needs the module's route in the path9exportconst confluenceGlobalPage =`${SITE}/wiki/apps/${APP}/${ENV}/${ROUTE}`;
Issue panels and macros are the exception: their parameters get stripped, so they aren't deep-linkable. You reach those by driving the host page that contains them rather than by URL.
Drive the iframe
Now the whole pattern. Go to the deep link, grab the Custom UI frame, wait for the app's root to mount, then assert on the app like any page, just scoped to the frame.
tests/dashboard.spec.ts
ts
1import{ test, expect }from"./fixtures";2import{ jiraGlobalPage }from"./deeplink";34test("the dashboard renders and loads its data",async({ site })=>{5const page =await site.newPage();6await page.goto(jiraGlobalPage);78// Custom UI lives in one iframe; your app mounts into #root inside it.9const app = page.frameLocator('iframe[data-testid="hosted-resources-iframe"]');10awaitexpect(app.locator("#root")).toBeVisible();1112// From here it's ordinary Playwright, scoped to `app` instead of `page`.13awaitexpect(app.getByRole("heading",{ name:"Portfolio"})).toBeVisible();14awaitexpect(app.getByText("No data").first()).toBeHidden();15});
Success
The one caveat about that selector.data-testid="hosted-resources-iframe" is an internal Atlassian attribute, not a documented contract. It's held for me across Jira and Confluence through 2026, but if a run suddenly can't find the frame, don't trust me — dump the page's frames and grab whichever one hosts your #root:
ts
1for(const f of page.frames())console.log(f.name(), f.url());
The config, and the one line that scales it wrong
Two things earn a mention. Retries: live UI carries real network latency, so a couple in CI is reasonable. Workers: this is the one that will bite you. A single on-disk profile can't be opened by two Chromium processes at once — the directory is locked — so the whole suite has to run on one worker. Playwright parallelizes by default, and the first time a second worker tries to launch the same profile it just dies.
playwright.config.ts
ts
1import{ defineConfig }from"@playwright/test";23exportdefaultdefineConfig({4// One shared profile → one worker. Chromium locks the user-data-dir; parallel workers can't share it.5 fullyParallel:false,6 workers:1,7 retries: process.env.CI?2:0,8});
If a single worker is too slow once you have real coverage, the way up is to copy the warm profile into a per-worker temp directory at startup so each worker gets its own lockable copy. I ran on one worker far longer than I expected to before that mattered.
When one app becomes several
A single app is one spec file, and honestly you can just watch it yourself. The math changes when you're maintaining a handful of them, each with its own transitions and release cadence, and "did that last change break anything in the real product" stops being a question you can answer by hand.
Live testing earns its keep on the static checks too, not just the UI: forge lint cannot see calls made through a request helper, so an app can pass lint, pass deploy, pass install and still 401 on its first write. Only something that calls the real API catches that.
Tip
This is how we keep CogniRunner — our AI-backed Jira workflow validator, full disclosure it's one of ours — green across releases. Every rule type it ships gets driven through the real transition UI on a live site, and a failing run hands back the video plus a written note of what broke instead of a bare red X. If you want to see the kind of app this is built for, it's on the Atlassian Marketplace. The technique is the same whether the app is ours or yours.
Key takeaways
The persistent-profile trick is the whole ballgame. Everything after it is normal Playwright.
Scope every locator through the frame. Custom UI is a sandboxed iframe, and the host page has none of your app.
Config use: { video, trace } doesn't reach a context you launch yourself — record it in the fixture, or you'll ship a suite that captures nothing.
The same harness drives LeanZero Management, and its release post covers the scheduling engine those live tests exercise, including the two-engine parity problem that made testing against a real deployment necessary.
That parity problem now has its own write-up: two scheduling engines, one schedule walks through why the two engines carry opposite off-by-one conventions, and the fixed-point harness that keeps the preview honest about what Apply will write.
I don't work for Atlassian, so treat the internal selector as "worked for me" rather than gospel. The shape of this holds regardless of which attribute names the platform uses this month. What's the bug a live test caught for you that every offline test let through?
Sentinel Vault: you cannot block an edit in Confluence, so we undo it instead
