Knowledge Base

Atlassian insights & tutorials

Real Atlassian solutions to real problems — no fluff, no SEO spam.

Articles 7

Muse Spark Open benchmark: where the open-weight models we ran this month broke as coding agents

Meta's Muse Spark Open scored 0.091 and 0.0877 out of 1 on our Forge 2.0 coding benchmark and never got a Gauntlet verdict. The other open-weight models we ran locally, Google's Gemma 4 among them, did no better, and neither did our own Qwen3.8-27B fine-tunes. The best open result on our Macs came from the plain Qwen3.8-27B base. We read every build to find where each one broke, and a fair share of the breakage was ours.

Mihai PerdumOct 22, 202620 min read

ArticleAI CodingAtlassian Forge

Four Qwen3.8-27B fine-tunes and merges on a coding benchmark: none beat the base

We ran four popular Qwen3.8-27B fine-tunes and merges through Forge 2.0, our long agentic coding benchmark, on one 64 GB MacBook next to the stock base, one build each in our own 8-bit conversions. In our setup one model's write calls came back malformed, one wrote careful code and then got stuck hunting for a method that doesn't exist, one finished fast with placeholders, and the merge got the hardest parts right and scored 0.1105. The base's median is 0.1852.

Mihai PerdumOct 19, 202617 min read

ArticleLocal AIAtlassian Forge

Qwen3.8 Flash vs Max Prime: the cheap one beat the flagship on our Forge 1.0 benchmark

Qwen3.8 Flash beat Qwen3.8 Max Prime on an Atlassian Forge app built with no internet, 0.7961 to 0.6953, and lost to it on a payments web app, 0.6846 to 0.8364. Every check behind both results, the one manifest line that decided Forge, a probe in which neither model showed it knew the rule until it was handed the docs sentence, and what fine-tuning our own 27B taught us about forgetting.

Mihai PerdumOct 8, 202638 min read

ArticleLocal AIAtlassian Forge

Claude Opus 5.5 vs Sonnet 5.5 vs GPT-6.1 Sol: pricing and our coding benchmarks

Claude Opus 5.5 vs Sonnet 5.5: Anthropic's prices, context window, effort levels and release dates side by side, then our two coding benchmarks. Sonnet edged Opus on a payments app, 0.7984 to 0.7926, a margin I read as level. On an Atlassian Forge app Opus won, 0.9767 to 0.8649 after a 15 October re-score. GPT-6.1 Sol and Sol Pro ran on both boards, and the full bill for fine-tuning our own 27B model is at the end.

Mihai PerdumOct 5, 202645 min read

ArticleLocal AI

v0.5 of our Atlassian Qwen3.8-27B: beats v0.4, now with GGUF for llama.cpp and Ollama

v0.5 of the Qwen3.8-27B we fine-tune for Forge, Jira, Confluence and JSM is out. It beat v0.4 on every gate we measure, with no waiver needed this time, using a training method that fixed the one thing capacity kept breaking. GGUF versions for llama.cpp and Ollama, Q8_0 and Q6_K, are now published alongside the existing MLX release, parity proven across three separate measured links plus a served-model probe.

Mihai PerdumSep 20, 202611 min read

ArticleLocal AIAtlassian Forge

v0.4 of our Atlassian model: 21 of 25 apps compile, and the loop that was a diagram

Three days after v0.3, the Qwen3.8-27B we teach Forge went from 12 of 25 compiling apps to 21 of 25. On the way our own release rule rejected it, correctly by its own text, over a 40-character window repeated 13 times at 32k context. The window was a line of box-drawing characters in an ASCII diagram. Here is what changed, what the metric got wrong, and what we changed so it cannot happen the same way again.

Mihai PerdumSep 13, 202612 min read

ArticleLocal AIAtlassian Forge

I taught a 27B model to write Forge apps. As far as I can find, nobody had done that before

Qwen3.8-27B knows almost nothing about Atlassian Forge: ask it for an app and it invents a manifest format and imports packages that do not exist. Over four days on one Mac Studio I trained it until 14 of 25 manifests passed Atlassian's own validator and 12 of 25 apps compiled, from zero. The weights are public. Here is what it is, what it is not, and why I think it is a first.

Mihai PerdumSep 10, 202611 min read

Atlassian insights & tutorials

Articles 7

Muse Spark Open benchmark: where the open-weight models we ran this month broke as coding agents

More articles

Four Qwen3.8-27B fine-tunes and merges on a coding benchmark: none beat the base

Qwen3.8 Flash vs Max Prime: the cheap one beat the flagship on our Forge 1.0 benchmark

Claude Opus 5.5 vs Sonnet 5.5 vs GPT-6.1 Sol: pricing and our coding benchmarks

v0.5 of our Atlassian Qwen3.8-27B: beats v0.4, now with GGUF for llama.cpp and Ollama

v0.4 of our Atlassian model: 21 of 25 apps compile, and the loop that was a diagram

I taught a 27B model to write Forge apps. As far as I can find, nobody had done that before

Atlassian insights & tutorials

Articles 7

Muse Spark Open benchmark: where the open-weight models we ran this month broke as coding agents

More articles

Four Qwen3.8-27B fine-tunes and merges on a coding benchmark: none beat the base

Qwen3.8 Flash vs Max Prime: the cheap one beat the flagship on our Forge 1.0 benchmark

Claude Opus 5.5 vs Sonnet 5.5 vs GPT-6.1 Sol: pricing and our coding benchmarks

v0.5 of our Atlassian Qwen3.8-27B: beats v0.4, now with GGUF for llama.cpp and Ollama

v0.4 of our Atlassian model: 21 of 25 apps compile, and the loop that was a diagram

I taught a 27B model to write Forge apps. As far as I can find, nobody had done that before