Notes from the work

One email when a tutorial or migration write-up goes live. Nothing else, and an unsubscribe link in every email.

LeanZero

Two people in Romania doing Atlassian migrations, Forge apps and practical AI work for teams that would rather talk to the person doing the job. Most of what we learn ends up on this site.

Services

  • Atlassian migrations
  • Atlassian FastShift
  • Atlassian maintenance
  • Forge app development
  • AI development consultation

Topics

  • Jira
  • Jira Service Management
  • Confluence
  • Bitbucket
  • Atlassian Forge
  • Cloud Migration
  • Local AI
  • AI Coding
  • Certifications
  • Atlassian Team
  • All topics

Company

  • Blog
  • Tutorials
  • Contact

Community

  • Join Discord
  • Support this site

© 2026 LeanZero. All rights reserved.

Privacy policyTerms of service24/7 SupportTrust centerSecurityDPALegal notice
  1. Home
  2. Blog
Knowledge Base

Atlassian insights & tutorials

Real Atlassian solutions to real problems — no fluff, no SEO spam.

Read the latestFollow the tutorials
JiraConfiguration, workflows and fixes from real projectsLocal AIModels and agents running on your hardwareAI codingCoding tools and workflows tested in practice

Articles 7

All topicsAI CodingAtlassian ForgeAtlassian TeamBitbucketCertificationsCloud MigrationCogniRunnerConfluenceCost SlasherJiraJira Service ManagementLeanZero ManagementLocal AISentinel Vault
Are AI benchmarks reliable? We had 32 models build the same app twice
ArticleAI CodingAtlassian Forge

Are AI benchmarks reliable? We had 32 models build the same app twice

Gemini 4 Argon built our Forge 2.0 test app twice and scored 0.79, then 0.29. Claude Sonnet 5.5 scored 0.59 and 0.55. I wanted to know which models are the unpredictable ones. Two builds can't tell you, and part of one swing was our own harness.

Mihai PerdumOct 22, 202620 min read

More articles

Best model for agentic coding in October 2026: who won our benchmarks, and who surprised
ArticleAI CodingAtlassian Forge

Best model for agentic coding in October 2026: who won our benchmarks, and who surprised

We ran 44 models through two agentic coding benchmarks, then read their builds line by line. Paying per token I'd use GPT-6.1 Sol, and MiMo V2.6 Flash came 7th on our Jira benchmark at 31 cents a build. The winners mostly did dull things right, and Claude Haiku 5.5 stopped with the app half written.

Mihai PerdumOct 20, 202615 min read
Jev Router topped our coding benchmark. 117 of its 118 calls went to GPT-6.1 Sol
ArticleAI CodingAtlassian Forge

Jev Router topped our coding benchmark. 117 of its 118 calls went to GPT-6.1 Sol

TypeSafe's Jev Router took the top median on our Forge 2.0 agentic coding benchmark, 0.746 from two builds. OpenRouter's own generation records show it sent 117 of its 118 calls to GPT-6.1 Sol, which sits third on the same board. What a model router is, what Jev Router did here, and why two long builds per model can't tell a router from the model it picked.

Mihai PerdumOct 17, 202614 min read
DeepSeek V4.1 Flash coding benchmark: 0.33 on a Forge app upgrade, Opus 5.5 got 0.67
ArticleAI CodingAtlassian Forge

DeepSeek V4.1 Flash coding benchmark: 0.33 on a Forge app upgrade, Opus 5.5 got 0.67

I ran DeepSeek V4.1 Flash twice through Forge 2.0, our Atlassian Forge app-upgrade benchmark. It scored 0.2893 and 0.3790, a median of 0.3342 against 0.6745 for Claude Opus 5.5 and 0.5283 for MiMo V2.6 Flash. Then I read the code behind its biggest penalty, and I don't think our scorer got that one right.

Mihai PerdumOct 13, 202614 min read
Claude Haiku 5.5 vs GPT-6 Luna on two coding benchmarks: same price, 3.5x the agent bill
ArticleAI CodingAtlassian Forge

Claude Haiku 5.5 vs GPT-6 Luna on two coding benchmarks: same price, 3.5x the agent bill

Claude Haiku 5.5 and GPT-6 Luna both list at $0.10 in and $0.50 out per million tokens. In our agent runs Haiku billed 3.5 times as much, mostly because 64 of its 111 calls crossed the 100k-token line where its price goes up five times, and part of that was a bug in our own agent. It also built the better apps: 0.9661 to 0.395 on an Atlassian Forge app, 0.661 to 0.4797 on a payments app. The checks that decided both, the call-by-call bill, and how to keep Haiku under the step.

Mihai PerdumOct 12, 202622 min read
Qwen3.8 Flash vs Max Prime: the cheap one beat the flagship on our Forge 1.0 benchmark
ArticleLocal AIAtlassian Forge

Qwen3.8 Flash vs Max Prime: the cheap one beat the flagship on our Forge 1.0 benchmark

Qwen3.8 Flash beat Qwen3.8 Max Prime on an Atlassian Forge app built with no internet, 0.7961 to 0.6953, and lost to it on a payments web app, 0.6846 to 0.8364. Every check behind both results, the one manifest line that decided Forge, a probe in which neither model showed it knew the rule until it was handed the docs sentence, and what fine-tuning our own 27B taught us about forgetting.

Mihai PerdumOct 8, 202638 min read
Claude Opus 5.5 vs Sonnet 5.5 vs GPT-6.1 Sol: pricing and our coding benchmarks
ArticleLocal AIAtlassian Forge

Claude Opus 5.5 vs Sonnet 5.5 vs GPT-6.1 Sol: pricing and our coding benchmarks

Claude Opus 5.5 vs Sonnet 5.5: Anthropic's prices, context window, effort levels and release dates side by side, then our two coding benchmarks. Sonnet edged Opus on a payments app, 0.7984 to 0.7926, a margin I read as level. On an Atlassian Forge app Opus won, 0.9767 to 0.8649 after a 15 October re-score. GPT-6.1 Sol and Sol Pro ran on both boards, and the full bill for fine-tuning our own 27B model is at the end.

Mihai PerdumOct 5, 202645 min read

Found my advice helpful?

If I helped you solve something for free, coffee money is always appreciated (but never expected).

Buy Me a Coffee

Join the community

Join the community to ask questions, share your work, and connect with other professionals.

Join Discord