ACP practice exams: one question, then all 2,990

Three days ago Mihai wrote up the four ACP-120 questions a candidate had sent us, and what they exposed. She kept going. After a couple more twenty-question mocks she wrote again, and this time she was careful to say she only really wanted to tell us about one:

An ACTIVE company-managed workflow is shared by several live projects. Without copying it or touching the workflow scheme, which TWO of the following changes can be made directly to it? (Choose two.)

Our key said that "Add an outgoing transition where several exist" was wrong. Her words: "that's not true! You can do this in both the old and new workflow editors. I think you have misunderstood the page. The point being made is that in the new editor you can add outgoing transitions where none exist, whereas this was not possible in the old editor."

She had read the page more carefully than we had. The Atlassian workflow editor documentation says, under "Edit and publish instantly": "you can delete statuses in active workflows, and if a status has no outgoing transitions (not including global transitions), new outgoing transitions can be added (regular or global)." That is a sentence about a case that was newly supported. Our explanation had read it as a boundary, as if adding a transition anywhere else were forbidden, and then built a choose-two question on that reading. The page says nothing of the sort. When the documented case is "now you can also do X", you cannot key a question on "therefore you cannot do Y".

That question is not on the site. It has been rebuilt with the true-but-marked-wrong option replaced, the editor named, and the Update workflow step and status remapping written in, and it stays held until someone reads the rebuilt version cold and the behaviour has been tried in the editor rather than inferred from the page. I would rather have 837 questions I can defend than 838.

What one question is worth

A single report like this one is not a single defect. It is a pointer at a way of reading documentation that will have produced the same mistake elsewhere, in questions nobody has reported yet. Turning "supported" into "the only thing supported" is a habit, not an accident, and the only honest response is to go and look for it everywhere.

So the whole bank was read again. All 2,941 questions that were on the site or in the candidate pools, ACP-120 853, ACP-220 780, ACP-420 455, ACP-520 450 and ACP-620 403, were read cold, stem and options first, the way a candidate meets them, before the stored key and its explanation were opened. That pass is a triage, not a proof. It finds the question that cannot be answered on one reading, the option that is also true, the explanation that claims more than its source, and the stem that quietly gives its own answer away. Each of those then gets read whole against the current Atlassian page, and either repaired, or held out of the pool with the reason written into the record.

The recurring finding was the one she had spotted. The documentation describes what a feature does; a question had turned that into what the feature cannot do. A sufficient condition had become a necessary one. A qualifier like "only" had been removed for readability, which made the wrong answer defensible. An explanation for the right answer taught a rule that was broader than the source. Fixing the reported instance and leaving its siblings would have meant another email in a week.

The fix that was worse than the live site

Here is the part I think is worth telling, because it is the part that was easiest to skip. Before anything was published, the repaired bank was built exactly the way the website builds it and compared, question by question, against what was actually live. Not against the previous draft, against the deployed pools that candidates were sitting that day.

That comparison said no. The rebuild article from the eighth had already established the principle that a question which cannot earn its place is held, not rewritten around, and the same standard applied to our own repairs. Out of 681 changed or removed questions read as before-and-after pairs, 239 were better and 301 were equivalent, but 21 were flagged as worse than the live version and another 120 could not be settled without more work. The pattern in the confirmed cases was consistent. A rewrite had removed a qualifier from a wrong option and turned it into a true one. A shortened stem had dropped the condition that made the answer unique. An explanation still refuted the old wording of an option that no longer said that. Every structural test on the website passed on that draft. The tests check the pool, the paper draw and the scoring. They do not read English.

So it went back. All 141 of those concerns were closed one at a time against the current question and its source page: 121 were repaired and read again, and 20 were taken out of the pool rather than argued for. Then the comparison was run again on the final bank. Across the five exams, 133 questions that were live are gone, every one with a written hold reason in its record, 1,389 retained questions changed in wording, options or explanation, and not one retained question changed its answer key or how many options it asks you to pick. If you sat a paper last week and sit one this week, a question you saw will either read better or be gone. It will not have flipped its answer.

A hundred more per exam

The pools have only ever shrunk in this series. This time they grow. Each exam gained 100 new questions, written in five batches of 20, and every batch went through the same gate before it counted. An independent reader solved the batch blind, from the stem and options only, with the key and the source unread, and recorded their answers. Only then was the key opened and compared. Where the reader disagreed, or found a second combination that also satisfied the stem, the question was rewritten and solved again. Batches were also compared against the nearest existing questions in the bank, because a new question that tests the same decision as an old one in different clothes is padding, not coverage. Several were replaced for exactly that reason.

The new questions keep the shape of each real paper. ACP-120 keeps its share of choose-two items and its five and six option questions; ACP-520 stays at three options and one answer throughout; ACP-420 and ACP-220 stay single-answer. ACP-620 is the one I will not overclaim: the only reference we have for its shape is a harvested bank rather than transcribed papers, and its new questions lean more heavily on four options than that reference does. They pass the same checks as the others. They are not a measured match to the real paper.

The live pools today are ACP-120 837, ACP-220 751, ACP-420 502, ACP-520 468 and ACP-620 432, which is 2,990 questions, up from 2,606. You can sit any of them at leanzero.net/certifications, free, and the paper draws the real exam's size and timing.

What this still is not

Reading every question against its documentation page is not the same as trying every claim in Jira. Where a claim could be checked in a real site cheaply, it was, and a few of the new questions carry a JQL query that was actually run against a test project before it was accepted. Most product behaviour was checked against the current page and nothing more. Where Atlassian's own pages disagree with each other, and they do in a handful of places, the question is held rather than keyed on whichever page we preferred. The held pile is 451 records across the five banks, and it is not shrinking by being argued with.

The candidate who started this did not send us a spreadsheet. She sent one question, a link, and a sentence explaining what the page actually meant. That was enough to find a class of error across five exams. If something reads wrong to you, that is the level of detail we need: the question, what you think the answer is, and why. Write to office@leanzero.net, mihai@leanzero.net or gabriela@leanzero.net, or say it in the Discord. We will read it the way she read our page.