goose compaction: my 27B re-read the file 95% of the time — quoting it in the summary didn't help
Mihai Perdum
Author
12 min readAugust 11, 2026
Somewhere past turn 40, the worker that owns the integration pass in our swarm stops making progress and starts reading. It has already read store.py. It has already run the tests. Then the context fills up, compaction runs, and it goes back and reads store.py again.
I spent a while blaming the prompt for that. The prompt was not the problem. The compactor was throwing away the only copy of the file the model had. And, as it turns out, you cannot fix that by writing a summariser that quotes the file back. I tried that. It changed almost nothing.
This is what shipped in our fork of goose this week, and the three-way measurement that made me rewrite half of what I thought I knew about it.
What compaction actually did
goose compacts by summarising. When the conversation crosses a token threshold it asks the model to summarise the history, then rewrites the conversation so the agent sees the summary instead of the original turns. Nothing is deleted. The original messages stay in the session for the transcript, they are just marked invisible to the agent.
Exactly one message survived that verbatim:
rust
1// Find and preserve the most recent user message for non-manual compacts2let(preserved_user_message, is_most_recent)=if!manual_compact {3let found_msg = messages.iter().enumerate().rev().find(|(_, msg)|{4 msg.is_agent_visible()5&&matches!(msg.role,rmcp::model::Role::User)6&&has_text_only(msg)7});
Read has_text_only carefully, because that predicate is the whole bug. It requires the message to carry text and no tool content at all:
In a chat session that is fine, because the recent tail of a chat is mostly a human typing. In an agent loop it is worthless, because the recent tail of an agent loop is entirely tool traffic: a request to run cat store.py, the response carrying the file, a request to run pytest, the response carrying the failure. Not one of those messages is text-only. So the preserve-the-last-user-message mechanism reaches back past the entire tool loop to whatever the user last typed, and everything the loop actually produced goes into the summary and nowhere else.
I checked what that leaves rather than assuming it. I built the compactor at the commit before the change and dumped the messages the agent can actually see, for a five-message conversation that ends on a test failure:
text
1===== PRE-CHANGE compact_messages — what the model actually sees =====
2 [1] User/text: <mock summary of conversation>
3 [2] Assistant/text: Your context was compacted. The previous message contains a summary…
4 [3] User/text: add a retry to fetch_rows in store.py
5 -> 3 agent-visible messages
Three messages. The contents of store.py are gone. The test failure is gone. The model is left with a prose summary and the original ask, and its next move is the only move available to it.
That is the same behaviour our own judge kills workers for. There is a standing note in our bench findings that the integration worker spends roughly 30% of its calls on cat, ls, find and grep over a tree whose full manifest is already in its prompt, at about 80 seconds a call. Call it six and a half minutes per run spent re-discovering things it was already told. Compaction was quietly manufacturing more of exactly that.
The fix, and the two parts that are not obvious
The change adds GOOSE_COMPACT_KEEP_TAIL: summarise everything except the last K messages, and put those back verbatim after the summary.
rust
1let keep_tail = keep_tail.min(messages.len().saturating_sub(2));2letmut cut = messages.len()- keep_tail;3// A kept tail may not OPEN on a tool response whose request was summarized away — that is an4// orphan `role:tool` message and OpenAI-compatible servers reject the request outright. Extend5// the tail backward until its first message carries no ToolResponse (the paired request then6// rides along); `cut == 0` degrades to keeping everything except the summary, which is safe.7while cut >08&& cut < messages.len()9&& messages[cut]10.content
11.iter()12.any(|c|matches!(c,MessageContent::ToolResponse(_)))13{14 cut -=1;15}16let messages_to_compact =&messages[..cut];
The naive version of this is messages[len-K..], and it is wrong twice.
K is a floor, not a ceiling. If the message at cut is a tool response, its matching tool request is on the other side of the cut and about to be summarised away. That leaves a role:tool message with no tool_calls before it. The loop walks the cut backwards until the tail opens on something that is not a tool response, dragging the paired request along with it. Ask for three, get four:
text
1===== GOOSE_COMPACT_KEEP_TAIL=3 — what the model actually sees =====
2 [1] User/text: <mock summary of conversation>
3 [2] Assistant/text: Your context was compacted. The previous message contains a summary…
4 [3] Assistant/tool_request: shell {"cmd": "cat store.py"}
5 [4] User/tool_response: "def fetch_rows(conn, q):\n cur = conn.cur…
6 [5] Assistant/tool_request: shell {"cmd": "pytest -q"}
7 [6] User/tool_response: "1 failed: test_retry - AssertionError: expec…
8 [7] User/text: add a retry to fetch_rows in store.py
9 -> 7 agent-visible messages
The preserved user message can now be a duplicate. If the most recent text-only user message falls inside the kept tail, appending the preserved copy as well hands the model the same instruction twice, and that is the one instruction it attends to hardest. So the append is skipped when the tail already carries it. In the dump above the ask sat in the summarised head, so the copy is still appended at [7]; that is correct, and it is the case anyone would test with, which is exactly why the other case is easy to miss.
The default is 0, and 0 is meant to be the old behaviour exactly. I did not take that on trust. I built both trees, ran the same conversation through each, and diffed the serialised output rather than eyeballing a printout: normalising only the wall-clock created stamp, the pre-change compactor and the post-change compactor at K=0 produce byte-identical JSON, for the agent-visible set and for the full message list including the invisible originals. The swarm runner sets K=3 for its own workers at run start unless the environment already says otherwise, so nothing outside the swarm changes shape at all.
Note
If you build two git worktrees of the same crate and point them at one CARGO_TARGET_DIR, they collide on the same artifact hash. Mine silently served me the oldlibgoose rlib for a build of the new tree, and it presented as "unresolved import" for a function I could see in the source. Either give each worktree its own target dir, or touch the crate root before you trust a result.
Measuring it on the model that has the problem
Everything above is mechanism. The claim underneath it is behavioural, so I tested it on the model our fleet actually runs, on the machine that runs it: a Mac Studio M3 Ultra with 96 GB, serving DavidAU's Qwen3.6-27B-Fable-Fusion-711 Heretic NEO MAX MTP at Q8_0 through llama-server from llama.cpp b10330. Different quant from the one I put up against MLX this week, same story on load: llama.cpp logs the blk.64.nextn.* MTP tensors as unused and carries on.
Three histories, identical except for compaction shape. All three offer the same two tools, shell and write_file, and the same instruction to take exactly one action.
Arm A is the pre-change output: system prompt, the prose summary, the "your context was compacted" notice, the user's ask.
Arm B is the K=3 output: the same four, plus the verbatim tail — the cat store.py call, the file, the pytest call, the failure.
Arm C is the control, and it is the arm that matters. Same shape as A, but with the file contents and the failure text pasted into the prose summary. This is what "just write a better summariser" looks like.
The summary in A and C is deliberately a good one. It states in plain English that the file was read, what the function does, that pytest ran, and that test_retry failed expecting three attempts and getting one. Every fact is in there. A strawman summary would have proved nothing.
Twenty samples per arm at temperature 0.7, counting how often the model's first action is a read rather than the edit:
arm
what the model sees
first action is a read
of those, reads store.py
prompt tokens
A — pre-change
summary, notice, the ask
19/20 (95%)
19
159
C — control, bytes pasted into the summary
summary including the file, notice, the ask
18/20 (90%)
18
217
B — keep-tail=3
summary, notice, the last four turns verbatim, the ask
3/20 (15%)
0
283
4 rows × 5 columnsHeader row enabled
Two things in that table surprised me.
Quoting the file into the summary does almost nothing. Arm C carries the same bytes as arm B. It still went back and ran cat store.py 18 times out of 20, statistically indistinguishable from having no bytes at all. Whatever is happening, the model is not treating "the file, described to me" as "the file, in my hands". Putting the same content back in its original tool-message position is what changed the behaviour; putting it in prose did not.
Arm B's three reads are not the same behaviour. None of them touched store.py. All three went looking for the test file — grep -A 30 "test_retry" test_store.py, cat test_store.py — which the model was genuinely never given and could only infer from the failure text. That is not a re-reading spiral, that is a worker that has what it was handed and is going after what it wasn't. On the metric that actually matters, re-reading a file it already had, the rate goes from 19/20 to 0/20.
I have to be honest about the one confound I cannot fully close: arm B is also the longest prompt of the three, at 283 tokens against C's 217. I cannot prove from these runs that length contributes nothing. What I can say is that C holds the same facts as B and behaves like A, which points at the form the content arrives in rather than the amount of it. Why that would be (role and position in the template, the <tool_response> block a chat template renders a tool message into, something else entirely) I did not measure, and I am not going to invent a mechanism to explain a behaviour I only observed.
One more limit worth stating plainly. The commit that shipped this says "a strong model survives prose-only recall; a 27B does not", and I only measured half of that sentence. I measured the 27B. I did not run a frontier model through the same three arms, so the first half is a design assumption — a well-motivated one, and the reason the default is 0 rather than 3, but not something I can show you a number for.
The comment about orphan tool messages is too strong
That code comment says an orphan role:tool message is one "OpenAI-compatible servers reject outright". I nearly wrote that into this article as fact. Then I sent it four ways at a local server and it did not happen.
python
1TOOL_REPLY ={"role":"tool","tool_call_id":"call_1","content":"def fetch_rows(...)..."}23# B — the tail sliced naively: the tool reply's request was summarised away4call("B orphan tool (naive last-K slice)",[5{"role":"user","content":"<summary of the conversation so far>"},6{"role":"assistant","content":"Your context was compacted."},7 TOOL_REPLY,8{"role":"user","content":"continue"},9])
I ran four shapes against llama-server b10330 in three configurations: the small model with its own chat template, the same model with --chat-template chatml to take the template out of it, and the 27B we actually run:
shape sent
Qwen3-0.6B --jinja
Qwen3-0.6B chatml
Qwen3.6-27B --jinja
A well-formed pair
200 accepted
200 accepted
200 accepted
B orphan tool (naive last-K slice)
200 accepted
200 accepted
200 accepted
C orphan tool, entire array
200 accepted
200 accepted
500 template raised
D keep-tail shape (cut extended back)
200 accepted
200 accepted
200 accepted
5 rows × 4 columnsHeader row enabled
Row B is the one that matters. It is the shape a naive last-K slice produces, the shape the backward walk exists to prevent, and it is accepted everywhere. llama.cpp performs no protocol-level validation of tool pairing at all; it renders what you hand it.
The single non-200 in that grid is not validation either. It is Qwen3.6's own Jinja chat template hitting raise_exception('No user query found in messages.') on a message array that contains nothing but one tool reply, and llama.cpp surfaces that as a 500, not a 400. So whether a malformed history errors depends on the model's template rather than the server, and when it does error it looks like a server fault rather than your bug.
The strict end of the spectrum is real. OpenAI's Chat Completions rejects it with Invalid parameter: messages with role 'tool' must be a response to a preceding message with 'tool_calls', an error common enough to have its own writeups and a trail of framework bugs behind it: one in OpenAI's own agents SDK, and one where OpenRouter returns the same 400. Elsewhere in the same repo the OpenAI request formatter already carries a comment about emitting a placeholder tool_calls entry for an unparseable call, and that one is worded properly: "which strict OpenAI-compatible APIs reject."
So the guard is right and the stated reason for it was overstated. The reason to keep the backward walk is not that every server rejects an orphan; it is that you do not get to choose which end of that spectrum you are pointed at, and the permissive end is arguably worse. A 400 tells you immediately. A server that happily renders an orphan tool result into the prompt hands your model a file it was never told it asked for, and you find out from the output quality three days later.
Warning
"It worked against my local server" is not evidence that a message array is well-formed. llama.cpp accepted an orphan tool message that OpenAI and OpenRouter both return a 400 for. If you build histories by slicing — compaction, truncation, retry-with-fewer-messages — validate the pairing yourself, because the lenient path will not do it for you.
The bug I thought I found in the guard, and didn't
While I was attacking this I went after the escape hatch in that same comment: cut == 0 "degrades to keeping everything except the summary, which is safe". I built a conversation of four consecutive tool responses, drove the walk to zero, and got what looked like a clean hit: four orphan tool responses sitting in the agent-visible set, produced by the very code that exists to prevent them.
It was my fixture that was broken, not the code. I had built tool responses with no tool requests anywhere in the conversation, so of course they were orphans; the walk never had a request to drag along. Feed the same code a well-formed conversation — one assistant turn issuing three parallel calls, then their three responses, which is the realistic way to get consecutive tool responses — and the walk behaves:
The loop never examines message 0, so cut can genuinely reach zero on a valid conversation — but when it does the tail is the whole conversation, so it opens on message 0, and that is an orphan only if message 0 is itself a tool response. Which is invalid input. The escape hatch holds; my counterexample did not. I am including the dead end because a reviewer who does not chase their own findings down to the fixture will publish that one as a defect, and a wrong bug report costs more than the bug would have.
Picking K, and what it costs
K is a token budget question and the answer depends entirely on how big your tool results are. In our case that is bounded on purpose: the swarm caps any single tool result at 30,000 characters before it spills to a temp file, so the worst case for K=3 is a handful of oversized reads rather than something unbounded. A handful, not exactly three, because the backward walk can push the tail past K.
That cap has its own scar tissue, recorded in the source right next to the keep-tail line. It used to be 8,000, which sounds generous until a routine cat store.py on an 8.6 KB file trips it, the worker then cats the temp file the spill wrote, that also trips the cap, and the loop runs until the 900-second timeout. If you are setting a tool-result cap, set it above a normal source file and a normal test run, or you have built a spill loop.
Three is a small number on purpose. The tail exists to carry the thing the model is holding in its hands right now, the file it is editing and the error it is fixing. It is not a second memory system. If a worker needs something from thirty turns ago, that is the summary's job, and if the summary keeps failing to carry it, a bigger K is the wrong lever.
Key takeaways
Summarisation-based compaction has a blind spot shaped exactly like an agent loop: the recent tail is all tool traffic, and a "preserve the last user message" rule that requires text-only content preserves none of it.
Quoting the file into the summary did not fix it: 18/20 runs still re-read it, against 19/20 with no file at all. Returning the same bytes as the original tool messages took re-reads of that file to 0/20. The form the content arrived in mattered more than the content.
Never slice a message tail at a fixed offset. If the cut lands on a tool response, its request is on the other side of it and you have built an orphan.
llama.cpp does not validate tool pairing at all. The naive-slice orphan was accepted in every configuration I tested, while strict APIs return a 400 for it. Validate the pairing yourself instead of trusting whichever server you happen to be pointed at.
Ship the knob defaulting to off, then prove off is unchanged by running the old and new code against the same input rather than reasoning that it must be.
Chase your own findings down to the fixture before you report them. Mine died there.
What is not measured yet
The registered check for this change is a run-level one: on a post-compaction run, the integration worker's count of re-reading its own owned files should drop against the existing four-run corpus. That has not been run yet. The comparison above is a controlled probe of the mechanism, not a swarm run, and I am not going to present it as one. When the run lands I will report the number whichever way it goes.
For the surrounding context I have written before about what the swarm gets right and where it is slow and about how the plan-execute-judge loop fits together. This is the kind of thing that only turns up once you are running real work through local models for weeks rather than benchmarking them for an afternoon. The failure was never in the model, and it was never in the prompt.
What would you check first if your agent started re-reading files it had already read?
