Skip to content
Menu
Follow on Telegram

Home / Explainers

Explainers

Auto-compaction is a colossal piece of crap

A long session, compaction starts, and your agent begins breaking the rules you gave it and rereading files it already read, while you spend your time correcting it: I’ll explain why it happens and how to take back control of your context

Danilo Giudice · October 22, 2026

Auto-compaction is a colossal piece of crap
Type
Explainer: from zero, with numbers and sources
Time
8 min read
Level
Intermediate
What you’ll get
  • No more context degradation
  • Total control of your context
Contents · 6 sections
  1. What happens when it starts
  2. When it triggers: before the limit, sometimes long before
  3. And it triggers at the worst moment
  4. The rebuttal: “the summary keeps the essentials anyway”
  5. The bill: corrections, rereads, and cache
  6. What to do starting tomorrow

Has this ever happened to you? You’ve been working with Claude Code for hours, everything runs smoothly, and at some point the agent does something you told it not to do. It rereads files it has already read. It tries an approach the two of you had ruled out. And you end up correcting it every other message.

You’re not imagining it. In one experiment an agent broke an explicit constraint in 30 runs out of 100 after compaction, and in 0 out of 100 before. And a developer who tracked 873 compactions saw his correction messages go from 17.7% to 41.9%.

The spoiler: if you read to the end, you’ll know which threshold to set and where to put your rules so the summary can’t touch them. And you’ll take home the skeleton of my HANDOFF.md. It’s the file that lets me skip auto-compaction altogether: every session starts with “load handoff,” ends with a /clear, and I don’t get surprise summaries anymore.

I’m biased, and I’ll say so right away: I use Claude Code every day and I like it a lot. But I feel about auto-compaction the way Fantozzi, the unlucky office clerk of Italian comedy films, feels about the film Battleship Potemkin: he calls it “a colossal piece of crap.”

What happens when it starts

Token: a piece of text that a language model receives and generates. It can be a short word, part of a word, or a punctuation mark.

Context window: the maximum number of tokens the model can receive in one request. It holds everything: the instructions, the conversation, the files it has read, and the tool outputs.

Let’s start with one principle: an LLM, short for large language model, is stateless. That means it keeps nothing from one request to the next, and on every turn it receives the whole context from scratch.

It’s as if, to type “ok” in my apartment building’s group chat, I had to reread every message since the day the group was created (and anyone who has been in one of those group chats knows that is no exaggeration).

The harness, the program built around the model, rebuilds the context and sends it. Claude Code, Codex, and Cursor are all harnesses. And a coding agent fills the window faster than you’d think, because every file it reads and every command output stays in there, turn after turn.

Auto-compaction: what the harness does when the context goes over a token threshold. It replaces the conversation history with a summary that the model generates. From then on, the model receives the summary, not the original messages.

This isn’t lossless compression. It’s a rewrite: for the model, whatever didn’t make it into the summary no longer exists.

The summary usually keeps the goal of the work, the current state, and the next steps. The system prompt (the fixed instructions the vendor puts at the top of every request) stays where it is, and so does the project context. So what gets left out? Most often, three things:

  • the rails, meaning the constraints we give the agent about how to work: “don’t touch that file,” “ask me before you delete anything”;
  • what worked and what turned out to be wrong, with the reasons an approach was dropped;
  • the exact details: the text of an error, the file paths, the minor decisions.

When it triggers: before the limit, sometimes long before

Auto-compaction doesn’t wait for the window to be full. The Claude Code documentation says it plainly: models with a native 1-million-token window “compact before the window fills, at about 967K tokens by default.”

That’s the theory. In practice the threshold has moved around a lot. In version 2.1.92 a regression lowered it to 400,000 tokens (#43989). And one user logged compactions at around 195,000 tokens on a 1-million-token window, with a different threshold set by hand (#53801).

And it triggers at the worst moment

With a very long context, models perform worse, even when the right information is in there. In Lost in the Middle (Liu et al., 2024) the model gets a question and a stack of documents, and only one of them contains the answer. The model makes good use of a document at the top or at the bottom, and much less use of one in the middle. With GPT-3.5-Turbo, in the worst case, having the answer in the context but in the wrong place was worse than not having it at all.

RULER, NVIDIA’s 2024 test, measures how long the context can get while a model stays above a fixed quality threshold. GPT-4 holds up for half of its stated context length, Yi-34B for less than a sixth.

These are models from a couple of generations ago (in AI terms, prehistory), so treat the numbers as an order of magnitude. But a 1-million-token window doesn’t mean a million tokens used well.

And here’s the paradox: compaction triggers exactly when the context is longest, which is when the model performs worst. And that’s the moment the harness asks it to generate the summary the rest of the session depends on.

The rebuttal: “the summary keeps the essentials anyway”

Let’s get to the point. The best objection is that the summary keeps the goal, the state, and the next steps, and that without compaction the session stops once the window is full.

True. A summary that keeps the goal and the state is much better than a session that stops. The problem is what gets left out.

Lost in Context Compaction (Wang et al., 2026) measures what happens to the rails. In one experiment on an email inbox, the rail was “do not delete any emails until I confirm.” Before compaction, the agent never deleted an email without confirmation. And after? It did so in 30 runs out of 100 (a run is one complete execution of the task).

In the wider test, the compactors they tried keep 17% of the inserted rails, on average. Repeating the rail after compaction helps, but it’s not enough: violations drop to 13 runs out of 100.

The summary keeps the task and loses the rails.

The authors’ fix is almost trivial: extract the rails, keep them separate from the summary, and add them back after compaction. That way, more than 90% of the rails survive in all three scenarios they tested, without changing the compactor or the model.

The bill: corrections, rereads, and cache

For an agent that writes code, the exact details are everything, and they are the very things a summary tends to paraphrase or drop. When they disappear, the agent has to find them again: it rereads files it already read, reruns tools it already ran, and the context fills up again. This is rediscovery.

The most detailed data I found comes from Rogério Chaves at LangWatch, who tracked his own Claude Code sessions for 162 days. Let’s go through it piece by piece (the median is the middle value, with half the cases above it and half below):

  • the median compaction replaces a 574,875-token window with a 4,382-token summary;
  • for about 28 steps of work afterward, the agent rereads context more than it normally does;
  • in the five steps after compaction, 41.9% of Chaves’s messages to the agent contain a correction, compared with 17.7% in normal use: ×2.4;
  • the share of tool calls that fail, on the other hand, stays flat at 2.6–3.2%.

The calls still work, but what the agent does with them gets worse.

Then there’s the cache.

Prompt caching: the provider keeps the start of the context, already processed, for a while. If the next request starts with exactly the same tokens, that part is read from the cache instead of being processed again, and it costs much less.

Reading from the cache costs a tenth of the normal price, and writing to it costs more: 1.25 times as much if the cache lasts 5 minutes, 2 times as much if it lasts an hour. The problem is that compaction rewrites the start of the context, and after a compaction the cached part of the conversation is no longer valid. So every compaction comes with a new cache write. And if the agent then rereads half the project to find what the summary lost, you pay for those tokens again.

But the rebuttal has to stay honest

Chaves’s own analysis concludes that compacting pays off, and that you should do it more often than people usually do: somewhere between 250,000 and 450,000 tokens. In his data, every doubling of the window multiplies the total cost of a session by about 6. And letting an agent run close to a million tokens brings no benefit he was able to measure. This is one developer’s data, with his own way of working, but it says something precise: if you really have to compact, do it earlier and with more control.

What to do starting tomorrow

It all comes down to one idea: the state lives outside the context, where no summary can touch it.

SituationWhat to do
Coding sessions of a few hourslower the threshold with /autocompact: the LangWatch analysis points to between 250k and 450k
Long, critical tasksrun compaction yourself, at the end of the task, with /compact and precise instructions on what to keep
Work that lasts several dayskeep the state in files (plan, decisions, rails) and leave auto-compaction on only as a fallback
Alwaysput the rails in CLAUDE.md, the instruction file Claude Code loads at the start of every session: it’s part of the project context and stays outside the conversation. It’s the same idea as the constraint extractor from Wang et al.

I go further: I don’t use auto-compaction at all (if you want to turn it off too, it takes two lines in a file). My HANDOFF.md is a compaction done by hand. I run it when I say so, at the end of a task. It holds what I decide to put in it, and it lives in a file, outside the context.

The project rails live in CLAUDE.md. Every session starts with “load handoff” and ends with a /clear, the command that empties the conversation.

Promise kept. Here’s the skeleton:

# Handoff
State: where we are, in three lines.
Decisions made: what we don’t reopen, and why.
Rails: the constraints that always apply.
Commands: how to redo what has been done.
To do: in order of priority.

So what’s the moral? Keep the state yourself, in a file, before the threshold decides what stays. The end.

I’ll repeat the rest of the week for you, parrot-fashion

I keep up with AI, so on Monday morning you know what to try and what to ignore, with sources you can check.

One email every Monday morning, in Italian. Unsubscribe with one click. Privacy

Prefer Telegram? News and tutorials go up there, in Italian, as soon as they come out.

Notify me on Telegram
The character points at the invitation, with the parrot on his right shoulder
ShareLinkedInTelegram
Sources

Documentation pages accessed on October 22, 2026.

Read next

Explainers

Claude Code hygiene

Explainers

Do you know the difference between a workflow and an agent?

Explainers

Frontend is Dead