Context Rot & Attention Decay

1. Concept Overview & Systemic Problem

With the advent of models featuring massive context windows (Claude 3.7 with 200K, Gemini 2.5 with 1M+ tokens), developers have fallen into a dangerous illusion: it seems possible to endlessly maintain a single chat, dumping megabytes of logs, full file texts, and code drafts.

However, in practice, after 15–20 steps of autonomous operation, rapid degradation of agent behavior begins:

1. Rule Amnesia: The agent suddenly forgets the prohibition on using any or starts returning fabricated function names, despite clear constraints in the systemic prompt.
2. Negative Reasoning Transfer: After seeing 5 failed attempts to execute a command in its own history, the model begins to consider the erroneous paradigm as the norm and repeatedly suggests the same outdated code options.
3. Economic Catastrophe: The user pays for the full length of the context (e.g., 120,000 input tokens) on each reply just to fix a comma in one file.

Context Rot is a silent killer of vibe coding and agentic workflows, turning an intelligent assistant into a disoriented spaghetti code generator.

2. Architectural Taxonomy & Mental Model

Attention degradation within neural networks develops along three main axes:

• 1. Attention Dilution: In the mathematical formula of the transformer $\text{Softmax}(QK^T / \sqrt{d})$, the denominator sums the exponentials of all tokens in the context. As the number of secondary tokens increases from 2,000 to 100,000, the attention weight assigned to critically important systemic instructions exponentially dilutes.
• 2. Lost-in-the-Middle Effect: Attention is distributed in a U-shaped curve. Information that falls within the range of 20% to 80% of context depth experiences the highest percentage of distortions and hallucinations.
• 3. Feedback Pollution: Each failed linter output, 404 error, or commented-out code snippet becomes part of the training distribution for generating the next token. The model subconsciously begins to generate code in the style of its failed attempts.

3. Technical Pipeline & Internal Mechanics

The lifecycle of context degradation and countermeasures:

1. Inception & High Fidelity (Clean Start Phase, Steps 1–5): The context contains only clean systemic rules and the task. Execution accuracy is highest, and the model strictly adheres to the structure.
2. Accumulation & Intermediate Noise (Accumulation Phase, Steps 6–15): Context begins to include stack traces, git status outputs, and long JSON responses. Initial attention dilution occurs.
3. Attention Saturation & Drift (Degradation Phase, Steps 16+): The model starts to lose the thread of reasoning, changes naming styles, suggests outdated libraries, and repeats previous hallucinations.
4. Compaction & Thread Reset (Engineering Defense):
   • Rolling Compaction: automatic folding of old messages into a 3-paragraph digest.
   • Clean Forking: creating a new session that carries over only the final state (Final Diff).

4. Production Engineering Scenarios

01. Session Discipline in Claude Code and Cursor

Experienced developers never keep a single chat longer than one logical step. After the authorization module is written and tested, the session is closed. A new chat for the billing module is opened from a clean slate, adding links to the freshly created files.

02. Strict Output Sanitization (Stdout Truncation)

When the agent runs a build command or tests, the runtime intercepts the output. If npm run build produces 2000 lines of logs, the agent receives only the last 20 lines containing the actual error text, preventing immediate context saturation with noise.

03. Two-Step Pipelines in RAG

Before adding found web pages to the generator prompt, they must pass through an HTML-to-Markdown stripper and relevance filter. Only useful factual information is presented in the context, excluding site navigation panels and footers.

5. Pitfalls, Common Mistakes & Security

• Sunk Cost Fallacy: The user continues to convince the agent in the same long chat: “No, you forgot the import again! I told you that 10 messages ago!” Instead, it's simpler to hit /clear or open a new thread.
• Ghost Variables: The agent references variables or functions it created in step 3, but which you removed from the code in step 8, as they still linger in the chat history.
• Unchecked Spending: Working in a 'stale' context of 150K tokens can burn a developer's monthly budget in an hour without any benefit to the product.