Meta Researchers Propose Giving LLMs Full Control Over Their Own Context

Context Language Models (CLM) remove predefined context-management logic and let models rewrite their own history in a file accessible through Bash, reducing compute costs on long tasks.
Meta researchers have proposed Context Language Models (CLM), an approach that gives models full control over their own context.
Normally, an agent's context is managed with a simple rule: old context + new response -> new context. As the history grows, it eventually has to be compressed or cleaned separately.
In CLM, the authors propose removing predefined context-management logic entirely and allowing the model to rewrite its own history.
Technically, the current context is stored in a regular file accessible to the model through Bash. The model can delete unnecessary messages at any time, replace a large chunk of history with a short note, preserve an important result verbatim, or completely restructure the content. After each change, the file becomes the model's new context.
The model receives no explicit rules about how to manage history. In experiments, agents began adding notes inside the context, writing functions to delete old search results, and building compact tables with the states of other agents. For example, in one test, a model maintained a working context within 6–8K tokens over 163 edits.
The authors went further and tried to train the model for this skill using reinforcement learning (RL). After training Qwen3.5-9B on BrowseComp-Plus, performance rose from 28.8% to 42.5%, while computation was 38.8% lower than a baseline trained the same way with ordinary summarization.
Arbitrary context editing, however, breaks standard caching. If something in the middle of the history changes, all states after that point formally need to be recomputed. The authors introduced Suffix Cache Reuse: for unchanged parts of the context, they continue to use the old KV-cache even if the text before them has changed. In experiments, this reduced server-side computation by another ~35% without noticeable quality loss.
On long tasks, the difference becomes substantial. On a 12-hour EdgeBench run, CLM scored 5% higher while reducing compute by 59%. In a 24-hour code-optimization test, CLM achieved a 65% better result than the baseline under the same compute budget.
All of this works on top of ordinary LLMs, without changing the model architecture itself.
Why it matters
AnalysisThis matters because self-managed context can lower inference costs and make AI agents more autonomous on long, multi-step tasks.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
Llama Family (Meta Llama)
A series of foundational open language models from Meta (Llama 3, 3.1, 3.3) that have become the industrial standard for the Open Weights ecosystem, local AI, and enterprise fine-tuning.
Meta Llama (The Premier Open Standard for Artificial Intelligence)
The flagship series of open language models from Meta (Llama 3, 3.3). Available for free download, modification, and local deployment on personal servers without restrictions or censorship.
Jev and System One AI: A Complete Practical Guide to the Decision Model from TypeSafe
A deep engineering breakdown of the flagship Jev model: System One architecture versus LLMs, Choice/Score/Noul primitives, request parallelism, confidence mathematics, video use cases, code, and an overview of local open-weight alternatives (Laya, GLiNER2.5, CLM-8B).
DeepSeek open-sources part of its chat interface
DeepSeek has released the source code for a portion of its chat UI, including web search, deep thinking, and conversation history features.