Context Language Models
Quick summary
arXiv:2609.37725v1 Announce Type: new Abstract: We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOP
Key takeaways
- arXiv:2609.37725v1 Announce Type: new Abstract: We introduce Context Language Models (CLMs), language models that natively manage their own context.
- We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file.
- This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files.
Why it matters
“Context Language Models” should be evaluated beyond branding and benchmark scores. Its practical importance will emerge in task accuracy, latency, unit cost, safety and integration with real workflows.
Member comments