AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Self-Compacting Language Model Agents

Collected Oct 13, 2026

Researchers at Johns Hopkins University have proposed SELFCOMPACT, a scaffold that lets a language model decide for itself when and how to compact its own accumulated context during long agent runs. The work is presented as a machine learning research publication, with authors Tianjian Li, Jingyu Zhang, William Jurayj, Xi Wang, Chuanyang Jin, Mehrdad Farajtabar, Eric Nalisnick and Daniel Khashabi, and no specific release date is given.

The problem it targets is structural. Agent traces made of chains of thought plus tool calls accumulate stale content that anchors later generations, and those traces eventually outgrow the context window. Current scaffolds handle this with fixed-interval compaction: once the trace crosses a token threshold, it is summarized or truncated on a schedule. Because that trigger ignores the shape of the trajectory, the authors argue it can throw away partial results in the middle of a derivation or a search.

SELFCOMPACT instead pairs two inference-time elements. The first is a compaction tool that the model itself invokes to summarize the accumulated context. The second is a lightweight rubric specifying when compaction should fire and when it should be suppressed. According to the description, the tool should fire when a sub-task has resolved or when the trajectory is converging, and should be suppressed mid-derivation or when the model is stuck. The authors state that both elements are necessary: the tool alone is used unevenly across open-weight models, often at unhelpful moments or not at all, while the rubric alone cannot take any action. Combined, they are said to elicit effective adaptive compaction with no fine-tuning and no external supervision.

The empirical work covers six benchmarks spanning competitive math and agentic search, evaluated across seven models. The reported results say SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost. Measured against a no-summarization baseline, the method improves results by up to 18.1 points on math and by 5 to 9 points on agentic search, while cutting per-question cost by 30 to 70%.

The authors frame the finding as evidence of a meta-cognitive gap: unprompted models cannot reliably tell when their own context is rotting, but a lightweight rubric closes that gap. Their conclusion is that deciding when to compact can be treated as a capability a scaffold can supply without training.

The benchmarks and model list are described in aggregate rather than broken out per model or per task, and the reported cost figures are per-question reductions relative to a no-summarization baseline rather than absolute token counts. The improvement figures are likewise stated as upper bounds on math and as a range on agentic search. No model weights, harness code or dataset details are described in the material.

For developers, the practical point is that this is a prompting-and-scaffolding technique, not a fine-tuned model. The two components map onto things an agent loop already has: a summarization tool the model can call, and instructions that tell it what state of work justifies calling that tool. In practice, that means the behavior change lives in the system prompt and tool schema, and the main engineering work is writing a rubric that distinguishes resolved sub-tasks from mid-derivation states, then verifying the model actually invokes the tool at those moments.

A likely trade-off is added unpredictability. Fixed-interval compaction is trivial to reason about, easy to budget for and easy to reproduce; model-decided compaction shifts that control to the model's own judgment, which the authors themselves note is unreliable without the rubric. Teams that need hard token ceilings or deterministic replay may still prefer threshold-based triggers, possibly as a fallback if self-directed compaction does not fire.

The cost claims also point at a common billing asymmetry. Agent runs are usually charged per token and can spend most of a long trajectory re-reading a context that has stopped being useful, so a scaffold that compacts at the right moment reduces spend even when answer quality stays flat. Whether that holds on a given workload depends on how often the rubric fires and how much detail each summary preserves, which the reported aggregate numbers do not isolate. Independent reproduction across other model families and longer tool-heavy tasks would show how much of the 18.1-point math gain comes from summary quality as opposed to the timing of the summaries.

Why it matters: teams building long-running agents inherit the context-window problem directly, and this work suggests a prompt-level fix, a compaction tool plus a firing rubric, that can be adopted without retraining. If the reported 30-70% cost reduction holds on their tasks, agent products could run longer trajectories for less. The caveat is that the same research shows models cannot reliably judge their own context decay unaided, so the rubric is not optional; inference, but the quality of that rubric may matter as much as the model choice.

Read at Apple Machine Learning Research

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SELFCOMPACT, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time el