🤖 AI

Context as an Environment: Programmatic Context Management for Long-Horizon Agents

The paper introduces Scroll, a programmatic context management system that treats agent sessions as executable environments with an append-only event log and a persistent Python kernel, enabling LLMs to dynamically manage long-horizon state through code while achieving state-of-the-art performance on long-context benchmarks.

Original authors: Yin Lin, Elaine Ang, Erkang Zhu, Bolin Ding, Jingren Zhou

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yin Lin, Elaine Ang, Erkang Zhu, Bolin Ding, Jingren Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a conversation that never ends. You are talking to a very smart computer program, one that can help you write code, plan complex trips, or research deep topics. At first, the conversation is easy. The program remembers everything you have said in the last few minutes. But as the conversation stretches over hours or days, the amount of information grows. Eventually, it becomes too much for the program to hold in its immediate memory. In the world of artificial intelligence, this immediate memory is called a context window. It is a fixed size, like a desk that can only hold a certain number of papers. When the pile of papers gets too high, the program must throw some away to make room for new ones.

The problem is that the program does not know which papers are important until it is too late. If you ask a question about something you mentioned three days ago, the program might have already discarded that paper. Traditional solutions try to solve this by summarizing the old papers into a short note or by picking out a few facts to keep in a separate notebook. But these methods are risky. If the program throws away a specific detail, like a date or a number, that detail is gone forever. It cannot be recovered. For tasks that require looking back at a long history to find exact facts or to track how a situation changed over time, losing even a single detail can cause the program to fail.

A team of researchers has proposed a different way to handle this growing conversation. They call their system Scroll. Instead of trying to fit the entire history into the program's immediate memory, they treat the history as a separate, permanent environment that the program can visit whenever it needs to. Think of it as a vast, organized archive that sits just outside the program's desk. The program does not read the whole archive at once. Instead, it writes a small instruction, like a search query or a command, to go into the archive, find the specific page it needs, and bring just that page back to the desk.

The core of this system is a permanent record of every single interaction, called an Event Log. This log is like a tape recorder that never stops, capturing every message, every tool result, and every decision in exact detail. It is stored safely outside the program's immediate memory, so nothing is ever lost. Alongside this log is a persistent workspace, a kind of digital sandbox where the program can keep its current thoughts and calculations. When the program needs to remember something from the past, it does not rely on a summary. It writes a piece of code to search the Event Log, find the exact moment it is looking for, and load that information into its workspace.

This approach changes how the program thinks about its own memory. Instead of guessing what to keep, the program decides what to look up at the moment it is needed. If the program is planning a trip and needs to remember a preference the user mentioned weeks ago, it writes a command to search for that specific preference. It then loads the exact words the user used and uses them to make a decision. The program only brings the necessary information into its immediate view. The rest of the conversation history remains safely stored in the archive, waiting to be retrieved if the program needs it later.

To keep the immediate view from getting too full, the system has a way to clear out old information that is not currently being used. When the workspace gets crowded, the system moves the oldest parts of the conversation back into the archive. However, unlike other systems that might delete or summarize these old parts, Scroll keeps them exactly as they are. It creates a small map, an index, that tells the program exactly where in the archive those old parts are stored. If the program later realizes it needs to check an old conversation, it can use this map to jump directly to the right spot and retrieve the original text. This ensures that the program can always go back to the source, even if the information is no longer in its immediate view.

The researchers tested this system on several difficult challenges designed to see how well an artificial intelligence can handle long conversations. In one test, the system had to answer questions based on a history of over one million words of conversation. In another, it had to act as an agent that makes decisions over a long period, using tools to gather information and solve problems. The results showed that this new method was highly effective. On a test measuring how well the system could remember and reason over long histories, it achieved a score of 94.8 percent. On another test involving ten million words of history, it scored 73.1 percent, which was higher than any other system previously published. In a test where the system had to manage a growing environment while solving complex tasks, it reached 86.7 percent, far surpassing the best previous results.

The success of this system comes from its ability to treat memory as a programming task. By allowing the program to write its own instructions for finding and using information, the system leverages the program's growing ability to write code. It does not force the program to compress its history into a summary that might lose important details. Instead, it gives the program the tools to access the full, unedited history whenever it is needed. The researchers found that this method works well across different types of powerful programs, suggesting that the ability to manage long-term memory through code is a general skill that can be learned by various systems.

This work suggests a shift in how we build intelligent agents. Rather than trying to make the program's immediate memory larger or better at summarizing, we can give it access to a permanent, searchable record of its own life. The program learns to navigate this record, deciding what to bring forward and what to leave behind. This keeps the immediate view clear and focused, while ensuring that the full truth of the past is always available. The researchers believe that this approach could help artificial intelligence handle even longer and more complex tasks in the future, allowing it to learn from a much deeper history without losing the details that matter.

Technical Summary: Scroll – Programmatic Context Management for Long-Horizon Agents

1. Problem Formulation

Long-horizon tasks for Large Language Model (LLM) agents—such as repository-level software engineering and deep web research—generate interaction histories that far exceed a single model's context window. Existing solutions primarily rely on context compression (truncating, summarizing, or discarding stale spans) or external memory (extracting facts into a separate store). Both approaches are fundamentally lossy: they commit to what information to preserve before future needs are known. Consequently, if a summary fails to capture a specific detail or an extraction misses a nuance, that information becomes unrecoverable, even if the raw log exists on disk. This limitation hinders agents from performing exact historical comparisons or nontrivial computations over past events where the required information is not known in advance.

2. Methodology: The Scroll Framework

The paper introduces Scroll, a context manager that reframes session history not as text to be serialized into prompts, but as an executable Session Environment. This environment allows the agent to programmatically interact with its own history.

Core Components

Append-Only Event Log ( $L_t$ ): A durable, ground-truth record of all interactions (user messages, model responses, tool calls, results). Every event is assigned an immutable, monotonically increasing sequence address (seq). The log is stored in SQLite, utilizing BM25 for deterministic full-text search rather than embeddings.
Durable Storage ( $P_t$ ): Raw payloads (e.g., full tool outputs, artifacts) are stored externally (JSON, filesystem) and referenced by the Event Log via deferred handles. This ensures the log remains lightweight while preserving full fidelity.
Persistent Python Kernel ( $V_t$ ): A sandboxed Python kernel persists across model calls. It maintains a typed namespace of resident variables (Python objects, DataFrames, deferred handles). Crucially, intermediate computations and tool results remain in this kernel and are not serialized into the prompt unless explicitly requested.

Programmatic Interface

The model constructs its context by writing code executed within the persistent kernel. The interface, exposed via a capability object ms, factors context construction into four operations:

LOCATE: ms.search(query, k, ...) finds candidate Event Log records via full-text search and structured filters, returning stable seq addresses.
MATERIALIZE: ms.expand(seq) recovers exact turns or spans and loads externalized payloads as Python objects.
COMPUTE: The model performs filtering, joining, aggregation, and derivation using standard Python and permitted database interfaces on resident variables.
EXPOSE: print(value) is the only mechanism that crosses the boundary into the model's working view. Only explicitly printed projections enter the next model context.

Eviction and Navigation

As the working view approaches the token budget, Scroll employs an eviction procedure that removes stale spans from the view but never from the Event Log.

Recoverability: Evicted events remain verbatim in the Event Log.
Eviction Index: To prevent the need for full-log searches, Scroll maintains a tiered index of "headlines" (task, state, next action) for evicted spans. This index allows the agent to navigate directly to evicted regions using address anchors, rolling up older entries into coarser ranges to maintain $O(k \log k n)$ complexity.

3. Key Contributions

The paper makes three primary contributions:

Deferred Selection Policy: Unlike existing methods that fix the context map before future needs are known, Scroll defers the selection of the working view to query time. The model writes a program $\pi_t$ at each step to decide what to recall, compute, and expose.
Executable Context Substrate: The implementation of a persistent, sandboxed Python kernel combined with an append-only Event Log. This shifts context management from text processing to programming, leveraging the LLM's coding abilities to search, transform, and materialize state.
Lossless Navigation Algorithm: An eviction mechanism that keeps the working view bounded without losing history. Evicted spans remain intact and are indexed by compact, address-anchored entries, allowing the agent to navigate directly to evicted regions rather than searching the full log.

4. Experimental Results

The authors evaluated Scroll using Qwen3.8-Max as the backbone across three benchmarks:

LongMemEvalS (Long-term Memory Retrieval): Scroll achieved 94.8% accuracy, performing competitively with state-of-the-art systems.
BEAM10M (10M Token History): Scroll achieved 73.1%, surpassing the best published memory system by 5.1 points. It notably excelled in categories requiring exact historical evidence, such as knowledge update (92.5%) and contradiction resolution (88.1%), where lossy summarization typically fails.
LOCA256K (Long-Horizon Acting): Scroll achieved 86.7% accuracy. In direct comparison with other context-management strategies using the same backbone, Scroll demonstrated superior stability as context grew, with only a 2.6% drop from 128K to 256K tokens, compared to drops of ~21% for summarization and retrieval agents.

Ablation Studies confirmed that the lossless Event Log is critical (removing it caused a drop to ~20% on BEAM10M), the persistent kernel improves performance by 7.3 points by enabling in-kernel aggregation, and the eviction index provides a 1.8-point gain by aiding navigation in scattered histories.

Cost Efficiency: Scroll exposes only a small fraction of the corpus to the model. On BEAM10M, the median input token count was ~105K (approx. 1% of the corpus), with output tokens an order of magnitude smaller.

5. Significance and Claims

The paper claims that Scroll fundamentally changes context management from a compression problem to a programming task. By treating the session as an executable environment, the agent inherits the improving coding abilities of LLMs to manage its own context dynamically.

The authors emphasize that Scroll does not dictate what state to keep or how to write retrieval code; these decisions are left to the model. This flexibility allows the system to scale with the model's capabilities. The paper suggests that successful trajectories can be distilled from frontier models to smaller ones via supervised fine-tuning, keeping the underlying context mechanisms fixed.

The work posits that for long-horizon tasks requiring exact historical evidence or complex computation over past events, lossless retrieval via programmatic access is superior to pre-computed summaries or fixed memory stores, as it avoids the irrecoverable loss of information inherent in current compression paradigms.