🤖 AI

PEEK: Context Map as an Orientation Cache for Long-Context LLM Agents

PEEK is a system that enhances long-context LLM agents by caching reusable orientation knowledge about recurring external contexts in a small, persistent "context map," which significantly improves reasoning accuracy and efficiency while reducing computational costs compared to existing baselines.

Original authors: Zhuohan Gu, Qizheng Zhang, Omar Khattab, Samuel Madden

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhuohan Gu, Qizheng Zhang, Omar Khattab, Samuel Madden

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective hired to solve a series of different cases, but every single case involves investigating the same massive, 50,000-page filing cabinet in a basement.

In the past, when AI agents (smart computer programs) tried to solve these problems, they had two main ways of handling that filing cabinet:

The "Memory Lane" Approach: They kept the entire conversation history of every previous case in their head. But after a few cases, their brain got so full of old chatter that they couldn't focus on the new clues. It was like trying to find a specific needle in a haystack that was also filled with every conversation you've ever had.
The "Search Engine" Approach: Every time they got a new question, they ran to the filing cabinet, searched for a few relevant pages, and brought them back. This was fast, but they kept forgetting how the cabinet was organized. They had to re-learn the layout every single time.

The Problem:
The paper argues that for repeated work on the same large context, the AI is missing a crucial tool: Orientation Knowledge.

Think of it like a human analyst who has worked in that basement for a month. They don't need to re-read the whole cabinet every time. They have a mental map: "Oh, the sports stats are in the red binders on the left, the financial data is in the blue ones on the right, and there's a weird rule about how dates are written that I learned on day one."

The paper calls this missing tool a Context Map.

Enter PEEK: The "Peek" into the Cabinet

The authors created a system called PEEK (which stands for Context Map as an Orientation Cache).

Think of PEEK as a sticky note that lives permanently on the AI's desk (inside its prompt). This sticky note is small and has a strict size limit, so it can't hold the whole filing cabinet. Instead, it holds a summary map of the cabinet.

Here is how PEEK works, using a creative analogy of a Library Intern:

1. The Distiller (The "Sniffer")

After the AI solves a problem, the Distiller looks at the AI's messy work log. It asks: "What did the agent actually learn about the library's layout?"

It ignores the specific answer to the question (e.g., "The answer is 42").
It focuses on the orientation (e.g., "The sports section is always in the back," or "The dates are written in a weird format").
It filters out the noise and keeps only the reusable "map" knowledge.

2. The Cartographer (The "Map Maker")

The Cartographer takes those notes and turns them into a clean, structured update for the sticky note.

If the AI learned that "Sports" is in the red binders, the Cartographer writes that down clearly.
If the AI realized a specific rule about dates, it adds that to a "Constants" section.
It makes sure the map doesn't get cluttered with facts that only apply to one specific question.

3. The Evictor (The "Janitor")

The sticky note has a limited size (a token budget). If the AI learns something new, the Evictor has to decide what to throw away to make room.

It uses a priority system. It keeps the most important "map" info (like the layout of the building) and throws away less important details (like a specific date format that was only useful for one question).
This ensures the note stays small, fast, and always relevant.

Why is this a big deal?

The paper tested PEEK on two types of difficult tasks:

Reasoning: Finding clues scattered across the massive document and putting them together.
Learning: Figuring out the rules of the document and applying them to new questions.

The Results:

Smarter: PEEK solved problems much more accurately than the old methods (improving scores by 6% to 34%).
Faster: Because the AI didn't have to re-learn the layout of the filing cabinet every time, it needed far fewer attempts (iterations) to get the right answer.
Cheaper: Fewer attempts meant the system cost significantly less money to run (up to 5.8 times cheaper than the previous best method).

The Bottom Line

The paper claims that for AI agents working repeatedly on the same large dataset, the secret isn't just "remembering more" or "searching faster." It's about keeping a small, curated map of the world right in front of them.

PEEK gives the AI a "peek" into the context that stays with it, allowing it to navigate the massive external world efficiently, just like a seasoned librarian who knows exactly where every book is without having to re-read the catalog every day.

Technical Summary: PEEK - Context Map as an Orientation Cache for Long-Context LLM Agents

Problem Statement

Large Language Model (LLM) agents increasingly operate over long, recurring external contexts, such as document corpora or code repositories, where the external data remains static while the tasks change. Existing approaches to managing these contexts fall into four categories:

Shared Chat/History: Preserves the agent's trajectory but accumulates noisy, low-density context.
History Compaction: Summarizes past activity but discards reusable knowledge about the external context itself.
Passive Access (RAG, Offloading, Compaction): Provides access to raw or condensed material but lacks a curated, persistent artifact about the context's structure and organization.
Prompt Learning: Curates task-level strategies but does not maintain contextual knowledge about the recurring external environment.

The authors argue that a critical gap exists: the lack of a bounded, managed artifact that preserves reusable orientation knowledge (e.g., what the context contains, how it is organized, which entities/constants are useful) for repeated workloads on the same context.

Methodology: PEEK System

The authors introduce PEEK, a system that maintains a Context Map—a small, constant-sized artifact residing in the agent's system prompt. This map acts as an "orientation cache," providing the agent with a persistent "peek" into the external context. The system is inspired by hardware caches and database indexes but operates at the agent level, focusing on curated meaning rather than token state compression.

PEEK employs a programmable cache policy consisting of three modular components that update the map after each query execution:

Distiller: Analyzes the agent's execution trajectory (reasoning steps, tool calls, observations) to extract transferable contextual knowledge. It distinguishes between orientation work (understanding the context's structure, which is reusable) and task-specific work (answering a specific question, which is not). It outputs a diagnosis, tags for existing map items, and candidates for new cache entries.
Cartographer: Translates the Distiller's output into structured edit operations (ADD, DELETE, REPLACE) for the Context Map. It ensures updates are local, traceable, and deduplicated, preventing the map from becoming noisy or duplicative.
Evictor: Enforces a fixed token budget ( $B$ ) for the map. It uses a priority-based eviction strategy, removing lower-value items (e.g., parsing schemas, error patterns) before higher-value items (e.g., domain constants, context understanding) when the budget is exceeded.

The Context Map itself is structured into sections:

Context Roadmap: An index of content and location.
Context Understanding: High-level descriptions, key entities, and relationships.
Domain Constants: Exact values, formulas, and schemas.
Reusable Results: Derived computations applicable to multiple queries.
Parsing Schema: Format and delimiter structures.

The system supports an "evolve steps" parameter ( $m$ ), allowing the map to be updated for the first $m$ queries and then frozen for reuse, or updated fully online ( $m=n$ ).

Key Contributions

Conceptual Framework: The paper defines the "Context Map" as a distinct quadrant in the design space of context management, filling the gap of active external-context state.
System Design: PEEK decouples knowledge extraction (Distiller) from structural editing (Cartographer), a design choice shown via ablation to be critical for preventing task-specific facts from polluting the cache.
Efficiency Mechanism: By maintaining a small, constant-sized map, PEEK avoids the super-linear token growth associated with shared chat history and the high latency of full-context compaction.

Experimental Results

The authors evaluated PEEK on two benchmarks: OOLONG (long-context reasoning and aggregation) and CL-bench (context learning). The base agent used was Recursive Language Models (RLM) with GPT-5-mini.

Performance Gains:

Quality: PEEK outperformed all baselines, including the State-of-the-Art (SOTA) prompt-learning framework ACE.
- On OOLONG, PEEK improved scores by 6.3–34.0% over RLM and 7.8–15.0% over ACE.
- On CL-bench, PEEK improved solving rates by 6.0–14.0% and rubric accuracy by 7.8–12.1% over RLM, and 6.0% / 9.9% over ACE.
Efficiency (Iterations): PEEK required 93–145 fewer iterations than ACE on OOLONG while achieving higher quality. It consistently occupied the high-quality, low-iteration region of the Pareto frontier.
Cost: PEEK incurred 1.7–5.8× lower cost than ACE on OOLONG and 1.4× lower cost on CL-bench. This was achieved by reducing the number of iterations and avoiding the verbose, token-heavy adaptation loops of ACE.

Generalization:

Model Independence: Gains were consistent when swapping the base LLM to GPT-5.5 and the open-source Qwen3-Coder-Next-FP8.
Agent Independence: PEEK improved performance when applied to a different backbone agent, OpenAI Codex, demonstrating that the context map is a generalizable artifact.

Ablation Studies:

Removing the Distiller (using a monolithic update) reduced performance by ~7.7%, confirming the necessity of separating diagnosis from editing.
Removing the Evictor (freezing the map at budget) still yielded gains over the base RLM, but the full policy added an additional ~10.2% improvement, highlighting the value of active maintenance.
Varying the cache size ( $B$ ) showed that the presence of a map matters more than its exact size, with even small budgets (512 tokens) providing significant improvements.

Significance and Claims

The paper claims that PEEK demonstrates that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently. The authors argue that:

Orientation Knowledge is Transferable: Unlike raw conversation history, knowledge about the structure and organization of a corpus is reusable across different tasks.
Active Management is Superior: A small, curated, and actively maintained artifact outperforms both passive retrieval (RAG) and unmanaged accumulation (Shared Chat).
Cost-Quality Tradeoff: PEEK achieves a superior cost-quality tradeoff, lying on the Pareto frontier where other methods (like ACE) either cost significantly more for lower quality or require more iterations for similar results.

The authors conclude that this approach opens a new research agenda for agent-side caching, distinct from model-level KV-cache optimizations, specifically targeting the semantic understanding of persistent external environments. They note that the current work is a first step and call for future benchmarks that natively support repeated queries over the same long context to further stress-test this paradigm.