💻 computer science

ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents

ContextWeaver is a memory framework for LLM agents that improves long-context reasoning by organizing interaction history into a dependency-structured graph of reasoning steps, allowing for selective, summarized, and validated context retrieval.

Original authors: Yating Wu, Yuhao Zhang, Sayan Ghosh, Sourya Basu, Anoop Deoras, Jun Huan, Gaurav Gupta

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yating Wu, Yuhao Zhang, Sayan Ghosh, Sourya Basu, Anoop Deoras, Jun Huan, Gaurav Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a massive, complex mystery in a giant mansion.

As you move from room to room, you find clues: a muddy footprint in the kitchen, a broken vase in the library, and a mysterious note in the attic.

The Problem: The "Goldfish" Detective

Current AI "agents" (the digital detectives) have a problem called "Context Window" limits. Think of this like the detective having a very small notebook. As they find more clues, they run out of pages. To keep going, they usually do one of two things:

The Sliding Window (The "Forgetful" Method): They simply rip out the oldest pages to make room for new ones. The problem? They might rip out the page containing the first clue (the muddy footprint), forgetting that the footprint is the only reason they are looking in the attic in the first place!
The Summarizer (The "Lazy" Method): They try to write a tiny summary of everything they've seen so far. But in their rush, they might write "Found some clues in the kitchen," forgetting the crucial detail that the footprint was left-footed.

When the detective forgets the "why" behind their actions, they start walking in circles, repeating mistakes, or looking in the wrong rooms entirely.

The Solution: ContextWeaver (The "Web of Clues")

The researchers created ContextWeaver. Instead of a simple notebook, imagine the detective now has a giant, glowing spiderweb hanging in the center of the mansion.

Every time the detective finds a new clue, they don't just write it down; they tie a string from that new clue back to the specific old clues that led them there.

ContextWeaver works using three clever tricks:

1. The Dependency Web (The "Why" String)

Instead of just remembering things in the order they happened (First, Second, Third...), ContextWeaver remembers them by logic.

Example: "I am looking at this broken vase (New Clue) because I saw the muddy footprint in the kitchen (Old Clue)."
The AI builds a "map" of connections. If it needs to solve a problem in the attic, it doesn't just look at the last thing it did; it follows the strings back to the most important "root" clues.

2. The Smart Summarizer (The "Highlight Reel")

Since the AI can't remember everything in high detail, ContextWeaver creates "mini-stories" along the strings. Instead of a messy pile of notes, it creates a streamlined narrative: "Started at the kitchen $\rightarrow$ found footprint $\rightarrow$ traced to library $\rightarrow$ found broken vase." This keeps the "story" of the investigation short and easy to read, saving precious space.

3. The Reality Check (The "Truth Filter")

The AI also has a built-in "Fact Checker." If the detective tries to pick a lock and fails, ContextWeaver marks that attempt as "FAILED." When building the web, the AI is told: "Don't tie your new ideas to failed attempts; only tie them to things that actually worked or gave us real information." This prevents the AI from getting stuck in a loop of repeating its own mistakes.

Does it actually work?

The researchers tested this on "SWE-Bench," which is basically a high-level exam where AI agents have to fix real-world computer code (a very messy, "mansion-sized" task).

The results showed:

Better Success: The AI was better at solving complex problems that required connecting dots across different files.
Less Wandering: The AI didn't repeat the same mistakes as often.
More Efficient: It reached the solution using fewer "steps" and less "brain power" (tokens) because it wasn't wasting time re-reading things it already knew or following dead ends.

In short:

ContextWeaver turns an AI from a forgetful worker who only cares about what happened just now, into a strategic thinker who understands how every single step is connected to the big picture.

Technical Summary: ContextWeaver

ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents

1. Problem Statement

Large Language Model (LLM) agents, particularly those performing complex software engineering tasks, face significant challenges in long-context interactions. As agents use tools and interact with environments, their interaction history grows rapidly, often exceeding context limits or degrading model performance.

Existing context management strategies—such as sliding windows (focusing on recency), prompt compression (distilling salience), and retrieval-based methods (semantic similarity)—suffer from a critical flaw: they fail to capture the causal and logical dependencies between reasoning steps. In agentic workflows, a current action often depends on a specific earlier decision, a previous tool output, or a hypothesis generated many steps ago. When these dependencies are lost due to recency-based pruning, agents often repeat mistakes, break ongoing plans, or lose track of cross-file architectural requirements.

2. Methodology

The authors propose ContextWeaver, a framework that organizes an agent’s interaction history into a Directed Acyclic Graph (DAG) of reasoning steps rather than a linear stream of tokens. The framework consists of three core components:

Dependency-Aware Context Construction:
- Node Extraction: Each reasoning step (Thought, Action, Observation) is converted into a structured node $N_k$ .
- Parent Selection: An LLM-based Logical Dependency Analyzer identifies which earlier nodes ( $S_k$ ) are logically or causally required by the current node. This is treated as a reasoning task (identifying information flow) rather than a simple similarity search.
- Ancestry Dependency Construction: The system performs a breadth-first search (BFS) upward through the graph to identify the "ancestry set" ( $A$ )—all nodes that directly or indirectly support the current step.
- Context Weaving: The final context is "woven" by providing full details for nodes in the ancestry set and highly compressed versions (omitting large observations) for non-ancestor nodes.
Dependency Summarizer:
To maintain interpretability without massive token overhead, the system incrementally constructs "dependency summaries." Instead of re-summarizing the entire history, it combines the current node's summary with the summaries of its parents, creating a compact narrative of the reasoning path from the root to the current step.
Validation and Test Layer:
The framework incorporates execution feedback (e.g., from pytest). A TestTracker parses tool outputs, and a ValidationDetector assigns status labels (passed, failed, unknown, or superseded) to nodes. This ensures that the agent does not build new reasoning steps on top of "failed" or "superseded" (obsolete) states.

3. Key Contributions

Structural Memory: Moves beyond recency/similarity-based memory to a dependency-based graph structure that preserves the "why" behind agent actions.
Incremental Summarization: Introduces a method to condense long-range reasoning paths into reusable, compact units, reducing computational costs.
Execution-Feedback Integration: A lightweight validation layer that uses real-world tool outcomes to prune unreliable nodes from the reasoning graph.
Efficiency-Performance Balance: Demonstrates that an agent can maintain high performance within strict token budgets by being selective about what information is kept, rather than just how much is kept.

4. Results

The framework was evaluated on the SWE-bench Verified and Lite benchmarks using models like Claude 3.5 Sonnet and GPT-4o.

Improved Performance: In a "Hybrid" setting (using a strong model to build the graph for a task-performing model), ContextWeaver significantly outperformed the sliding-window baseline. For example, GPT-5 achieved 58.6% on Verified compared to 57.4% for the baseline.
Stability and Robustness: ContextWeaver showed lower variance across multiple runs, suggesting it provides a more stable reasoning foundation.
Token Efficiency: While the total tokens per run remained similar to the baseline, ContextWeaver reduced the tokens per successful resolve, meaning it achieved more "work" per token spent.
Iteration Reduction: The agent required fewer iterations to solve tasks, as the structured context helped avoid redundant exploration and "drifting" from the original goal.
Qualitative Success: Case studies showed that ContextWeaver excels in multi-component tasks (requiring coordination across different files) where sliding windows fail due to losing early architectural context.

5. Significance

ContextWeaver represents a shift in LLM agent research from context management (how to fit more text) to context organization (how to structure information). By modeling the logical flow of reasoning, it provides a scalable mechanism for agents to handle long-horizon, complex tasks. This approach is particularly significant for specialized domains like software engineering, where the ability to link a current code edit to a root cause identified dozens of steps earlier is the difference between success and failure.