🤖 machine learning

Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents

The paper introduces SLEUTH, a language agent framework that scales multi-hop reasoning by replacing implicit context with an explicit, structured epistemic working memory (tracking confirmed facts, ranked hypotheses, and open questions), which significantly improves performance on complex benchmarks and reveals that organizing reasoning state is more critical than raw model capability for overcoming context dilution and evidence sufficiency issues.

Original authors: Ning Liu

Published 2026-07-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ning Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery that requires connecting four different clues. You have a notebook, but it's not a normal notebook. Every time you find a new clue, you scribble it on a piece of paper and stick it into a giant, growing stack of papers. The problem? As the stack gets taller, the clues you found at the very bottom get buried under the new ones. By the time you reach the top, you've forgotten what you found at the start. This is exactly what happens to many AI "agents" when they try to solve complex, multi-step questions: they get lost in their own history, a problem the authors call context dilution.

The paper introduces a new method called SLEUTH (Structured Layered Evidence-Updated Tracking of Hypotheses) to fix this. Instead of letting the AI's brain just be a giant, messy stack of papers, SLEUTH forces the AI to keep a very specific, organized "working memory" that looks like a detective's whiteboard. This whiteboard has three distinct sections that the AI must update after every single step:

Confirmed Facts: Things the AI has actually found and verified, with a note on where it found them (like a source citation).
Active Hypotheses: Possible answers the AI is guessing, ranked by how much evidence supports them (High, Medium, or Low confidence).
Open Questions: The specific things the AI still needs to find out to solve the mystery, which directly tell the AI what to search for next.

The authors tested this on five different tricky question-answering challenges, including HotpotQA and MuSiQue, which range from 2-step to 4-step reasoning chains. They found that as the questions got harder, SLEUTH got much better at solving them compared to other methods. On the hardest 4-step questions, SLEUTH improved the success rate by +11.1 points over the standard method (ReAct) when using the same AI model.

Here is the really cool part: The paper argues that how the AI organizes its thoughts matters more than how "smart" the AI model is. To prove this, the researchers took a weaker AI model (GLM-5) and forced it to use the SLEUTH notebook structure. This weaker model, when organized, actually beat a much stronger model (Claude Sonnet) that was not using the structure. The weaker model with the notebook scored 48.9% on the hardest problems, while the stronger model without it only scored 42.0%. This suggests that a simple, structured way of thinking can outperform raw computing power.

However, the paper also points out a new problem that SLEUTH sometimes creates: the "Evidence Sufficiency Problem." Because the AI is so good at organizing its facts, it sometimes keeps searching for more proof even after it already has enough to solve the puzzle. It gets stuck in a loop of "just one more check," running out of its time limit (called a "turn budget") before it ever gives an answer.

To fix this, the authors added a "nudge." Imagine a coach blowing a whistle when the detective has used up 70% of their allowed time. The whistle tells the AI: "Stop searching! You have enough facts; now use what you have to write the answer." This simple rule helped the AI stop wasting time. When they tested this, the AI's success rate jumped from 49.1% to 53.1% on the hardest problems.

Crucially, the paper shows that this "nudge" only works if the AI has the organized notebook. If they gave the same "stop searching" whistle to an AI with a messy, unorganized brain, it didn't help at all (the score stayed at 42.2%). The organized state is the necessary ingredient; the nudge just helps the AI use it.

The authors also tested how well this works with different amounts of time and different search settings. They found that SLEUTH's advantage holds up whether the AI can only find one paragraph per search or five. They also noted that the method doesn't require any special training or new hardware; it works just by changing the instructions (the "prompt") given to the AI.

In short, the paper suggests that for AI agents to get better at solving long, complex puzzles, we shouldn't just make the AI bigger or smarter. Instead, we should teach it to keep a tidy, structured notebook of what it knows, what it guesses, and what it still needs to find. This simple change in organization allows even weaker models to solve harder problems than stronger ones that are left to their own messy devices.

Technical Summary: SLEUTH – Structured Epistemic Working Memory for Multi-Hop Reasoning

1. Problem Statement

Language agents that interleave reasoning with tool use (e.g., ReAct) exhibit a sharp performance degradation as reasoning chains lengthen, even when individual sub-tasks are straightforward. The authors identify the root cause as context dilution: the agent's investigative state (what is confirmed, what is suspected, and what remains to be determined) exists only implicitly within a growing context window. As the chain extends, early discoveries are buried under later retrievals, causing the agent to lose track of resolved sub-questions, viable hypotheses, and necessary next steps. This leads to two primary failure modes:

Information Loss: Confirmed evidence is lost to context dilution.
Evidence Sufficiency Paralysis: Agents gather sufficient evidence to answer but fail to commit, exhausting their turn budget on needless verification due to a lack of a principled stopping criterion.

2. Methodology: SLEUTH

The paper introduces SLEUTH (Structured Layered Evidence-Updated Tracking of Hypotheses), a framework that makes an agent's epistemic state explicit and actionable through a structured working memory. Implemented entirely via prompting (no fine-tuning or architectural changes), SLEUTH requires the agent to maintain and update a tripartite state $W_t = (F_t, H_t, Q_t)$ after every tool call:

Confirmed Facts ( $F_t$ ): A monotonically accumulating set of verified statements extracted from tool outputs, each grounded to a specific source. Once confirmed, facts are never retracted within an episode, preventing information loss.
Active Hypotheses ( $H_t$ ): Candidate answers ranked by evidentiary support (High/Medium/Low confidence). Each hypothesis is linked to the facts that support or contradict it.
Open Questions ( $Q_t$ ): Uncertainties whose resolution would most significantly alter hypothesis rankings. Each question includes a proposed action (e.g., a search query) and a priority level. The highest-priority question directly drives the agent's next tool call.

The Commitment Mechanism (Nudge)

To address the evidence sufficiency problem, SLEUTH employs a lightweight nudge mechanism. This trigger fires when two conditions are met:

The agent has consumed a fraction $\alpha$ (set to 0.7) of its turn budget.
The agent has accumulated at least $\tau$ (set to 2) confirmed facts.

When triggered, the agent receives a synthesis instruction to construct a final answer from its current structured state without further search. This forces the transition from exploration to exploitation, preventing budget exhaustion on already-resolved chains.

3. Key Contributions

Explicit Epistemic State: SLEUTH operationalizes the human practice of maintaining structured working notes (facts, hypotheses, questions) for language agents. Unlike methods that externalize state via code or training, SLEUTH relies solely on prompting.
Scaling with Difficulty: The framework's advantage grows with reasoning complexity. It outperforms established baselines (ReAct, Reflexion, Self-Ask, IRCoT) across five benchmarks, with gains widening from +4.7 points on 2-hop questions (HotpotQA) to +11.1 points on 4-hop chains (MuSiQue).
Isolation of Failure Modes: The paper identifies and addresses two distinct failure modes:
- Information Loss is mitigated by structured fact-tracking.
- Evidence Sufficiency Paralysis is mitigated by the commitment nudge.
  Crucially, the authors demonstrate that the nudge is ineffective without the structured state; applying the nudge to an unstructured agent (ReAct) yields no improvement, isolating organized epistemic state as the necessary condition for effective commitment.
Protocol Adherence over Model Capability: By enforcing strict adherence to the working memory protocol on a weaker model (GLM-5), the authors recover up to +19 points on the hardest problems. This demonstrates that how an agent organizes its reasoning is a more critical factor for scaling multi-hop reasoning than raw model capability.

4. Experimental Results

Experiments were conducted on HotpotQA, MuSiQue (2/3/4-hop), and 2WikiMultiHopQA using Claude Sonnet 4.6 and GLM-5.

Performance: SLEUTH achieves the highest Exact Match (EM) scores on all datasets. On MuSiQue 4-hop, it reaches 53.1% EM compared to 42.0% for ReAct (a +11.1 point gap).
Ablation Studies:
- Maintaining only structured facts (Facts-Only) recovers most of the gain, confirming that preventing context dilution is the dominant effect.
- Adding hypotheses or questions in isolation provides minimal benefit; the full tripartite structure is required for synergistic gains.
- The commitment nudge alone (applied to ReAct) provides no benefit (+0.2 points), whereas it boosts SLEUTH by +4.0 points.
Cross-Model Generalization: SLEUTH's benefits scale with difficulty even on weaker models. When protocol adherence is enforced on GLM-5, it outperforms the unstructured ReAct running on the stronger Sonnet model on 4-hop problems (48.9% vs. 42.0%).
Error Analysis: SLEUTH significantly reduces timeout rates (from 17.8% to 11.1% on 4-hop) and reasoning errors. The remaining errors are primarily due to inference quality over correctly retrieved evidence, not retrieval or state management failures.

5. Significance and Claims

The paper argues that as agents tackle increasingly complex tasks, the bottleneck shifts from reasoning capability to reasoning organization. SLEUTH demonstrates that substantial performance gains can be achieved without additional training, parameters, or trajectory search, simply by structuring the agent's internal state.

The central lesson is that protocol adherence is the active ingredient: a weaker model held to a rigorous structured protocol outperforms a stronger model operating with implicit, unstructured state. The authors position epistemic state management as a design dimension complementary to scaling model size or test-time compute. They note that while the framework addresses information loss and commitment paralysis, it does not solve the fundamental challenge of inference quality over correctly retrieved evidence, which remains a bottleneck for future work.