Technical Summary: HMARS

Problem Statement

Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across extensive documents, dialogues, and interaction histories. Standard Retrieval-Augmented Generation (RAG) approaches typically reduce this challenge to a passive top-$K$ chunk retrieval process based on embedding similarity. The authors argue that this passive interface is fundamentally flawed for complex reasoning tasks because it assumes relevance is local and statically measurable. Consequently, RAG often discards relevant evidence before reasoning begins, particularly when:

1. Evidence Breadth: Required information is scattered across distant regions of a document, requiring the integration of many non-adjacent chunks.
2. Context-Dependent Relevance: A passage becomes relevant only when interpreted against broader context or information stated elsewhere (e.g., an early mention of a budget constraint becoming essential to a later question about housing), despite having low lexical overlap with the query.

Existing solutions, such as larger context windows, hierarchical summaries, or multi-agent sequential reading, often remain passive or centralized, failing to provide a true memory system where local regions actively participate in query-conditioned assessment without early exclusion.

Methodology: HMARS

The authors propose HMARS (Hierarchical Multi-Agent Memory System), which treats long contexts as managed memory rather than a flat retrieval corpus. The system employs an asymmetric model hierarchy ($L_{sub} \ll L_{mid} \ll L_{base}$) and operates in two phases:

1. Indexing Phase

• Partitioning: The input document or interaction history is partitioned into bounded local regions.
• Sub-Agents: Each region is managed by a lightweight sub-agent ($L_{sub}$). The region is further divided into fixed-size pages. Each sub-agent creates a Local Memory Card containing a summary, topics, entities, and a page index. This card exposes what the sub-agent knows to the next level without serving as final evidence.
• Mid-Agents: Sub-agents are grouped under mid-agents ($L_{mid}$). A mid-agent aggregates the memory cards of its assigned sub-agents to construct a Shared Regional Memory. This includes a region-level summary, a mapping of sub-agents to their topics/entities, and a cross-reference mechanism to track dependencies (e.g., shared entities, temporal updates) between isolated sub-agents.

2. Query Phase

• Broadcast & Interpretation: Upon receiving a query, each mid-agent interprets the query against its shared regional memory. Instead of making hard routing decisions, it generates a region-level query interpretation ($\gamma$) and specific hints ($h$) for each sub-agent.
• Parallel Local Assessment: Crucially, every sub-agent performs a query-conditioned assessment of its local pages in parallel, guided by the mid-agent's hints. This ensures no document region is discarded by an early top-$K$ filter.
• Grounded Evidence Retrieval: Selected pages are deduplicated, sorted by global token offset, and reconstructed into a grounded evidence context.
• Final Reasoning: A frontier base model ($L_{base}$) performs the final reasoning and answer synthesis strictly over the retrieved grounded evidence pages.

Key Contributions

1. Diagnostic Benchmarks: The authors constructed two benchmarks specifically designed to stress-test evidence breadth and context-dependent relevance, moving beyond simple sequential hops:
   • A Long-Document Benchmark derived from LongBench-v2, filtering for questions requiring evidence from at least four distant chunks with low RAG recall.
   • A Cumulative Interaction Memory Benchmark inspired by RealMem, featuring multi-turn dialogues where key facts are scattered and become relevant only under later context.
2. Problem Framing: The paper frames long-context failures as limitations of passive, chunk-based memory access, specifically highlighting the inability of standard RAG to handle broad evidence coverage and context-dependent relevance.
3. HMARS Architecture: The introduction of a hierarchical system that separates local memory management, regional coordination, and final reasoning, ensuring full-context consideration by local agents while reserving the powerful frontier model for final synthesis.

Experimental Results

HMARS was evaluated against dense RAG (with varying budgets and chunk sizes), dense reranking, full-context prompting, GraphRAG, and agentic baselines (Chain-of-Agents, MemAgent).

• Performance: HMARS achieved the best overall performance on both benchmarks. It outperformed all baselines on the multi-turn memory task and achieved the highest accuracy in most long-document reasoning categories (Synthesis, Comparison, Causal, Cross-reference).
• Evidence Coverage: Analysis revealed that HMARS covers 99.4% of annotated supporting chunks, whereas RAG variants (even with high budgets) covered only ~57%. This confirms that HMARS's gains stem from retrieving required evidence more completely rather than merely changing the final answer prompt.
• Efficiency: Despite using multiple model calls, HMARS offers a superior accuracy-compute trade-off. It achieves higher accuracy than Full-Context prompting (which uses the largest FLOPs) and high-budget RAG, by shifting computation from the expensive frontier model reading the entire context to lightweight local and mid-level models managing memory access.
• Ablation Studies:
  • Full Fanout: Restricting participation to only mid-agent-selected sub-agents significantly reduced accuracy, validating the need for every local region to assess relevance.
  • Mid-Agent Coordination: Removing the mid-agent layer degraded performance, particularly in multi-turn tasks, proving the necessity of shared regional context.
  • Model Allocation: Using a larger model for sub-agents provided no significant gain, supporting the asymmetric design where lightweight models manage local pages.

Significance and Claims

The paper claims that long-context reasoning should not be framed solely as retrieving more chunks or extending context windows. Instead, it argues for building memory systems that actively manage, access, and ground evidence across the input.

The authors assert that HMARS demonstrates that:

1. Passive top-$K$ retrieval fails when evidence is scattered or context-dependent.
2. Organizing inputs as managed local memory, where every region participates in query-conditioned assessment, is more effective than centralized or sequential processing.
3. Asymmetric model allocation allows for full-context consideration without incurring the prohibitive cost of feeding the entire context to a frontier model.

The paper concludes that while HMARS improves evidence access, it does not guarantee perfect recall, as local agents may still miss pages. Furthermore, the authors note that their diagnostic benchmarks specifically target evidence breadth and context-dependence, and the system's benefits are most pronounced in cumulative interaction memory tasks.