Technical Overview: HyMem – Hybrid Architecture for Memory with Dynamic Retrieval Planning

Problem Statement

Large Language Model (LLM) agents currently struggle with long-context dialogues due to inefficient memory management. Existing approaches face a fundamental trade-off between efficiency and effectiveness:

• Compression-based methods (e.g., distilling raw text into lightweight summaries) reduce computational overhead but risk losing critical details, leading to "forgetting" in complex or detail-dependent queries.
• Raw text retention methods preserve information but incur unnecessary computational costs for simple queries and fail to mimic the flexible, hierarchical retrieval patterns observed in human cognition.
• Static retrieval mechanisms in current systems do not adapt to diverse problem scenarios and often rely on fixed workflows that cannot dynamically allocate resources based on query complexity.

The identified core limitation lies in the reliance on monolithic memory representations and static retrieval strategies, which prevent agents from achieving the "cognitive economy" observed in human memory systems—where the retrieval of cost-effective summaries is prioritized for routine tasks, while deep, resource-intensive reasoning is reserved for complex needs.

Methodology: HyMem Architecture

The authors propose HyMem, a hybrid memory architecture designed to dynamically plan memory resources based on task requirements. The system integrates four core components:

1. Dual-Granularity Memory Storage

HyMem divides raw dialogues into discrete units and constructs a two-tier memory structure:

• Memory Tier 1 (Summary): A lightweight, vectorized summary of each event unit containing core information (context, time, location, participants) while omitting redundant dialogue details. This serves as the primary entry point for efficient retrieval.
• Memory Tier 2 (Raw Text): The original, uncompressed dialogue text corresponding to each event unit.
• Linkage: Explicit links are established between Tier 1 and Tier 2 units, enabling the system to trace back from a summary to the full raw text when deeper context is required.

2. Dynamic Retrieval Planning

The inference process applies a principle of cognitive economy to adaptively route queries:

• Lightweight Module: For each query, the system first attempts to retrieve relevant Tier 1 summaries via vector similarity (cosine similarity). This module generates a lightweight context and attempts to produce an answer with minimal latency and token consumption.
• Deep Memory Module (Conditional Activation): If the lightweight module detects "forgetting" (incomplete context due to retrieval failure or information loss), it dynamically activates the deep module.
  • Two-Stage Retrieval: The deep module first performs a coarse-grained retrieval to narrow the candidate scope, followed by an LLM-guided self-retrieval mechanism. This mechanism analyzes the semantic relevance, logical connection, and causal dependency between the query and the Tier 1 summaries to select the most critical events.
  • Traceback: Once key events are identified, the system retrieves the corresponding Tier 2 raw text to restore comprehensive context for the generator.

3. Reflection Module

Before finalizing an answer, a reflection module performs a completeness check. It autonomously evaluates whether the query has been decomposed into minimally answerable units and whether each unit is addressed.

• If the answer is incomplete, the module decomposes the problem, reformulates the query, and triggers a new retrieval round (iterative reasoning).
• This mechanism specifically targets complex, multi-step queries where a single retrieval operation might only yield superficial answers.

Main Contributions

1. Hybrid Architecture: The proposal of HyMem, which anchors memory management in the principle of cognitive economy, utilizing dual-granularity storage (summary vs. raw text) and dynamic demand-based planning to align efficiency with reasoning performance.
2. Adaptive Retrieval Mechanism: The introduction of an LLM-guided self-retrieval system that dynamically activates deep memory access only when needed, combined with a self-reflection process for iterative refinement. This enables precise information localization without the constant overhead of deep retrieval.
3. Empirical Validation: Comprehensive experiments demonstrate that the approach significantly improves answer flexibility and reliability across diverse scenarios compared to static baselines.

Experimental Results

The authors evaluated HyMem on the LOCOMO and LongMemEval benchmarks using various LLM backbones (e.g., gpt-4o-mini, gpt-4.1-mini).

• Performance: HyMem achieved state-of-the-art results, particularly in multi-step and open-domain tasks. On LOCOMO, it outperformed full-context baselines in multi-step (88.16% vs. 77.70%) and open-domain tasks (77.08% vs. 71.88%). On LongMemEval-S, it achieved a leading score of 78.80%.
• Efficiency: Dynamic planning significantly reduced computational costs. HyMem lowered computational costs by 92.6% compared to full-context approaches while maintaining or exceeding their accuracy.
• Latency and Tokens: Compared to baselines like Mem0 and SimpleMem, HyMem reduced retrieval-stage latency by 83.3% and 74.9%, respectively. It achieved the best balance between performance and token consumption, using significantly fewer tokens than naive RAG approaches with equivalent or higher retrieval counts.
• Ablation Studies: Removing the deep memory module and Tier 2 storage led to a 17.40% drop in accuracy, confirming the necessity of raw text for complex reasoning. Conversely, removing the lightweight module tripled token consumption (from ~1.5k to ~3.2k) with only a marginal accuracy gain, validating the efficiency of the hybrid approach.

Significance and Claims

The work claims that HyMem establishes a new state-of-the-art balance between efficiency and performance in long-term memory management for LLM agents. By moving away from static, monolithic memory representations, the framework demonstrates that adaptive, resource-aware memory systems are crucial for enabling robust, long-term, and efficient agent interactions.

The authors emphasize that their work addresses the "efficiency-performance conflict" by mimicking human cognitive economy: prioritizing cost-effective summaries for routine tasks while reserving expensive, deep reasoning resources for complex challenges. This approach not only mitigates the "forgetting" associated with aggressive compression but also avoids computational waste by processing raw text for simple queries. The framework is presented as a fundamental step toward scalable dialogue agents capable of handling ultra-long contexts without prohibitive computational costs.