TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents

1. Problem Statement

Long-horizon conversational agents face a critical bottleneck: interaction histories grow unbounded, quickly exceeding the finite context windows of Large Language Models (LLMs). Existing memory frameworks often fail to address two key issues:

• Lack of Temporal Structure: Most systems rely on semantic similarity for retrieval, treating time as auxiliary metadata. This leads to fragmented memories where evidence from different periods is aggregated without clear temporal boundaries, causing unstable personalization.
• Inefficient Consolidation: Current methods either store raw context (inefficient) or use parametric approaches (limited by model architecture) without explicitly transforming episodic details into stable, abstracted persona representations.

The core challenge is to transform long-term, dynamic experiences into compact, temporally grounded representations that support both factual accuracy and stable persona modeling.

2. Methodology: The TiMem Framework

TiMem introduces a Temporal-Hierarchical Memory framework that treats temporal continuity as a first-class organizing principle. It operates without requiring additional fine-tuning of the underlying LLM. The framework consists of three core components:

A. Temporal Memory Tree (TMT)

The TMT is a hierarchical structure that organizes memories across five abstraction levels, enforcing explicit temporal containment (parent intervals cover child intervals):

1. Segments (L1): Fine-grained factual summaries of individual dialog turns.
2. Sessions (L2): Merged, non-redundant event summaries for a specific conversation session.
3. Daily (L3): Captures routine contexts and recurrent interests within a day.
4. Weekly (L4): Integrates evolving behavioral features and preference patterns.
5. Profile (L5): An incrementally refined, stable persona representation (updated monthly) capturing core values, personality, and long-term preferences.

B. Memory Consolidation

This module converts raw dialog into structured memories using instruction-guided prompting rather than fine-tuning.

• Process: A "Memory Consolidator" takes child memories (from the level below), historical context (a sliding window of same-level memories), and level-specific instruction prompts to generate the next level of abstraction.
• Scheduling:
  • Online: L1 segments are generated immediately after every user-assistant exchange.
  • Scheduled: Higher levels (L2–L5) are generated automatically when their temporal windows (e.g., end of day, week, month) close.

C. Complexity-Aware Memory Recall

Instead of retrieving all memories, TiMem adapts the retrieval scope based on query complexity:

1. Recall Planner: An LLM analyzes the user query to classify it as Simple (factual), Hybrid (multi-fact integration), or Complex (reasoning/prediction). It determines which TMT levels to search (e.g., Simple queries skip intermediate layers and go straight to facts and profiles; Complex queries traverse the full hierarchy).
2. Hierarchical Recall:
   • Leaf Selection: Uses dual-channel scoring (semantic similarity + lexical matching) to select relevant L1 segments.
   • Propagation: Collects the ancestors of selected leaves up to the levels specified by the planner.
3. Recall Gating: A final filtering step where an LLM decides which candidate memories to retain based on the query and complexity, removing redundancy and irrelevant noise before feeding the context to the answer generator.

3. Key Contributions

1. Temporal Memory Tree (TMT): A novel structure that enforces explicit temporal containment and granularity, organizing memories from raw segments to abstract profiles.
2. Instruction-Guided Consolidation: A plug-and-play framework that progressively abstracts raw dialog into patterns and personas using prompt engineering, requiring no model fine-tuning.
3. Complexity-Adaptive Recall: A dynamic retrieval mechanism that balances precision and efficiency by adjusting the search scope and filtering depth based on query complexity.
4. Comprehensive Evaluation: Demonstrates state-of-the-art performance with significant efficiency gains, supported by manifold analysis showing improved persona separation and noise suppression.

4. Experimental Results

TiMem was evaluated on two long-term conversational benchmarks: LoCoMo and LongMemEval-S.

• Accuracy:
  • LoCoMo: Achieved 75.30% accuracy (LLM-as-Judge), outperforming the best baseline (MemOS at 69.24%).
  • LongMemEval-S: Achieved 76.88% accuracy, surpassing all baselines.
• Efficiency:
  • TiMem reduced the recalled memory length by 52.20% on LoCoMo compared to baselines (511 tokens vs. ~1070 tokens for Mem0), significantly lowering inference costs and latency.
• Manifold Analysis:
  • On LoCoMo, hierarchical consolidation improved user separation (clustering quality increased 6.2×), effectively distilling unique persona features.
  • On LongMemEval-S, consolidation reduced spatial dispersion by 50%, suppressing noise while retaining core attributes.
• Ablation Studies: Confirmed that the full hierarchy (L1–L5) combined with the adaptive planner and gating yields the best results. Removing the hierarchy or the gating mechanism led to significant performance drops or increased context length.

5. Significance

TiMem represents a paradigm shift in long-horizon agent memory design:

• Temporal Continuity as a Constraint: By prioritizing time over pure semantic similarity, TiMem constructs coherent evidence chains essential for counterfactual reasoning and stable personalization.
• Scalability: The ability to compress raw dialog into high-level profiles allows agents to maintain long-term context without linearly increasing token usage.
• Interpretability: The hierarchical structure provides a clear trace from raw facts to abstract traits, making the agent's "memory" more transparent and debuggable.
• Practicality: The framework is model-agnostic and requires no fine-tuning, making it immediately applicable to existing LLM backends for building robust, long-term personalized companions.