Technical Summary: ArborMem

Problem Statement

Large language models (LLMs) are increasingly deployed as persistent conversational assistants that must maintain continuity over days or weeks. While existing approaches improve memory access through long-context windows, selective retrieval, and structured organization (e.g., graphs, trees), they share a critical limitation: they treat memory access primarily as retrieving semantically relevant past information without first determining which prior interaction state the current turn resumes.

In long-running conversations, users often interleave multiple tasks, people, and plans that may be interrupted and later revisited. A relevance-based retriever may conflate semantically similar but structurally distinct threads (e.g., mixing a discussion about a paper revision with a discussion about a conference trip). This "interaction-state localization" failure leads to context confusion, where the model retrieves facts from the wrong trajectory, breaking the continuity required for coherent reasoning.

Methodology: ArborMem

ArborMem is an online memory framework that represents long-running conversations as a navigable forest of interaction states. It decouples interaction continuity from semantic relevance, organizing memory into two primary components:

1. Conversation Forest ($F_t$): A tree-like structure where each node represents a committed local state after a user-assistant interaction.

   • Branches: Each root-to-node path forms a locally coherent trajectory (e.g., a specific task or topic).
   • Navigation: Parent edges encode state continuation. The system determines whether a new input continues an existing branch or initiates a new root.
   • Content: Nodes store original interactions, summaries, entities, and vector representations.

2. Reusable Evidence Store ($M_t$): A separate store for information that can be reused across different trajectories.

   • Atomic Facts: Compact, verified facts (preferences, identifiers, dates).
   • Structured Records: Assistant-generated artifacts, events, and mutable state entries with provenance and revision metadata.

Operational Workflow

For each new input $u_t$, ArborMem executes three stages:

1. State Localization:

   • The system identifies the interaction state under which $u_t$ should be interpreted.
   • It employs a priority cascade: explicit continuation cues attach directly to the active state; temporal-return cues use the global timeline.
   • For ambiguous inputs, it constructs a hybrid candidate set via semantic recall (FAISS), keyword/entity recall, and topic matching.
   • A branch-aware cross-encoder reranks candidates using a representation that includes the node's topic, path summary, and entities, scoring the entire trajectory leading to the node, not just the node in isolation.
   • If the highest score exceeds a threshold $\tau$, the corresponding node is selected as the parent ($p_t$); otherwise, a new root is created.

2. Context Assembly & Generation:

   • Branch-Local Context: If a parent is selected, the system reconstructs the root-to-$p_t$ path. Older turns are compressed into summaries, while recent turns remain raw.
   • Cross-Branch Evidence: The system retrieves supplemental turns, atomic facts, and structured records from $M_t$ that are relevant but outside the active branch.
   • Generation: The LLM generates a response $a_t$ using the assembled evidence bundle $E_t = [C_{local}^t; E_{turn}^t; E_{fact}^t; E_{struct}^t; u_t]$.

3. Online Memory Commit:

   • After generation, the completed interaction $(u_t, a_t)$ is committed.
   • A new state node is attached as a child of $p_t$ (or as a new root).
   • Separate extractors update the node/path summaries, indexes, and the reusable evidence store ($M_t$).
   • Causal Boundary: Information extracted from the current turn is only readable from turn $t+1$, ensuring the current generation depends solely on previously committed states.

Key Contributions

1. Interaction-State Localization: The paper identifies that reliable long-term memory requires determining which prior trajectory is being resumed before selecting historical evidence.
2. ArborMem Framework: A novel architecture that organizes evolving conversational states as a navigable forest, restoring coherent branch-local context while retrieving reusable evidence across branches without conflating distinct trajectories.
3. BranchMemEval: A controlled diagnostic benchmark designed to isolate branch-structured challenges, including:
   • B1 (Backjump): Resuming an earlier trajectory after intervening dialogue.
   • B2 (State Update): Retrieving the latest value within the correct trajectory.
   • B3 (Assistant Artifacts): Recalling lists/records generated by the assistant.
   • B4 (Cross-Branch Bridging): Combining evidence across trajectories.
   • B5 (Twin Threads): Distinguishing structurally similar parallel agendas.

Experimental Results

ArborMem was evaluated on four benchmarks: LongMemEval, LoCoMo, BEAM 100K, and the new BranchMemEval.

• Performance: ArborMem outperformed the strongest baselines (including long-context models, retrieval-based systems like BM25, and structured memory systems like Graphiti, A-MEM, and Mem0) across all benchmarks.
  • LongMemEval: +9.00 percentage points over the best baseline.
  • LoCoMo: +10.31 percentage points.
  • BEAM 100K: +3.36 percentage points.
  • BranchMemEval: +5.00 percentage points over the strongest memory baseline and +15.00 points over full-context baselines.
• Read Budget Efficiency: Under constrained read budgets (256–1K tokens), ArborMem's advantage was most pronounced, exceeding competitors by 16–26 percentage points. This demonstrates that state localization enables more effective evidence selection when only a small portion of memory can be accessed.
• Efficiency: Despite additional routing and context assembly, complete memory queries remain below 0.5 seconds. ArborMem showed moderately faster memory-construction costs (1.363s/turn) compared to Mem0 (1.543s/turn) and more than twice as fast as A-MEM (3.091s/turn).
• Ablation Studies: Removing state localization caused a significant drop in accuracy (from 82.0% to 70.0% on LongMemEval with a 30B model), confirming that globally retrieved facts cannot fully replace coherent branch-local context.

Significance and Claims

The paper claims that interaction-state localization is a fundamental requirement for reliable long-term conversational memory, distinct from simple semantic retrieval. By separating the "where" (trajectory localization) from the "what" (evidence retrieval), ArborMem preserves interaction continuity while enabling information reuse.

The authors emphasize that existing benchmarks often fail to isolate branch-structured challenges, leading to an overestimation of retrieval-based methods. BranchMemEval provides a controlled setting to diagnose these specific failures. The results suggest that for persistent agents, maintaining a navigable structure of interaction states is more effective than merely exposing longer contexts or retrieving semantically similar snippets. The framework supports state-consistent interpretation and long-term information reuse, particularly in scenarios involving interleaved tasks and delayed resumption.