Technical Summary: SaliMory: Orchestrating Cognitive Memory for Conversational Agents

1. Problem Definition

Conversational agents designed as lifelong companions require persistent memory to maintain context across interactions. However, current approaches face two primary limitations:

• Retrieval-Augmented Generation (RAG) Noise: Simply expanding context windows or retrieving raw interaction histories often leads to noisy, repetitive, and irrelevant data that degrades reasoning quality.
• Credit Assignment Bottlenecks: Training memory agents via standard Reinforcement Learning (RL) using only final outcome rewards creates a severe credit assignment problem. In multi-stage pipelines, it is difficult to determine whether a failure in the final response stems from poor memory formation, ineffective retrieval, or generation errors.

Error analysis on the LoCoMo benchmark reveals that memory errors (loss or distortion of salient information) account for up to 52% of failures in state-of-the-art baselines, significantly outpacing errors in profile generation or response generation.

2. Methodology: The SaliMory Framework

SaliMory is an agentic memory-management framework that trains a single language model to manage a cognitively structured memory system end-to-end. It is inspired by human cognitive science, specifically the distinction between working memory, long-term factual knowledge, and subjective preferences.

2.1 Cognitive Memory Architecture

Instead of a flat buffer, SaliMory maintains a structured memory bank $M = (\bar{F}, \bar{P}, \bar{W})$ consisting of three dedicated stores:

1. Factual Bank ($\bar{F}$): Objective, verifiable statements about the user (e.g., "lactose intolerant").
2. Preference Bank ($\bar{P}$): Subjective tastes and leanings acting as soft criteria for stylistic tailoring (e.g., "prefers oat milk").
3. Working Memory ($\bar{W}$): Recent, salient turns the user likely still has in mind.

2.2 Unified Policy and Roles

A single policy model $\pi_\theta$ is trained via RL to perform three complementary roles, conditioned on distinct instruction prompts:

• Saliency Filter: Decides which conversation turns contain salient information worth recording.
• Memory Booster: Updates and consolidates the factual and preference banks, and manages the sliding window of working memory.
• Memory Utilizer: Retrieves and synthesizes relevant entries from the three stores into a query-adapted profile for the downstream generator.

The architecture separates the offline memory management stage from the runtime generation stage. A frozen, large language model ($\pi_{gen}$) generates the final response based on the synthesized profile, ensuring the memory manager is optimized specifically for context curation.

2.3 Training: Stage-Wise Process Rewards and Contrastive Refinement

To solve the credit assignment problem, SaliMory introduces two novel mechanisms:

A. Stage-Wise Process Reward System
Instead of relying solely on a final outcome reward, SaliMory decomposes the learning signal into three orthogonal rewards:

1. Response Quality ($R_1$): A multiplicative gate evaluating final answer accuracy and personalization. Personalization is only rewarded if the answer is factually correct.
2. Memory Quality ($R_2$): An asymmetric penalty evaluating the factual fidelity and vagueness of the memory bank itself, independent of the final answer. This encourages the booster to omit marginal memories rather than encoding vague ones.
3. Utilization and Filter Quality ($R_3$): Evaluates the effectiveness of the adapted profile (capturing constraints without distractors) and the accuracy of the saliency filter's gating decisions.

B. Reward-Decomposed Contrastive Refinement
Standard Group Relative Policy Optimization (GRPO) assigns a single advantage scalar to all traces, which can lead to unearned gradients for upstream modules. SaliMory constructs role-specific contrastive preference pairs:

• Booster Pairs: Selected strictly based on the memory quality score ($R_2$), ignoring the total episode reward.
• Utilizer Pairs: Selected based on utilization scores ($R_3$), but only formed if the memory banks of the compared rollouts are highly similar (via Jaccard similarity). This causal gating ensures the utilizer is only rewarded for its synthesis ability, not for receiving better upstream memories by chance.

3. Key Contributions

1. Cognitively Inspired Architecture: The separation of factual snapshots, subjective preferences, and short-term working memory, managed by a unified agent performing distinct roles (filter, booster, utilizer).
2. Memory-Anchored RL: A novel training paradigm combining hierarchical stage-wise process rewards with reward-decomposed contrastive refinement to provide fine-grained credit assignment for multi-stage memory systems.
3. New Benchmark and Protocol: The introduction of LoCoMo-P13n, an extension of the LoCoMo benchmark with personalizable queries (recommendation and implicit personalization), and a multi-step evaluation protocol that jointly assesses precise memory recall and personalization quality.

4. Experimental Results

Evaluated on the LoCoMo and LoCoMo-P13n benchmarks using a 9B parameter model (Qwen3.5-9B-Instruct) as the policy:

• End-to-End Performance: SaliMory achieves 72.9% end-to-end accuracy, outperforming the state-of-the-art (SOTA) by over 10%.
• Personalization: It delivers a 39.8% "Good Personalization" rate, more than doubling the performance of the next best baseline (Mem-R1 at 16.3%) and significantly outperforming zero-shot agentic methods powered by much larger models (e.g., Gemini-3-Flash).
• Error Reduction: SaliMory reduces memory-attributed failures by one-third compared to baselines. It nearly eradicates "Bad Personalization" (hallucinated or intrusive responses) to 1.4%.
• Memory Quality: The system reduces vague memory entries to 19.6% (vs. 35.7% for Mem-R1) and increases factual fidelity to 93.9%.
• Efficiency: SaliMory achieves a 76x speedup in memory formation latency (0.5s vs. 38s for zero-shot) and a 5x reduction in inference latency by synthesizing a compact profile rather than processing raw retrieval chunks.
• Generalizability: In real-world Chat AI traffic data, SaliMory more than triples the Good Personalization rate (1.7% to 5.5%) and significantly improves memory factuality.

5. Significance and Claims

The paper claims that the fundamental challenge in conversational memory is not storage, but management. SaliMory demonstrates that by grounding memory architecture in cognitive principles (separating facts, preferences, and working memory) and solving the credit assignment problem through decomposed rewards and contrastive refinement, it is possible to train compact models to outperform massive zero-shot models.

The authors assert that their approach provides the necessary "isolated supervision" for distinct memory operations, enabling agents to reliably capture user preferences while preserving salient details. This framework is presented as a critical step toward building effective lifelong digital companions that can maintain persistent, high-quality context without being overwhelmed by interaction history.