Technical Summary: ENPMR-Bench

Problem Statement

While memory-augmented language agents are increasingly deployed in affective applications like emotional support, existing research predominantly treats memory as a mechanism for factual retrieval. This approach overlooks the critical role of memory in shaping users' emotional experiences. In emotional support scenarios, users often express needs indirectly, requiring agents to infer latent emotional needs and proactively retrieve memories that are emotionally salient rather than merely semantically similar. Current benchmarks fail to evaluate this capability, as they typically rely on Question-Answering (QA) paradigms where users explicitly state retrieval targets, or they utilize static user modeling that lacks comprehensive representations of life goals and personal experiences. Consequently, there is a lack of systematic evaluation for Emotional Need-aware Proactive Memory Retrieval (ENPMR), a core component of emotional intelligence defined as the ability to infer latent needs and retrieve appropriate memories to foster empathetic interaction.

Methodology

The authors introduce ENPMR-Bench, a benchmark designed to systematically evaluate the capacity of agents to perform ENPMR. The methodology is grounded in Maslow's hierarchy of needs and involves the following key components:

• Theoretical Framework: The benchmark defines structured mappings between four levels of unmet user needs (Physiological, Love and Belonging, Esteem, and Self-Actualization) and specific types of supportive memories (e.g., Preferences, Relationships, Highlights, Goals, Powers).
• Data Construction:
  • User Profiles: Generated using a persona repository, enriched with ten distinct life themes and structured memory entries categorized into Highlights, Powers, Relationships, Goals, and Preferences.
  • Dialogues: Over 1,800 memory-augmented emotional support dialogues were constructed. These simulate scenarios where an agent must infer a latent need from a user's expression of distress and proactively recall a specific memory to provide support.
  • Negative Sampling: To ensure specificity, the construction process employs negative sampling, selecting memory entries with similar themes but different details as distractors to prevent generic thematic alignment.
  • Validation: Data quality was ensured through a two-stage filtering process using GPT-4o (based on the BLRI scale) and human validation by experts, checking for appropriateness, relevance, and correctness.
• Evaluation Protocol: The benchmark evaluates both Memory Retrieval (selecting the correct memory from a bank) and Response Generation (producing an empathetic response grounded in the selected memory).
  • Models Tested: The study evaluates various embedding models (e.g., Qwen3, gte-Qwen2), state-of-the-art LLMs (GPT-4o, DeepSeek-v3, Qwen-Max, Gemini-2.5-Flash), and Agentic Memory systems (Mem0, MemOS, Memu).
  • Metrics: Retrieval performance is measured via Recall, Precision, and nDCG. Response generation is evaluated on Fluency, Humanoidness, Information, and Empathy (adapted from psychological scales). A novel metric, Memory Type Match Rate (TMR), assesses whether the retrieved memory type aligns with the inferred emotional need.

Key Results

Experimental results reveal substantial deficiencies in current retrieval paradigms and their impact on downstream empathetic performance:

• Retrieval Limitations: Even the best-performing embedding models achieve only 46.41% Recall@10, with Top-1 accuracy remaining below 10% across all models. This indicates that semantic similarity alone is insufficient for identifying emotionally salient memories.
• Impact on Empathy: The quality of retrieved memory significantly dictates the empathy score of the generated response. For instance, DeepSeek-v3 scores 4.38 in empathy with retrieved memories but improves to 4.91 when provided with "golden" (ground truth) memories.
• Systematic Biases: LLMs exhibit systematic biases in memory selection. For "Love and Belonging" needs, models frequently over-select "Highlight" memories rather than the more appropriate "Relationship" memories. This misalignment correlates with lower empathy scores for this specific need dimension.
• Effect of CoT: Chain-of-Thought (CoT) prompting improves the alignment between inferred needs and retrieved memories to some extent, but a significant performance gap remains compared to the golden memory upper bound.
• Agentic Systems: While Agentic memory systems (e.g., MemOS) show moderate improvements over RAG-based approaches by aggregating historical interactions, they still fall short of the performance achieved with golden memories.

Significance and Claims

The paper claims that ENPMR-Bench provides a critical diagnostic tool for identifying limitations in current agents' emotional intelligence. The authors assert that:

1. Memory Appropriateness is Paramount: The ability to select the right type of memory based on inferred emotional needs is more crucial for empathetic support than surface-level generation capabilities or simple semantic retrieval.
2. Current Agents are Inadequate: Despite advances in memory retrieval and response generation, current agents fail to align user emotional needs with appropriate memory types, often due to systematic selection biases rather than random errors.
3. Benchmark Necessity: Existing benchmarks relying on explicit user queries fail to capture the nuances of emotional support. ENPMR-Bench fills this gap by evaluating proactive, need-aware retrieval in long-term, personalized scenarios.

The authors conclude that future research must move beyond factual retrieval to develop need-sensitive memory mechanisms that can dynamically align with the psychological state of the user to facilitate truly personalized emotional support.