Technical Summary: MemWeaver

Problem Statement

The landscape of user-internet engagement is shifting from reliance on implicit feedback signals (e.g., clicks, browsing) to the utilization of rich, explicit textual interactive behaviors. While this shift provides a high-fidelity narrative of user interests, prevailing personalization approaches remain limited. Traditional methods, rooted in ID-based collaborative filtering, fail to interpret the semantic content of language. Conversely, current Retrieval-Augmented Generation (RAG) approaches often treat user history as a flat, unstructured list of text chunks. These methods typically rely on simplistic retrieval heuristics (e.g., recency or similarity) and fail to model the intricate temporal evolution of user interests and the latent semantic connections between disparate topics. Consequently, existing systems struggle to achieve deep personalization that aligns with a user's dynamic and nuanced preferences.

Methodology: MemWeaver

To address these limitations, the authors propose MemWeaver, a framework that weaves a user's entire textual history into a hierarchical memory comprising two complementary components: Behavioral Memory and Cognitive Memory.

1. Memory Construction

The framework transforms raw, chronologically ordered user history ($H_u$) into a structured dual-memory system:

• Behavioral Memory (Concrete Context):

  • Graph Construction: User behaviors are represented as nodes in a graph. Edges are established via temporal links (connecting consecutive behaviors) and semantic links (connecting behaviors within the same thematic cluster, identified via $K$-means clustering on embeddings).
  • Extraction: To retrieve relevant context for a specific query, the system employs a context-aware random walk. This process simulates human associative recall, transitioning between nodes based on a score that balances semantic relevance to the query with temporal biases (recency and continuity). This yields a dynamic subset of the history grounded in specific, query-relevant actions.

• Cognitive Memory (Abstract Preferences):

  • Hierarchical Abstraction: The user's history is first partitioned into temporally coherent phases based on semantic breakpoints.
  • Summarization: A two-stage LLM process synthesizes these phases. First, local summaries are generated for each temporal segment. Second, these summaries are integrated into a single, global narrative that abstracts the user's long-term, evolving preferences and core traits.

2. Memory-Augmented Generation

The final generation process conditions the Large Language Model (LLM) on three inputs: the user query, the extracted Behavioral Memory, and the abstracted Cognitive Memory.

• The Behavioral Memory provides concrete, in-context examples to ground the response in specific user actions.
• The Cognitive Memory offers high-level guidance regarding long-term preferences to ensure global consistency.
• This joint conditioning allows the model to balance short-term relevance with long-term user alignment.

3. Incremental Updating

To ensure scalability, MemWeaver employs lightweight incremental update mechanisms. New behaviors are integrated into the graph without full reconstruction, and the cognitive memory is updated by synthesizing new local summaries with existing ones. This approach bounds token costs and enables real-time adaptation to evolving interests.

Key Contributions

1. Novel Framework: The proposal of MemWeaver, which moves beyond flat logs to construct a hierarchical memory from textual interactive behaviors.
2. Dual-Memory Methodology: The introduction of a specific architecture featuring an extracted Behavioral Memory for concrete context and an abstracted Cognitive Memory for evolving preferences.
3. Empirical Validation: Extensive experiments on the LaMP (Language Model Personalization) benchmark across six datasets (covering classification and text generation tasks) demonstrate state-of-the-art performance.
4. Synergy Analysis: In-depth analysis confirming that the synergy between concrete behavioral grounding and abstract cognitive guidance is essential for deep personalization.

Experimental Results

Experiments were conducted on the LaMP benchmark using Qwen3-8B and Llama-3.1-8B as backbone LLMs. MemWeaver outperformed strong baselines, including Vanilla (no history), Random, Recency, BM25, BGE, and advanced retrieval methods (ROPG, CFRAG).

• Performance: MemWeaver secured the top performance across all twelve evaluation metrics (Accuracy, F1, MAE, RMSE, ROUGE-1, ROUGE-L) on all six datasets.
• Ablation Studies:
  • Removing either the Behavioral or Cognitive memory significantly degraded performance, confirming their indispensable and synergistic roles.
  • Within the Behavioral Memory, semantic edges were found to be more critical than temporal edges for performance, though both are necessary. The edge weighting mechanism was identified as the most critical component.
  • Within the Cognitive Memory, the hierarchical segmentation and global summary steps were both proven essential for capturing interest evolution and core traits.
• Efficiency: The incremental update strategy achieved performance comparable to full memory reconstruction at a fraction of the computational cost, validating its suitability for continuous deployment.

Significance and Claims

The paper claims that MemWeaver addresses a critical gap in personalization: the lack of a framework capable of deriving hierarchical memory from textual interaction histories. By explicitly modeling both the temporal evolution of interests and the semantic relationships between activities, the framework enables LLMs to reason over both concrete behaviors and abstract cognitive traits.

The authors assert that this dual-memory approach leads to content generation that is deeply aligned with latent user preferences, moving beyond the limitations of ID-based models and flat-text retrieval. The work demonstrates that a comprehensive user representation requires the joint modeling of temporal correlations and semantic relationships, with the behavioral memory grounding the model in reality and the cognitive memory providing the necessary high-level guidance for coherent, personalized outputs.