PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Prediction

Technical Summary of PraMem: Practice-derived Experiential Memory for Long-horizon Behavior Prediction

Problem Statement
Long-horizon behavior prediction aims to infer a user's next action based on lengthy historical sequences of scene-action records. While Large Language Models (LLMs) offer a promising autoregressive approach, they face two core challenges in this domain:

1. Latent Pattern Induction: Fine-grained factors influencing user actions (e.g., specific preferences for brand, price, or quality) are hidden and scattered across ultra-long sequences, making them difficult for LLMs to capture accurately on the fly.
2. Model-Intrinsic Cognitive Biases: LLMs exhibit inherent biases during prediction, such as a tendency to follow the majority, training-induced prejudices, or disproportionate attention to recent records (proximal bias), which degrade prediction accuracy even when relevant information is provided.

Existing memory management methods primarily follow a "context-compression" paradigm, attempting to alleviate the burden of long sequences through extraction, aggregation, and retrieval. However, the authors argue this approach fails because compression does not equate to behavioral pattern induction, and these methods lack mechanisms to perceive or correct the LLM's intrinsic cognitive biases.

Methodology: PraMem
The authors propose PraMem, a training-free framework that shifts the paradigm from viewing historical sequences as a burden to treating them as a valuable resource for "practice." Inspired by dialectical philosophy, PraMem conducts beforehand practice over the historical sequence to build a user-specific, time-evolving Experiential Memory. This memory serves as assisted input for accurate prediction and consists of two components:

• Pattern Experience: Summaries of the user's latent behavioral patterns (e.g., "This user prefers Apple electronics").
• Bias-alert Experience: Alerts regarding the LLM's intrinsic cognitive biases when reasoning about this specific user (e.g., "Avoid proximal bias; do not over-rely on the last action").

The framework maintains this memory through an iterative loop comprising three stages:

1. Existing Experience Trial: The system samples segments from the historical sequence to construct practice samples with ground-truth labels. The LLM performs explicit "deep thinking" using the current experiential memory to predict the action, exposing shortcomings in the existing experience.
2. Reflective Proposal Generation: The LLM reflects on the trial results (comparing predictions to ground truth) to generate proposals for revising, pruning, or supplementing the memory.
3. Consensual Experience Adjustment: To ensure stability and avoid noise from occasional behaviors, adjustments are triggered periodically. The system adopts only operations that are consistently supported by multiple proposals in the pool.

Key Innovation: The Self-Review Mechanism
A critical component of PraMem is a self-review mechanism designed to ensure the quality of reflective proposals before they enter the proposal pool. This mechanism filters out proposals that lack groundedness or generalizability:

• Groundedness Review: The system perturbs the historical sequence of a practice sample to substantially alter the underlying behavioral patterns. If a proposal still holds after this perturbation, it is deemed not genuinely grounded in the data and is rejected.
• Generalizability Review: The system rewrites the prediction scene to generate multiple virtual scenes with similar structures but different entities. If a proposal can unambiguously identify only the original scene among these options, it is deemed overly specific and rejected. Only proposals that are both well-grounded and generalizable are retained.

Experimental Results
The authors evaluated PraMem on two benchmarks: OminiBehavior (covering video browsing, live streaming, ads, and e-commerce) and MovieLens-1M.

• Performance: PraMem significantly outperformed strong baselines, including Long-context inputs, Truncation, RAG, Summary-based methods, and advanced memory systems like Mem0, MemOS, and ProEx. On OminiBehavior, PraMem improved accuracy (ACC) from 73.5% (Truncation) to 84.7% and F1-score from 24.7 to 31.6.
• Generalizability: The experiential memory constructed by PraMem demonstrated transferability across different LLM backbones (GPT-OSS-120B and Qwen3.5-35B-A3B), indicating that the distilled patterns and bias alerts are robust.
• Evolution: Analysis showed that performance monotonically improved with more practice rounds, while the memory token count stabilized after an initial increase, avoiding the "length explosion" common in other memory accumulation methods.

Significance and Claims
The paper claims that PraMem offers a promising direction for long-horizon behavior prediction by reframing the historical sequence as a resource for iterative practice rather than a compression target. The authors assert that:

• PraMem successfully addresses the dual challenges of latent pattern induction and cognitive bias mitigation, which prior context-compression methods failed to resolve.
• The "practice-derived" approach allows the model to induce reliable behavioral patterns and self-correct biases, leading to superior prediction accuracy.
• The self-review mechanism is essential for ensuring the experiential memory remains faithful to the data and applicable to unseen scenarios.

The authors acknowledge a limitation regarding the construction cost, noting that the iterative practice process requires more time offline compared to extraction-based methods, though this does not impact online prediction latency. They emphasize that the method is training-free and relies on the LLM's inherent reasoning capabilities enhanced by this structured memory evolution.