The Past Is Prologue: A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory

Technical Summary: Janus – A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed as sequential agents that learn from past interactions via external memory. In this setting, agents retrieve memory, solve tasks, receive feedback, and update their memory for future use. While this "retrieve–solve–update" loop is intuitive, existing sequential memory systems suffer from a fundamental limitation: they typically deploy every locally generated memory update without verifying its global utility.

The paper identifies that updates optimized for the immediate task can be detrimental to the final memory state. Specifically, they may:

• Overwrite previously useful knowledge.
• Introduce noisy, over-specific rules.
• Bias the memory toward recent examples, degrading performance on future unseen tasks.

This creates a memory validation problem: determining when to compare a candidate memory against the current one, and what tasks to use for that comparison, without incurring the prohibitive computational cost of replaying the entire task history.

2. Methodology: Janus

The authors propose Janus, a method-agnostic, plug-in memory controller that wraps existing memory updaters. Janus does not alter the underlying update rules; instead, it acts as a gatekeeper, deciding whether to accept a candidate memory update ($\hat{M}_t$) or retain the previous memory ($M_{t-1}$).

Janus addresses the validation problem through two core mechanisms:

A. Memory Momentum Trigger (MMT)
To avoid the high latency of comparing memories after every task, Janus treats memory updates as a trajectory. It uses a text encoder $\phi(\cdot)$ to map memory states into a vector space.

• Update Direction: The direction of a candidate update is defined as $z_t = \phi(\hat{M}_t) - \phi(M_{t-1})$.
• Momentum Tracking: Janus maintains an exponential moving average of previous update directions: $m_t = \beta m_{t-1} + (1-\beta)z_t$.
• Trigger Condition: A comparison is triggered only when the candidate update deviates significantly from the recent momentum trajectory, indicated by $\cos(z_t, m_{t-1}) < \tau$.
• Rationale: Small, incremental updates aligned with the momentum are accepted directly. Sharp deviations, which may indicate either valuable new knowledge or harmful distortion, trigger an explicit validation step.

B. Hybrid Trigger-Time Evaluation Set
When a trigger fires, Janus evaluates both $M_{t-1}$ and $\hat{M}_t$ on a compact hybrid set $E_t$ rather than the full history. This set consists of three components:

1. Coverage Set ($S^{cov}_t$): A cluster of representative tasks summarizing the global distribution of all seen tasks.
2. Boundary Set ($S^{bdry}_t$): A collection of "flip" tasks where the choice of memory state previously changed the agent's correctness (i.e., tasks where $1_{corr}(x; M_{t-1}) \neq 1_{corr}(x; \hat{M}_t)$). These are highly sensitive to memory changes.
3. Fresh Set ($F_t$): A sample of tasks encountered since the last trigger to prevent overfitting to a static support set and to test performance on newly explored regions.

The final memory $M_t$ is selected as the one yielding higher performance on $E_t$.

3. Key Contributions

• Identification of a Deployment Challenge: The paper highlights that sequential memory systems often fail because locally generated updates are not globally useful. Blindly accepting updates can lead to final memories biased toward noisy, task-specific information.
• Efficient Plug-in Control: Janus is a method-agnostic controller that wraps existing updaters. It combines the Memory Momentum Trigger with a compact hybrid evaluation set, enabling selective deployment without modifying the base updater or replaying the full history.
• Empirical Validation: The approach is validated across six datasets, two backbone LLMs, and two distinct memory updater architectures.

4. Experimental Results

The authors evaluated Janus on six datasets covering mathematical reasoning (MATH500), scientific reasoning (GPQA), professional STEM (MMLU-Pro), code generation (HumanEval), and tool use (APIBench). The experiments used Qwen3-8B and DeepSeek-V4-Flash as backbones, paired with base updaters like DC-RS and ExpeL.

• Performance Gains: Janus consistently improved average accuracy over the base updaters by +2.7 to +4.6 percentage points.
  • On Qwen3-8B, DC-RS + Janus improved from 79.5% to 83.2% average accuracy.
  • On DeepSeek-V4-Flash, ExpeL + Janus improved from 79.6% to 82.3% average accuracy.
• Ablation Studies:
  • Trigger Timing: The MMT trigger outperformed random and periodic triggering strategies, achieving comparable or better accuracy with significantly fewer comparisons than an "always-trigger" policy.
  • Evaluation Set Composition: Removing any component of the hybrid set (Coverage, Boundary, or Fresh) degraded performance. The "Fresh" set was particularly critical to prevent stale, self-reinforcing comparisons.
  • Deployment Decisions: Intermediate memory checkpoints showed that Janus maintains a stable upward trend in accuracy, whereas base updaters often stagnate or degrade as the task stream progresses, confirming that newer memories are not always better.

5. Significance and Claims

The paper claims that the primary bottleneck in sequential LLM memory is not just how to generate updates, but whether to deploy them. Janus demonstrates that treating memory updating as a deployment decision—filtering locally optimized updates that may harm global generalization—significantly enhances the utility of the final memory.

The authors position Janus as a lightweight, efficient solution that can be integrated into existing sequential memory frameworks without retraining the base model or altering the core update logic. The results suggest that for reasoning assistants and interactive decision-making systems, maintaining a curated, validated memory is more effective than accumulating all generated experiences.