💻 computer science

MoEMory: Long-Term User Memory as a Routing Policy on Frozen Mixture-of-Experts Language Models

MoEMory introduces a novel long-term memory paradigm for frozen Mixture-of-Experts language models that stores per-user information as a routing bias to selectively activate existing capabilities without injecting new tokens or parameters, thereby achieving near-zero leakage into unrelated queries while remaining orthogonal and complementary to traditional injection-based memory methods.

Original authors: Chenxi He

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chenxi He

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a digital assistant that remembers you. It recalls your favorite coffee order, the way you prefer to speak, and the specific topics you enjoy. For years, the prevailing idea for giving computers this kind of long-term memory has been to force new information into their brains. Researchers have tried two main ways to do this. The first is like reading a diary aloud to the computer before every question, injecting the text directly into its immediate attention. The second is like rewiring the computer's internal circuits, permanently altering its weights to bake the memory into its very structure. Both methods work to some degree, but they share a hidden flaw: they are invasive. By adding new material to the computer's calculation, they risk confusing the machine. A memory about football might accidentally bleed into a question about music, causing the computer to answer a query about a famous singer with the name of a soccer player. This is not just a glitch; it is a fundamental side effect of how these systems are built, and it raises a serious question about whether we can ever have a truly helpful, personalized assistant that doesn't constantly lose its way.

A researcher at the University of Cambridge has proposed a different path, one that avoids this confusion by changing the rules of engagement entirely. Instead of injecting new content into the computer's mind, they suggest using memory as a selector. Their approach, which they call MoEMory, relies on a specific type of artificial intelligence architecture known as a Mixture of Experts. Think of this architecture not as a single, monolithic brain, but as a vast library of specialized tools, or "experts," where only a few are pulled out to work on any given task. In the standard version of these models, the decision of which tools to use is fixed and unchangeable. The researcher realized that they could store a user's memory not by changing the tools themselves, but by slightly adjusting the switch that decides which tools get picked. This switch is a simple bias applied before the selection happens, a tiny nudge that tells the system, "For this specific user, prioritize these particular tools."

The brilliance of this method lies in what it does not do. Because the underlying tools and the computer's core structure remain completely frozen and untouched, the memory cannot invent new capabilities or force the system to say things it doesn't already know how to say. It can only rearrange the order in which existing capabilities are used. If the computer is already capable of writing in a formal tone or discussing sports, the memory simply ensures those specific modes are more likely to be activated for that user. If the user asks a question about a topic completely unrelated to their stored preferences, the system reverts to its default behavior, unaffected by the memory. This creates a form of non-interference by design. The memory is confined to the boundaries of what the frozen computer can already do, preventing the kind of "over-memorization" where a user's specific obsession corrupts unrelated answers.

To test this idea, the researcher compared their routing-based system against the traditional methods of injecting memory. They set up a scenario where a user's memory was installed and then asked the computer a series of questions, some related to that memory and many completely unrelated. The results were stark. The traditional methods, which add content to the computer's context or weights, leaked the stored memory into unrelated questions between 45 and 62 percent of the time. The computer would consistently bring up the user's specific interests even when they were irrelevant. In contrast, the new routing method leaked the memory into unrelated queries only 0.7 percent of the time. This represents a massive reduction in interference, roughly ninety times cleaner than the previous best approaches. The few times the new system did show a trace of the memory, it was exactly in the cases where the theory predicted it would be possible, confirming that the system was behaving exactly as designed.

The researcher also explored how this memory could be applied in two different ways. The first is a static approach, where the memory acts as a permanent disposition, like a user's general style or personality, influencing every single interaction. The second is a dynamic approach, where the memory is triggered only by the specific words in the current question, acting like a gate that opens only when the right key is turned. Both methods work without needing to retrain the computer or add extra processing time during the conversation. The system simply reads the user's history once to set the bias, and then operates with that bias permanently in place. This means the computer can remember a user's preferences without ever needing to store a history log or replay past conversations. It is a lightweight, efficient way to personalize a machine that keeps the machine's core knowledge intact and its behavior predictable.

The findings suggest that the future of personalized artificial intelligence may not lie in making models bigger or more complex, but in finding smarter ways to direct their existing power. By treating memory as a selection policy rather than an injection of data, the researcher has shown that it is possible to create assistants that remember who you are without forgetting how to be helpful in other contexts. This approach respects the boundaries of the machine's capabilities, ensuring that personalization enhances the experience rather than distorting it. While the system cannot store new facts that the computer doesn't already know, it excels at the subtle, dispositional tasks that define a good personal assistant: knowing your tone, your style, and your preferences, and applying them at just the right moment. In a field often obsessed with adding more data, this work offers a quiet, powerful reminder that sometimes, the most effective way to remember is to simply know which parts of yourself to bring forward.

Technical Summary: MoEMory

Problem Statement

Current paradigms for long-term memory in language models rely on injection: either injecting tokens into the context (retrieval-augmented generation) or injecting content into the model weights (per-user LoRA, memory tuning). While these methods are evaluated based on recall (the ability to reproduce stored content), they share a critical, unmeasured flaw: over-memorization. Because injection adds new material to the computation, installed memories often "leak" into unrelated queries, causing the model to inappropriately apply specific user preferences to general tasks (e.g., a football-fan memory causing the model to answer "name a famous musician" with a footballer). This phenomenon represents a form of catastrophic interference where targeted memory corrupts normal, unrelated capabilities. Furthermore, users often find verbatim, persistent recall unsettling, preferring memory that respects context and remains controllable.

Methodology: MoEMory

The author proposes MoEMory, a paradigm shift from memory as injection to memory as selection. Instead of adding content, MoEMory biases which of a frozen model's existing capabilities are activated.

Core Mechanism

MoEMory operates on a frozen Mixture-of-Experts (MoE) language model. It introduces a per-user additive bias, $r_u$ , to the router logits before the top- $k$ selection step.

Frozen Backbone: The model's experts, attention mechanisms, and router weights remain entirely frozen.
No Inference Overhead: No conversation history is stored or replayed at test time. No additional tokens or parameters are added during inference.
Non-Destructive by Construction: Because the bias only re-schedules which frozen experts fire, it cannot generate capabilities or content outside the model's existing "route-closure" (the set of outputs reachable by the frozen experts).

Two Modules

MoEMory implements two distinct routing bias strategies:

Static Module (Always-on Disposition):
- Computes a fixed bias $r_u$ per layer based on the contrastive difference between the user's historical routing signatures and a neutral reference corpus.
- Encodes global traits (language, style, persona) that should apply to every query.
- Has $O(1)$ addressable capacity.
Dynamic Module (Prompt-Gated Selection):
- Computes a query-dependent bias using Personalized PageRank (PPR) on a token–expert graph constructed from the user's history.
- The bias is seeded by the current prompt tokens. If the prompt is unrelated to the history (empty seed), the bias is exactly zero, recovering the base model bit-for-bit.
- Extends capacity to $O(M)$ and acts as a structural support gate to prevent interference.

Key Contributions

Identification of Selection as a Third Paradigm: The paper defines memory as selection (biasing frozen capabilities) as an orthogonal axis to context and parametric injection. It is non-destructive by construction and composable with injection methods.
Definition of Over-Memorization: The author introduces "over-memorization" (leakage into unrelated queries) as a critical, previously unmeasured quality dimension for memory systems, distinct from recall.
Theoretical Bounds:
- Route-Realizability Bound: Proves that a routing bias can only reach outputs already realizable by the frozen experts at the current input. It cannot inject arbitrary content.
- Capacity/Trigger Theory: Formalizes the roles of the static ( $O(1)$ ) and dynamic ( $O(M)$ ) modules.
Empirical Validation:
- Leakage Reduction: Injection-based baselines (Prompt, OPPU-LoRA, Lamini-MT) exhibited 45–62% leakage into unrelated queries. MoEMory reduced this to 0.7%, a $\sim90\times$ improvement.
- Residual Leakage: The minimal leakage observed in MoEMory exactly matched the theoretical "route-realizable" cases predicted by the author's theory.
- Selective Disposition: MoEMory achieved a +0.12 lift in selective disposition effects (engaging memory only when relevant).
- Multi-Memory Frontier: The dynamic module extended the limit of co-loaded memories from $\sim2$ (static budget saturation) to $\sim5$ without interference.

Significance and Claims

The paper claims that MoEMory solves the fundamental trade-off between personalization and capability preservation. By storing memory in the routing policy rather than the content or weights, the system guarantees non-interference structurally rather than through tuning.

The author emphasizes that MoEMory is complementary to, not competitive with, injection-based memory. Existing methods can supply the content (facts, specific text), while MoEMory supplies the control (when and how those capabilities are engaged). This approach aligns with user desires for "soft, contextually-appropriate continuity" rather than intrusive, verbatim recall, offering a mechanism where memory persists without eroding the model's general utility or privacy.