💻 computer science

Nous: A Predictive World Model for Long-Term Agent Memory

Nous introduces a novel agent memory architecture that replaces traditional fact storage with a predictive world model of categorical probability distributions, where information is retained as belief updates (deltas) and forgetting occurs naturally through entropy decay, achieving competitive performance on long-term conversational benchmarks without requiring external vector databases or graph engines.

Original authors: Pranav Singh

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pranav Singh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to remember a conversation with a friend. Most computer programs that try to do this act like a filing cabinet. They take every fact you say ("I like pizza," "I work at Google") and shove it into a folder. If you later say, "Actually, I hate pizza and I work at Amazon," the filing cabinet just adds a new note. It doesn't know how to resolve the conflict, so it ends up with a messy drawer full of contradictory papers.

The paper introduces Nous, a new way for AI agents to remember things. Instead of a filing cabinet, Nous acts like a living, breathing brain.

Here is how it works, using simple analogies:

1. The Core Idea: Prediction, Not Storage

The authors argue that human memory isn't about storing a perfect transcript of the past. It's about having a model of the world that predicts what will happen next.

The Old Way: "I have a database record that says 'Pranav works at Google'."
The Nous Way: "I have a mental picture of Pranav's job. Right now, I'm 87% sure he works at Google, 8% sure he works at Amazon, and 5% sure I don't know."

2. How It Learns: The "Surprise" Meter

Every time the AI hears something new, it checks its mental picture against what it just heard.

Low Surprise: If the AI thinks "Pranav likes coffee" and you say "Pranav likes coffee," the AI says, "Okay, that matches my guess." Nothing changes much.
High Surprise: If the AI thinks "Pranav likes coffee" and you say "Pranav hates coffee," the AI gets a shock. In the paper, this is called "surprise."
The Update: Instead of just writing down the new fact, the AI mathematically shifts its mental picture. The "coffee lover" part of the picture shrinks, and the "coffee hater" part grows. The AI remembers how much its mind changed, not just the new fact.

3. Forgetting: The Fading Echo

In a database, you have to manually delete old files. In Nous, forgetting happens naturally, like a sound fading in a large room.

If the AI hasn't heard about a specific fact in a long time, its certainty about that fact slowly drifts toward "I have no idea."
The paper calls this entropy decay. It's like a rubber band that slowly loses its tension if you don't pull on it. If you don't reinforce a memory, it naturally becomes fuzzy and uncertain.

4. Solving Conflicts: The "Delta"

When the AI updates its mind, it doesn't just overwrite the old file. It saves a Delta (a Greek letter often used for "change").

Think of a Delta as a diary entry that says: "On Tuesday, I thought Pranav worked at Google. Then I heard he moved to Amazon, so I shifted my belief."
This allows the AI to keep a history of how it learned, rather than just a list of static facts.

5. Handling Lists (The "Aggregation" Trick)

There is one tricky part. If you ask, "What places has Pranav visited?" a simple "best guess" model might only say "Paris" because that's the most likely answer. But you want a list of all places.

The Fix: Nous keeps a special log of every time it was "surprised" by a new place. When you ask for a list, it digs through that log to give you the full history, while still keeping its "best guess" for simple questions.

The Results: How Well Does It Work?

The authors tested Nous on a benchmark called LoCoMo, which involves long conversations with many questions. They compared it to other systems (like A-MEM and BeliefMem) using a standard AI model (GPT-4o-mini).

The Score: Nous scored significantly higher than the other systems in three out of four categories (Single-hop, Multi-hop, and Open-domain questions).
The Comparison: They also compared it to BeliefMem, a system built by a different team with a very similar "belief-based" idea. On the numbers reported by both teams, Nous came out ahead.
The Caveat: The authors are very honest. They admit that comparing their results to BeliefMem isn't a perfect "apples-to-apples" test because the two teams used slightly different methods to ask questions and grade answers. They plan to run a stricter, side-by-side test in the future.

Summary

Nous is an AI memory system that treats knowledge like a probability (a guess that gets better or worse) rather than a fact (a hard record).

It learns by measuring surprise.
It forgets by letting uncertainty grow over time.
It resolves conflicts by shifting probabilities instead of deleting files.

The paper claims this approach makes the AI better at remembering long conversations, handling contradictions, and answering complex questions, all without needing a massive external database. It is presented as a new, open-source tool that the authors are still refining.

Technical Summary: Nous – A Predictive World Model for Long-Term Agent Memory

Problem Statement

Current Large Language Model (LLM) agents suffer from a lack of continuously evolving long-term memory. While context windows provide short-term recall, conversational history is lost once a session ends unless externally persisted. Existing memory systems (e.g., RAG, knowledge graphs, episodic stores like MemGPT) treat memory as a static database of facts. The authors argue this "storage" paradigm leads to three structural failure modes:

Conflict Blindness: Contradictory records (e.g., a user changing jobs) accumulate without principled resolution.
Heuristic Forgetting: Deletion is based on time rather than semantic relevance or uncertainty.
Retrieval Granularity Mismatch: Fact stores return individual records, lacking mechanisms to aggregate values over time (e.g., listing all activities a person has ever done).

The paper posits that these failures stem from treating memory as storage rather than a predictive world model, analogous to how the brain maintains a generative model of the environment updated by prediction errors.

Methodology: The Nous Architecture

Nous implements memory as a collection of categorical probability distributions, termed dimensions, rather than deterministic facts.

Core Abstractions

Dimensions ( $D_{e,a}$ ): For every entity-attribute pair (e.g., Pranav, employer), Nous maintains a categorical distribution $p(v)$ over a vocabulary of possible values. The mode of this distribution represents the agent's current best belief.
Deltas ( $\Delta$ ): The primary stored artifact is not the fact itself, but the delta—a record of the shift from prior to posterior belief. A delta includes the prior distribution, posterior distribution, the information-theoretic surprise, timestamp, and supporting evidence.
Surprise-Driven Updates: Incoming observations are scored by information-theoretic surprise: $S = -\log_2 P(obs | D)$ . High surprise (contradictory evidence) triggers significant belief revision, while low surprise (confirming evidence) results in negligible change.
Bayesian Posterior: Updates are computed via a closed-form Bayesian posterior (Eq. 3). This allows probability mass to shift naturally toward contradictory evidence without explicit conflict detection logic.
Entropy Decay (Forgetting): Dimensions not updated for a duration $\Delta t$ undergo exponential mixing toward a uniform distribution (Eq. 4). This simulates natural forgetting as uncertainty grows over time, a property absent in heuristic deletion rules.
Identity Resolution: Entities are merged if the symmetrized KL divergence between their shared dimension sets is low.

Handling Multi-Valued Attributes

A limitation of unimodal categorical distributions is the inability to represent inherently multi-valued attributes (e.g., a list of visited cities) without discarding historical alternatives. Nous addresses this via delta-log aggregation: at query time, any value that appeared with surprise above a threshold $\theta_{min}$ is included in the context. This preserves the Bayesian world model for the "current best belief" while allowing exhaustive aggregation for specific queries.

Pipelines

Ingestion: An LLM extracts $(e, a, v)$ triples from sessions. For each, the system computes surprise, updates the posterior, and appends a delta to the log.
Querying: The system performs a breadth-first search over the entity graph (up to two hops) to seed relevant entities. It formats dimension states as fact lines and appends top-k delta records (based on BM25 similarity) as evidence. This context is passed to an answering LLM.

Key Contributions

Predictive World Model Paradigm: Formalizing agent memory as a collection of dimensions updated by surprise-driven Bayesian inference, rather than a fact store.
Open-Source Implementation: A complete system requiring no external vector database or graph engine.
Evaluation and Honest Benchmarking: Performance evaluation on the LoCoMo benchmark, accompanied by an explicit discussion of reproducibility issues found in the literature regarding baseline comparisons (specifically A-MEM) and a comparative analysis against BeliefMem, a concurrent system with similar premises.

Results

Evaluated on the LoCoMo benchmark (10 conversations, 1,540 questions) using GPT-4o-mini as the backbone:

Performance: Nous achieved the following Token F1 scores:
- Single-hop: 63.50
- Multi-hop: 55.32
- Temporal: 58.57
- Open-domain: 62.50
Comparison with A-MEM: Nous showed substantial gains over A-MEM's self-reported GPT-4o-mini numbers in three of four categories. The authors note a reproducibility issue where independent citations of A-MEM's results disagree on category assignment, though the underlying numerical values are consistent.
Comparison with BeliefMem: Against BeliefMem (another belief-based system), Nous's self-reported numbers exceeded BeliefMem's on all four categories (e.g., 63.50 vs. 48.41 on single-hop). However, the authors explicitly state this is not a fully controlled comparison due to differences in extraction prompts, retrieval mechanisms (BM25/BFS vs. similarity/decay), and unverified question counts.
Failure Analysis:
- Single-hop: High context recall (86.07%) but lower F1 (63.50), indicating the bottleneck is the LLM's answer generation, not retrieval.
- Open-domain: Lowest context recall (47.50%) and highest retrieval miss rate (37%), suggesting the entity-attribute dimension model struggles to map personality/habit questions cleanly.
- Temporal: Suffers from low context recall (61.82%) because temporal data is often embedded in evidence text rather than extracted as explicit dimension values.

Significance and Claims

The paper claims that Nous demonstrates a viable alternative to database-centric memory architectures. By modeling memory as a predictive system updated by surprise, it naturally handles contradiction (via probability shifts), forgetting (via entropy decay), and aggregation (via delta logs).

The authors position this work as a first public report on a new architecture. They emphasize modesty regarding the results:

The comparison with BeliefMem is "suggestive rather than conclusive" pending a controlled, self-run comparison with matched pipelines.
The unimodal nature of dimensions is acknowledged as a limitation for set-valued attributes, with a move toward independent Bernoulli distributions planned.
The evaluation is currently limited to LoCoMo and GPT-4o-mini; generalization to other backbones and harder benchmarks (LongMemEval) is planned.

The core significance lies in the architectural shift: treating the agent's memory not as a repository of "what happened," but as a continuously refined probability distribution over the state of the world, grounded in cognitive science principles like predictive coding and the free-energy principle.