💬 NLP

Simulating Human Memory with Language Models

This paper demonstrates that while out-of-the-box language models possess unrealistically reliable memory compared to humans, applying specific prompting strategies and memory compactors can induce human-like forgetting, thereby improving their effectiveness as user simulators in educational contexts.

Original authors: Qihan Wang, Nicholas Tomlin, Michael Hu, Brian Dillon, Tal Linzen

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qihan Wang, Nicholas Tomlin, Michael Hu, Brian Dillon, Tal Linzen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to be a student so you can test your teaching methods on it before trying them on real people. You'd want the robot to forget things just like a human does, right? If the robot remembers everything perfectly, it's not a very good test subject for a human teacher.

This paper is about a team of researchers who tried to build a "student robot" that forgets things the way humans do. They discovered that standard AI models are actually too good at remembering things, and they had to build special "forgetting machines" to fix that.

Here is the story of their findings, broken down into simple parts:

1. The Problem: The Robot with a Perfect Memory

The researchers set up a series of memory tests for both humans and AI models. Think of these tests like:

The Number Game: Remembering a list of numbers and repeating them backward.
The Story Game: Reading a short story and retelling it word-for-word.
The Map Game: Looking at a map of a city and then answering questions about how to get from point A to point B without looking at the map anymore.

The Result: The humans did okay. They remembered about 7 numbers, got a few details of the story, and knew the map pretty well. But the AI models? They were superheroes. They remembered 20 numbers perfectly, recited the entire story without a single typo, and navigated the map flawlessly.

Even when the researchers told the AI, "Hey, act like a human who is tired and forgetful," the AI just ignored the advice and kept remembering everything perfectly. It's like telling a camera to take a blurry photo; the camera just keeps taking sharp, high-definition pictures because that's what it's built to do.

2. The Solution: The "Four-Box" Backpack

The researchers realized that simply asking the AI to forget wasn't working. They needed to change the AI's "backpack."

In human psychology, there's a famous idea that our short-term memory is like a backpack with only four compartments (or "chunks"). If you try to put more than four things in there, the oldest ones fall out.

So, the researchers built a special tool called a COMPACTOR.

How it works: Before the AI tries to answer a question, it is forced to put all the information it just read into a "backpack" with only four slots.
The Catch: It has to summarize the information to fit. If the story is long, it has to squeeze the details into four tiny boxes. Anything that doesn't fit gets thrown away.
The Result: Suddenly, the AI started forgetting things! It made mistakes. It forgot parts of the map. It mixed up numbers. It finally started acting like a human student who is struggling to keep up.

3. The "Fake" Human vs. The "Real" Human

The researchers tried a few other tricks to make the AI act human:

The "Acting" Trick: They gave the AI examples of humans making mistakes and said, "Copy this."
- Result: It didn't work well. The AI could copy the style of a mistake, but it didn't actually forget the information. It was like an actor pretending to trip but still walking perfectly.
The "Summary" Trick: They asked the AI to just write a summary of the text and answer from that.
- Result: This helped a little, but the COMPACTOR (the four-box backpack) was the clear winner. It forced the AI to actually lose information, which is what real humans do.

4. Why Does This Matter? (The "Teaching" Test)

To see if this new "forgetting AI" was actually useful, the researchers ran a final test. They pretended the AI was a student in a class.

They showed the AI (and real humans) different versions of a biography:

Version A: Written in simple, easy language.
Version B: Written in very complex, hard language.
Version C: Full of repeated facts (redundant).
Version D: Full of distracting, useless information.

They asked: "Which version of the story will a human student remember best?"

The Standard AI (Perfect Memory): Said, "Oh, the human will remember all of them perfectly! It doesn't matter which one we use." (This is wrong because humans do struggle with complex or distracting text).
The COMPACTOR AI (Human-like Memory): Said, "The human will struggle with the complex one and the distracting one, but they'll do okay with the simple or repeated one."

The COMPACTOR AI was much better at predicting what a real human could handle. It realized that if you give a human too much hard stuff, they will forget it.

The Bottom Line

The paper concludes that if you want to use AI to simulate humans (for example, to test new teaching methods or design better user interfaces), you can't just use a standard AI. Standard AIs are too perfect.

You have to build artificial limitations into them—like a backpack with only four slots—to make them forget things the way we do. Only then can they be useful stand-ins for real people.

What the paper doesn't say:

It doesn't say this will cure memory loss in humans.
It doesn't say these robots will replace human teachers.
It doesn't claim the AI is now "perfectly" human; even the best "forgetting" AI still makes different kinds of mistakes than humans do. It's just a lot closer than before.

Technical Summary: Simulating Human Memory with Language Models

Problem Statement

While large language models (LLMs) have achieved human-level or superior performance on many benchmarks, their memory capabilities diverge significantly from human cognition. Current frontier models possess near-perfect, verbatim recall of long contexts, whereas human memory is characterized by systematic limitations, forgetting, and reconstruction errors. This discrepancy renders standard LLMs poor "user simulators" for applications requiring realistic human behavior, such as training AI assistants to collaborate with humans, simulating students for educational tools, or modeling population dynamics. The core problem addressed is how to constrain LLMs to exhibit human-like memory limitations, specifically in working memory capacity and forgetting patterns, to improve their utility as simulators.

Methodology

The Human Memory Simulation Benchmark

The authors constructed a benchmark comprising ten memory tasks grounded in cognitive science and psychology literature. These tasks evaluate different memory dimensions:

Working Memory: Digit span, Reverse digit span, N-back, Word recognition, and Variable mapping.
Episodic/Long-term Memory: Factual QA, Narrative QA, and Narrative free recall.
Relational/Spatial Memory: Map task and Craft task.

Human data was collected from $N=50$ participants via Prolific, who completed these tasks under time constraints similar to standard psychological experiments.

Evaluation Metrics

To quantify how closely an LLM simulates human memory, the authors use the Wasserstein distance (Earth Mover's Distance) between the distribution of human scores and model scores for each task. This metric is normalized and converted into a "Humanlikeness" score ( $1 - W_1$ ), where higher values indicate better alignment with human performance distributions.

Experimental Conditions

The study evaluated nine LLMs (including GPT-5.4, Claude Opus 4.6, Llama 3 variants, and Qwen3 variants) under four distinct conditions:

TASKPR (Task Prompt): The baseline prompt describing the task without reference to human simulation.
HUMPR (Human Prompt): Explicitly instructs the model to simulate a human participant, including specific time constraints.
MEMPR (Memory Prompt): Similar to HUMPR but explicitly reminds the model that humans have limited memory and make mistakes.
COMPACTOR: An agent-based approach that implements a human-like working memory bottleneck. This agent uses a key-value memory store with a hard capacity of $K=4$ entries (reflecting the "magical number 4" in human short-term memory). The model must summarize context into these four chunks during an "encode" phase and answer queries using only these stored chunks during a "recall" phase.

Ablation and Analysis

The authors also investigated:

In-Context Learning (ICL): Providing few-shot examples of human performance (in-domain vs. out-of-domain).
Error Patterns: Analyzing whether model errors (e.g., truncation vs. substitution) match human error patterns.
Downstream Application: A proof-of-concept experiment in an educational setting where simulators must rank document variants (varying in reading level, redundancy, and distractors) based on which would be most memorable for a human learner.

Key Results

1. Default Models Diverge from Humans

In the TASKPR condition, all evaluated models exhibited near-ceiling performance across all ten tasks, far exceeding human averages (e.g., recalling digit spans of 20+ vs. human average of ~7). This confirms that out-of-the-box LLMs are not suitable human simulators due to their superior memory.

2. Prompting Alone is Insufficient

Explicitly prompting models to simulate humans (HUMPR) or reminding them of memory limits (MEMPR) yielded limited and inconsistent improvements.

For most tasks, the change in Humanlikeness ( $\Delta$ ) was near zero or negative.
MEMPR showed some improvement in specific tasks like Narrative Free Recall, but failed to consistently induce human-like forgetting across the suite.
This suggests that simple instruction is insufficient to override the model's inherent capability for perfect recall.

3. Explicit Memory Constraints (COMPACTOR) Improve Alignment

The COMPACTOR agent, which enforces a structural bottleneck of 4 memory chunks, produced significantly more human-like score distributions than prompting alone.

Working Memory Tasks: COMPACTOR yielded the largest gains in tasks like Digit Span, Reverse Digit Span, and N-back. For example, in the Digit Span task, COMPACTOR improved Humanlikeness by up to +0.61 for certain models.
Long-term Memory: Improvements were more variable for tasks like Factual QA but still generally positive compared to baselines.
Ablation: COMPACTOR outperformed a simpler "summarizer" agent that produced a single free-form prose summary, indicating that the structured, capacity-limited nature of the memory module is critical, not just the act of summarization.

4. Error Patterns Remain Divergent

Despite improved score distributions, the nature of errors differed between humans and models.

Humans tend to make random errors or forget specific items.
MEMPR models often reproduced sequences with one or two incorrect digits in the middle.
COMPACTOR models frequently predicted truncated versions of sequences.
Conditional error rates ( $p(\text{err} | \text{prev wrong})$ ) were significantly different, suggesting that while COMPACTOR mimics the capacity limit, it does not yet fully replicate the mechanism of human forgetting.

5. Downstream Application: Educational Reranking

In the proof-of-concept educational task, models with COMPACTOR were better at predicting which document variants humans would remember best compared to baseline models.

Baseline models predicted that students would answer all questions correctly regardless of document difficulty.
COMPACTOR models more effectively identified that documents with high redundancy were easier to remember and those with distractors were harder.
However, even the best-performing models achieved less than 70% accuracy in pairwise reranking, indicating substantial room for improvement.

Significance and Claims

The paper claims that:

Current LLMs are "super-human" in memory: They fail to simulate human users effectively because they do not forget.
Prompting is insufficient: Merely asking a model to "be human" does not induce human-like cognitive constraints.
Architectural constraints are necessary: Implementing explicit, capacity-limited memory modules (like the 4-chunk COMPACTOR) is a more effective strategy for aligning model behavior with human memory distributions.
Practical Utility: Simulators with human-like memory constraints are more useful for downstream tasks, such as optimizing educational materials, because they can realistically model the cognitive load and retention limits of human learners.

The authors conclude that while their methods (COMPACTOR) represent a significant step forward, they are only a "first step." The detailed error patterns of models still differ from humans, and future work is needed to refine the compaction mechanism and explore other cognitive domains (e.g., reasoning) to build truly realistic user simulators. The paper releases the benchmark data and code to facilitate further research in this area.