Memory for Large Language Models
This survey presents a systematic, architecture-centric taxonomy for memory in large language models, unifying fragmented research through a three-axis framework of representation, update dynamics, and persistence to guide future scalable and adaptive design.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are teaching a brilliant but forgetful friend how to write a story. You sit them down, give them a massive library of books, and they memorize the whole thing. This is how today's most advanced computer brains, called Large Language Models (LLMs), work. They are like super-smart students who have read almost everything on the internet. But here's the catch: once they finish reading a book, they usually forget the details unless you keep the book open right in front of them. In the world of AI, this "book open on the desk" is called memory.
For a long time, these AI models had a very specific kind of memory. It was like a sticky note that only lasted as long as you were looking at it. If you tried to read a novel that was longer than the sticky note could hold, the AI would start forgetting the beginning of the story by the time it reached the middle. This was called "implicit memory"—it was just a side effect of how the computer was doing its math, not a dedicated tool for remembering. But recently, scientists have started asking a big question: What if we gave these AI models a real notebook? A place where they could write down important facts, keep them forever, and look them up later, just like a human does? This shift from "forgetting everything after the conversation ends" to "having a permanent, organized brain" is the big story happening right now in computer science.
This paper is a massive guidebook for that new world. The authors, a team of researchers, noticed that everyone is inventing different ways to give AI memory, but they are all using different names and confusing maps. Some people are building "sticky notes," others are building "filing cabinets," and some are even trying to teach the AI to rewrite its own brain while it talks. The paper's main job is to sort all these messy ideas into a neat, organized system. They propose a new way to look at AI memory using three simple questions: What is being stored? When does it get updated? and How long does it last?
The paper suggests that we are moving away from the old way, where memory was just a temporary byproduct of thinking, toward a new era where memory is a dedicated, controllable tool. They find that there are two main types of memory now. The first is Implicit Memory, which is like the AI's short-term working memory. It's fast and flexible, but it's tied to the immediate conversation and disappears the moment the chat ends. Think of it as the AI holding a conversation in its head; it can remember what you said five seconds ago, but it can't remember what you said yesterday unless you remind it.
The second type is Explicit Memory, which is the exciting new stuff. This is like giving the AI a real notebook or a digital hard drive. This memory is separate from the thinking process. The AI can write a fact into this notebook, close the chat, and come back days later to read it. The paper shows that researchers are building these notebooks in different ways: some are "parameterized," meaning the AI learns to write directly into its own brain weights; others are "lookup-based," where the AI searches a giant database of facts; and some are "mixture-of-experts," where the AI has different specialists it can call on for different topics.
However, the paper is very clear that this isn't a solved problem yet. While these new memory systems are powerful, they come with big trade-offs. The authors point out that just because you give an AI a bigger notebook doesn't mean it will remember better. Sometimes, the AI gets confused by too much information, or it writes down the wrong things and can't erase them. They also note that updating this memory while the AI is talking (called "online" updates) is risky and can make the AI unstable, whereas updating it only when it's learning (called "offline") is safer but less flexible.
The paper doesn't claim that we have built a perfect, human-like brain. Instead, it suggests that we are in a messy, experimental phase where we are trying to figure out the best rules for how AI should store and use information. They argue that the future isn't about picking just one type of memory, but about building hybrid systems. Imagine a model that has a quick sticky note for the current conversation, a personal notebook for long-term facts, and a library card for searching the internet, all working together. The authors conclude that to make AI truly smart and useful, we need to stop treating memory as an accident and start designing it as a core feature, carefully balancing how much it remembers, how fast it updates, and how long it keeps things. It's a roadmap for turning a forgetful genius into a truly wise companion.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.