The Geometry of Forgetting

This paper argues that core human memory phenomena, such as power-law forgetting and false memories, are not biological bugs but inevitable geometric features arising from interference and noise in high-dimensional semantic spaces, as demonstrated by their spontaneous emergence in unmodified production embedding models without any specific engineering.

The Gist

The Big Idea: It's Not Your Brain's Fault, It's Math's Fault

For over a century, psychologists have thought our memory problems (forgetting things, remembering things that never happened) are because our biological hardware is "leaky" or broken. They blame the wet, squishy brain.

This paper says: Stop blaming the brain.

The authors argue that forgetting and false memories aren't bugs in biology; they are features of geometry. If you organize information by meaning (like a library) rather than by address (like a filing cabinet), and you try to find things by looking for "similar" items, you will inevitably forget and make mistakes. It doesn't matter if you are a human or a computer; the math of high-dimensional space forces these errors to happen.

---

Analogy 1: The Crowded Library (Why We Forget)

Imagine your memory is a giant, magical library where books are arranged not by title, but by vibe.

• Books about "sleep" are all clustered together in one corner.
• Books about "eating" are in another corner.

The Old Theory (Decay):
People used to think that if you don't read a book for a long time, the ink fades, and the book disappears.

The New Theory (Interference):
The authors say the ink never fades. The book is still there, perfectly intact. But, imagine you keep adding new books to the "sleep" corner. Soon, the aisle is so crowded with books about "beds," "dreams," "tiredness," and "pillows" that when you try to find the specific book you want, you get lost in the crowd. You can't pull the right one out because the others are blocking it.

The Experiment:
The researchers built a computer memory system.

• When they gave it one fact to remember, it never forgot it, even after a long time.
• When they gave it 10,000 similar facts to remember, it started forgetting the original one, following the exact same curve humans do (the famous "Ebbinghaus forgetting curve").

The Lesson: Time doesn't make you forget. Crowding does. The more similar things you have, the harder it is to pick the right one out.

---

Analogy 2: The "Dimensionality Illusion" (Why Computers Are Just as Confused)

You might think, "But computers are smart! They have huge memory spaces with thousands of dimensions (directions)."

The authors found a trick called the Dimensionality Illusion.
Imagine a skyscraper that claims to have 1,000 floors. But when you look at the blueprints, you realize that all the people and furniture are actually squished into just 16 floors. The other 984 floors are empty air.

Even though modern AI models claim to have 1,000 dimensions, they actually squish all their information into about 16 effective dimensions to make sense of it. Because they are so "squished" (low effective dimensionality), they are just as crowded and confused as a human brain. They are stuck in the same "interference zone" where forgetting happens.

---

Analogy 3: The "Sleep" Trap (Why We Have False Memories)

Have you ever heard a list of words: Bed, Rest, Awake, Tired, Dream... and then been asked, "Did the word Sleep appear in the list?" You will confidently say YES, even though it wasn't there. This is called the DRM Effect.

The Paper's Explanation:
In the "Vibe Library" (the geometry of meaning), the word "Sleep" sits right in the middle of the cluster of Bed, Rest, Awake, Tired.
When your brain (or the computer) tries to retrieve the list, it looks for the "Sleep" vibe. Since all those words are physically close to each other in the math-space, the system gets confused. It thinks, "Oh, 'Sleep' is right here in this cluster, so it must have been there."

The Shocking Result:
The researchers didn't program the computer to make this mistake. They didn't tell it to hallucinate. They just let the computer look at the raw math of the words. The computer made the exact same mistake as humans (about 58% false alarm rate vs. 55% for humans) just by doing simple math.

The Takeaway: False memories aren't a glitch. They are the price of admission for having a smart memory. If you organize things by meaning, you must group similar things together. If you group them together, you must sometimes confuse them.

---

Analogy 4: The Spaced Repetition (Why Studying Works)

Why is it better to study for 1 hour spread over a week than to cram for 1 hour the night before?

In this geometric world, imagine your memory trace is a fresh, crisp photo. Over time, "noise" (static) gets added to the photo, making it blurry.

• Cramming (Massed): You take 3 photos at the exact same moment. They all get old and blurry at the same rate. When you test yourself, all 3 are blurry.
• Spacing: You take one photo today, one in 3 days, and one in a week. When you are tested a month later, the first two are very blurry, but the third one (the most recent) is still relatively fresh and clear.

The geometry shows that as long as you have one recent, clear trace, you can survive the noise. This explains why spacing works, purely based on how "age" affects the clarity of the data in the system.

---

The "So What?" Conclusion

The paper concludes that biology didn't design a broken memory.

Instead, biology built a system that organizes information by meaning (so we can understand the world). But the laws of geometry dictate that any system that organizes by meaning will:

1. Forget things when the crowd gets too big (Interference).
2. Confuse similar things (False Memories).
3. Need to space out learning to stay clear (Spacing Effect).

The Final Metaphor:
Imagine you are trying to find a specific grain of sand on a beach.

• If the beach is empty, you find it instantly.
• If the beach is covered in billions of other grains of sand that look almost the same, you will struggle to find it, not because your eyes are bad, but because the beach is crowded.

The authors are saying: Our brains are the beach, not the eyes. The "flaws" of memory are just the natural physics of a crowded, meaningful world.

Technical Summary

1. Problem Statement

The paper addresses a fundamental question in cognitive science: Why do humans forget, and why do we generate false memories?

• Conventional View: The standard explanation attributes these phenomena to biological "hardware" limitations (e.g., neural decay, synaptic leakage, or imperfect biological implementation).
• The Gap: While interference theories (competition between memories) and decay theories (fading over time) have competed for over a century, the specific mechanism causing power-law forgetting and false memories in representational systems remains unclear.
• Hypothesis: The authors propose that these "flaws" are not biological bugs but inevitable features of the geometry of high-dimensional embedding spaces. They argue that any system organizing information by meaning and retrieving it by proximity (similarity) will exhibit these failure modes due to mathematical constraints, regardless of whether the substrate is biological (wetware) or artificial (silicon).

2. Methodology

The authors utilized open-weight Transformer-based embedding models and public datasets to simulate memory systems without phenomenon-specific engineering.

• Core Architecture:
  • Memory Substrate: Contextually enriched embeddings (content vectors concatenated with positional, temporal, and episodic metadata).
  • Retrieval Mechanism: Cosine similarity search.
  • Models Used: all-MiniLM-L6-v2, BGE-base, BGE-large, CLIP (for cross-modal), and Qwen2.5-7B for generation.
• Experimental Design: Five distinct experimental domains were tested to replicate human memory phenomena:
  1. Power-Law Forgetting (Ebbinghaus): Simulated 1,000 facts over 30 days with temporal decay functions and varying numbers of competing memories (distractors).
  2. Interference vs. Decay: Isolated the effect of competitors by running decay-only simulations vs. decay + competition.
  3. Dimensionality Analysis: Investigated the "Dimensionality Illusion" by calculating the participation ratio (effective dimensionality, $d_{eff}$) of production models (nominally 384–1,024 dimensions) to see if they operate in low-dimensional regimes.
  4. False Memories (DRM Paradigm): Tested the Deese–Roediger–McDermott (DRM) word lists using raw pre-trained embeddings to see if false alarms emerge naturally from semantic clustering.
  5. Spacing Effect & Tip-of-Tongue (TOT): Simulated distributed vs. massed practice with age-dependent noise and analyzed retrieval rankings for partial recall states.
  6. Topological Analysis: Used persistent homology (Rips complex) to analyze the manifold structure of the memory space.

3. Key Contributions

1. Geometric Origin of Forgetting: Demonstrated that interference, not temporal decay, is the primary driver of power-law forgetting in embedding spaces.
2. The Dimensionality Illusion: Revealed that production embedding models (e.g., 1,024-dim) concentrate their variance in only ~16 effective dimensions ($d_{eff}$), placing them deep in an interference-vulnerable regime despite their high nominal dimensionality.
3. "Unbaked" False Memories: Showed that false memories arise naturally from the raw geometry of semantic space without parameter tuning. The semantic clustering required for useful retrieval inherently causes confusion between related concepts.
4. Unified Framework: Provided a single geometric framework that explains power-law forgetting, false memories, spacing effects, and tip-of-tongue states, suggesting these are universal properties of similarity-based retrieval systems.

4. Key Results

A. Interference vs. Decay

• Decay Only: When simulating memory with a decay function but no competitors, the forgetting exponent was $b \approx 0.009$ (50x smaller than human values).
• Interference: Adding 10,000 competing memories raised the exponent to $b = 0.460 \pm 0.183$, closely matching the human value ($b \approx 0.5$).
• Conclusion: Time alone does not produce forgetting; crowding (competition) does.

B. The Dimensionality Illusion

• Effective Dimensionality: Spectral analysis (participation ratio) showed that MiniLM ($d_{nom}=384$), BGE-base ($d_{nom}=768$), and BGE-large ($d_{nom}=1024$) all have an effective dimensionality of $d_{eff} \approx 16$.
• Impact: At $d=64$, interference is strong ($b \approx 0.16$). At $d \ge 128$, interference drops significantly. Because production models effectively operate at $d \approx 16$, they are highly susceptible to interference, explaining why RAG (Retrieval-Augmented Generation) systems degrade as database size increases.

C. False Memories (DRM)

• Result: Using raw cosine similarity on pre-trained embeddings for 24 DRM word lists, the system produced a critical-lure false alarm rate of 0.583, compared to the human rate of ~0.55.
• Significance: This occurred with zero parameter tuning. The geometry of semantic space naturally clusters related words (e.g., "bed," "rest," "awake") so tightly that the unstudied lure ("sleep") is indistinguishable from studied items.

D. Spacing Effect & Topology

• Spacing: Distributed practice outperformed massed practice because the most recent trace in spaced learning is less degraded by noise. The system reproduced the human ordering: Long > Medium > Short > Massed.
• Topology: Persistent homology revealed a phase transition in the memory manifold. At specific connectivity thresholds, the space forms transient loops (semantic "holes"), suggesting a hierarchical thematic structure in natural language.

5. Significance and Implications

• For Cognitive Science: The paper challenges the view that forgetting is a biological defect. Instead, it suggests that forgetting and false memory are the "price of admission" for a system that organizes information by meaning. A system that never confuses related concepts cannot generalize.
• For AI and Engineering:
  • RAG Systems: Retrieval accuracy in vector databases will inevitably degrade as a power law with database size due to geometric interference, not just hardware limits.
  • Design Warning: "Vector averaging" (compressing databases by averaging nearby embeddings) is geometrically destructive. It collapses angular distinctions, erasing the fine structure needed for retrieval.
  • Mitigation: To reduce false memories or interference, one must sacrifice semantic clustering or introduce non-geometric constraints (e.g., metadata filters, source attribution), as the geometry itself is the source of the error.
• Philosophical Shift: The boundary between biological and artificial memory is thinner than assumed. Both are subject to the same mathematical constraints of high-dimensional similarity search. Biology determines the parameter regime; geometry determines the failure modes.

Conclusion

"The Geometry of Forgetting" posits that core memory phenomena are not bugs of biological implementation but features of any system that organizes information by meaning. The paper provides quantitative evidence that the mathematics of similarity-based retrieval in finite-dimensional spaces naturally reproduces human-like forgetting curves, false memories, and retrieval failures, suggesting that these are universal geometric constraints rather than biological anomalies.