Geometric Indexing Does Not Improve Versioned Memory Retrieval: Three Pre-Registered Negative Results and a Silent Total-Failure Mode

The Memory Problem: Why AI Needs to Remember, Not Just Read

Imagine you are talking to a very smart robot friend. You tell it, "My favorite movie is Star Wars." Later, you change your mind and say, "Actually, I take that back; my favorite is The Matrix." If you ask the robot again, "What's my favorite movie?" you want it to remember the new answer, not the old one. This is the challenge of conversational memory. Unlike a library that just stores books (documents) for everyone to read, a personal memory needs to remember you, track how your thoughts change over time, and handle corrections without getting confused.

To do this, computers use something called embeddings. Think of an embedding as a magical map where every sentence is a dot. Sentences that mean similar things are placed close together on the map, while different ideas are far apart. When you ask a question, the computer looks for the dots closest to your question to find the answer. A popular idea in AI research has been to use the geometry of this map to handle changes. The theory was: if you change your mind, don't erase the old dot; just place the new dot slightly next to the old one. The idea was that the distance between the dots would tell the computer, "Hey, these two are related versions of the same fact!"

But does this clever trick actually work? That is exactly what this paper investigates. The researchers wanted to know if simply placing "revised" memories near "original" memories on this map helps the computer find the right answer later. They set up a strict, pre-planned experiment to test this, hoping to find a magic shortcut for AI memory. Instead, they found that the shortcut leads to a dead end, and in doing so, they uncovered a silent bug that could break many real-world AI systems.

---

The Experiment: Trying to Walk a Tightrope on a Map

The researchers set up a simulation with 3,000 fake facts, like "The director of Helios Laboratory is Ana." They then tested what happened when the "director" was corrected to "Beto," then to "Carlos," and so on. They compared three ways of storing these changes:

1. The "Overwrite" method: Erase the old fact and write the new one. (This loses history but keeps the current fact).
2. The "Duplicate" method: Store the new fact exactly where it belongs on the map based on its own words.
3. The "Budding" method (The Hero of the Story): Place the new fact slightly away from the old one, hoping the geometry (the shape and distance) would link them together.

The First Failure: The Drifting Cluster

In the first test, the researchers tried the "Budding" method with a fixed rule: every time a fact was corrected, move the new version a tiny, fixed step away from the original.

Imagine you are trying to walk in a straight line, but every time you take a step, you are forced to turn slightly to the left. After one step, you are fine. After four steps, you are far off course. After six steps, you are walking in the opposite direction.

That is exactly what happened to the AI's memory. With a fixed step size, the "cluster" of revised facts started doing a random walk away from the original fact.

• After 4 revisions, the connection to the original question dropped so low that the computer stopped recognizing the answer.
• After 6 revisions, the connection score actually went negative (from a high of 0.811 down to -0.139).

The computer was looking for the answer, but the "revised" memory had drifted so far away on the map that it looked like a stranger. The "Budding" idea failed because the distance kept piling up, pushing the correct answer out of the top results.

The Second Failure: The Unavoidable Tax

The researchers thought, "Okay, maybe the problem is that we let it drift too far. What if we force the new version to stay close to the original anchor, rather than the previous version?" They tried a "bounded" version where every revision stayed at a fixed distance from the start.

This time, the memory didn't drift away. The connection score stayed flat and stable at 0.8177. But here is the twist: it still lost.

Why? Because the "Budding" method paid a constant tax of about 0.036 on the score.

• The "Duplicate" method (storing the fact exactly where it belongs) scored 0.8540.
• The "Bounded Budding" method scored 0.8177.

The researchers realized that the computer's map was already perfect. When you say "The director is Beto," the computer naturally places that sentence in the right spot. By artificially moving it away to "link" it to the past, you are just moving it away from the perfect spot. It's like trying to help a friend find their house by telling them to stand three feet to the left of the front door. The house is already right there; moving them just makes it harder to find.

The Third Failure: The Silent Killer

The researchers weren't done. They suspected their conclusion relied on the fact that their test sentences were "self-contained" (they named the person and the fact clearly). In real life, people often use concise phrasing. They say, "No, it's Beto," without saying "The director of Helios is Beto." This is called an elliptical correction.

They set up a "worst-case scenario" test designed to make the geometric trick look good. They compared the "Budding" method against a "Hydrated" method (where a smart system rewrites "No, it's Beto" into the full sentence before saving it).

The results were shocking:

• Raw elliptical corrections (saving "No, it's Beto" exactly as spoken) had a 0.0000 success rate.
• The computer returned an answer every time, but it was never the right one.
• The failure was silent. The computer didn't say "I don't know." It confidently returned the wrong answer, ranked just as high as the right one would have been.

Even in this regime where geometry should have helped, it only won if the alternative system (the one that rewrites the sentences) was failing more than 45% of the time. Since real-world systems are much better than that, the geometric trick offers no advantage.

The Big Takeaway

This paper delivers a clear, pre-registered verdict: Geometric indexing does not improve versioned memory retrieval.

The idea that you can use the physical distance between dots on a map to link different versions of a fact is a dead end.

1. If you let the distance grow, the memory drifts away and gets lost.
2. If you stop the drift, you still pay a penalty that makes the memory worse than just storing the fact correctly in the first place.
3. If you try to use this for short, elliptical corrections (like "No, it's Beto"), the system fails completely, returning the wrong answer with total confidence.

The most important finding for anyone building these systems is a warning: If you save conversational turns without fixing the pronouns first, you lose 100% of the corrections. The system won't tell you it's broken; it will just confidently give you the wrong answer. The paper suggests that instead of trying to be clever with geometry, we should stick to simple, deterministic rules: fix the sentence, then save it.