Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression

Imagine a world where giant, super-smart computers act like tireless detectives, solving complex mysteries by writing out their entire thought process step-by-step before giving you the final verdict. This is how modern "Large Reasoning Models" work: they don't just guess the answer; they build a long, logical story to prove why they are right. But these stories can get incredibly long, taking up massive amounts of the computer's memory, kind of like a detective trying to keep every single clue, witness statement, and map in their head at once. To make these computers faster and cheaper to run, scientists have invented a trick called "KV cache compression." Think of this as a magical filing system that throws away the "boring" or "redundant" parts of the detective's notes to save space, keeping only the most important bits so the computer can still solve the case.

For a long time, everyone assumed that if this filing system kept the final answer correct, it must have also kept the important clues that led to that answer. It seemed logical: if the detective says "The butler did it," and the answer is right, then the notes must have been good, right? But a new study suggests this might be a dangerous illusion. The researchers found that you can have a detective who gets the right answer but has completely forgotten (or thrown away) the evidence that proves it. They discovered that the computer might be "hallucinating" a correct conclusion based on a broken chain of logic, and because we were only checking the final answer, we didn't notice the detective was actually making things up. This is a big deal because in real life, like in medicine or science, knowing why an answer is correct is just as important as the answer itself.

The Great "Right Answer, Wrong Reason" Mystery

In this paper, the researchers from the University of Illinois Urbana-Champaign decided to put this "magic filing system" to the test. They wanted to see if compressing the computer's memory actually preserves the reasoning behind the answer, or if it just preserves the answer itself while shredding the proof. To do this, they set up a clever experiment that acts like a time-travel replay.

First, they let a super-smart computer (without any memory compression) solve a bunch of hard problems, like tricky math puzzles, scientific questions, and medical calculations. They saved the computer's entire "thought trace"—the full, step-by-step story it wrote down. Then, they took that exact same story and asked different compression methods to "replay" the ending. The computer was forced to use the compressed, memory-saving version of its notes to finish the story. The key here is that the text of the story was fixed; the only thing that changed was the computer's internal memory of what it had just read. This allowed the researchers to isolate exactly what the compression was doing to the computer's ability to understand its own notes.

They tested eleven different compression methods, including ones that throw away old tokens, ones that merge similar ideas, and one that just shrinks the size of the notes without throwing anything away. They looked at three things:

1. Final Accuracy: Did the computer get the right answer?
2. Chain Consistency: Was the story it told actually logical and correct, or did it skip steps and make things up?
3. Faithfulness: If they slipped a wrong answer into the middle of the story, would the computer catch the mistake, or would it blindly follow the error?

The Shocking Discovery: The "Answer-Evidence Gap"

The results were eye-opening. The researchers found a huge gap between getting the right answer and having the right evidence. They call this the "Answer-Evidence Gap."

Here's what happened: On difficult math and science tasks, many of the compression methods were able to produce the correct final answer at a rate that looked almost as good as the full, uncompressed computer. However, when the researchers checked the reasoning behind those answers, it was a disaster. The computer was often producing "wrong-chain correct answers." This means the final number or choice was right, but the path to get there was full of holes, logical leaps, or made-up facts.

For example, on a math test called AIME, some compressed methods got the right answer about 50% of the time. But when they looked at how the computer got there, they found that in many of those "correct" cases, the computer had skipped crucial steps, used the wrong formulas, or just guessed the answer and then tried to write a fake story to justify it. It's like a student who gets the right answer on a test but wrote "I used magic" as their explanation.

The study showed that this gap is especially bad for methods that evict (throw away) parts of the memory. These methods seem to keep the "answer cues"—the bits of text that look like the final conclusion—while tossing out the "evidence support," like the intermediate calculations and verification steps. It's as if the filing system kept the "Case Closed" stamp but threw away all the police reports and fingerprints.

Interestingly, the researchers found that quantization (a method that shrinks the size of the notes without deleting any of them) performed much better. It kept the reasoning chain mostly intact, suggesting that the problem isn't just about having less memory, but specifically about losing access to the parts of the reasoning trace that prove the answer is right.

Why This Matters: The Danger of "Fake" Competence

The paper argues that relying only on "Final Accuracy" is a trap. If you only check if the answer is right, you might think a compressed computer is working perfectly. But in reality, it might be a "competent liar"—it gives you the right answer, but you can't trust it because the reasoning is broken.

The researchers tested this by "perturbing" the stories. They inserted a clearly wrong answer into the middle of the computer's notes and asked it to continue. The uncompressed computer usually caught the error and corrected itself. But the compressed computers? Many of them just went along with the lie, adopting the wrong answer because the evidence needed to spot the mistake had been thrown away.

This is a critical finding for anyone using these AI models in high-stakes situations, like diagnosing a patient or solving a complex engineering problem. If a doctor relies on an AI that says "The patient has Condition X" (which happens to be true), but the AI's reasoning is a jumbled mess of incorrect facts, the doctor has no way of knowing if the AI is actually right or just lucky. The paper suggests that we need new ways to evaluate these computers—one that checks not just what they say, but how they got there.

The Bottom Line

The study concludes that while compression can save memory and speed up computers, it often does so at the cost of reasoning faithfulness. The "Answer-Evidence Gap" is real: compressed models can preserve the final answer while degrading the validity of the support behind it. The authors suggest that future compression methods shouldn't just try to keep the "most important" tokens based on how often they appear; they need to be smarter about keeping the structure of the reasoning—the definitions, the intermediate steps, and the verification checks. Until then, we should be careful about trusting AI models that get the right answer but can't explain how they did it, because in the world of large reasoning models, a right answer without a valid reason is just a lucky guess.