Where To Look? : Causal Tracing of Vision Encoders in VLM

The Detective's Dilemma: How Do AI Eyes Actually See?

Imagine you are trying to teach a robot to understand the world. You give it a camera and a brain, and you ask it to describe a picture of a dog. For years, scientists have been amazed that these "Vision-Language Models" (VLMs) can do this with incredible accuracy. But a nagging question has remained: How do they actually do it? Do they really "see" the dog in the way we do, or are they just guessing based on tiny, hidden clues?

To answer this, researchers use a technique called causal tracing. Think of this like a detective's "what-if" scenario. If you take a specific piece of information out of the robot's brain and replace it with a different piece, does the answer change? If the answer changes drastically, that piece of information was "causally important." Usually, we assume that if a robot is talking about a dog, the most important part of its brain must be looking right at the dog. But what if the robot is actually paying attention to the grass, the sky, or the shadows, and just guessing the dog is there? This paper dives into that mystery, asking if the parts of the AI that matter most are actually looking at the right spot.

---

Where to Look? The Great AI Misdirection

In this study, the researchers at the LINGO Research Group decided to play a game of "spot the difference" with some of the smartest AI models in the world. They wanted to know: When an AI answers a question about an image, is the part of its brain that does the heavy lifting actually looking at the object in question?

To find out, they used a clever trick called activation patching. Imagine the AI's brain is a giant factory with thousands of workers (called "tokens") passing notes down a conveyor belt.

1. The Clean Run: They show the AI a clear photo and a question, like "Where is the lady with the blue shirt?" The AI gives a score for how well it understands the image.
2. The Corrupted Run: They take the same photo and add a bunch of static noise (like turning the TV to a snowy channel) so the AI gets confused and its score drops.
3. The Patched Run: This is the magic step. They take the "confused" noisy photo, but they sneak in a single, clean note from the original photo into the factory line. If the AI suddenly remembers the answer and the score goes back up, that specific note was the "hero" token.

The researchers then asked a simple question: Did that hero token come from the part of the image showing the lady in the blue shirt? To measure this, they used a math tool called IoU (Intersection over Union), which is just a fancy way of asking, "How much does this note overlap with the actual lady?"

The Shocking Discovery: The AI is Looking Everywhere Except the Right Place

The results were a bit like finding out your best friend is solving a math problem by staring at the ceiling instead of the numbers.

The team tested everything from the classic CLIP model to massive, modern giants like DeepSeek-VL, Qwen-2.5, and LLaVA. They expected to find that the most "causal" tokens (the heroes) would be clustered right on top of the target object. Instead, they found a weak, almost non-existent connection.

• The Numbers Don't Lie: In the CLIP model, the correlation between "being important" and "being in the right spot" was only 0.062. That is practically zero.
• The Pattern Holds: Even in the bigger, smarter models, the numbers stayed low. For DeepSeek-VL, the correlation was 0.0531 ± 0.0334 with one type of noise and 0.0520 ± 0.0041 with another. For Qwen-2.5, it was 0.0912 ± 0.0422.
• The "Zero Overlap" Surprise: In one specific example, a token that had a massive influence on the answer (a causal score of 0.718) had almost no overlap with the target object at all (an IoU of only 0.049). It was as if the AI solved the puzzle by looking at the background, not the subject.

The authors suggest that these models are incredibly good at using visual information, but they aren't necessarily grounded in the specific location of that information. They might be relying on a "distributed" web of clues—like the color of the sky, the texture of the floor, or the general vibe of the scene—rather than zooming in on the specific object.

Does Bigger Mean Better?

You might think, "Well, maybe the older, smaller models were just bad at this, and the new super-AIs have fixed it." The researchers checked this too. They looked at models from 2023 all the way to the latest 2025 releases.

The verdict? Nope. The pattern didn't change. Even the most advanced models, which can answer complex questions and follow instructions perfectly, still show this disconnect. The authors note that just because a model gets a higher score on a test doesn't mean it has developed a better "spatial map" of the image. It seems that strong performance doesn't automatically mean the AI is looking at the right place; it just means it's very good at guessing the right answer using whatever clues it can find.

The "String" Test: Do They Understand Shapes?

To dig even deeper, the team asked a different question: If you take away the "look" of an object but keep its shape, does the AI still get it? They used a task involving strings (lines) where they kept the geometry (the shape and connections) the same but changed the appearance.

They found that the models exploit visual cues to understand visual structures. When those appearance-based cues were removed, the models struggled to maintain their understanding of the structure. This suggests that the AI isn't really "seeing" the shape in a deep, structural way independent of its look; it relies heavily on those surface-level appearance tricks to make sense of the connections. When those tricks are gone, the AI gets lost.

What This Means for the Future

The paper concludes with a clear, if slightly unsettling, message: There is a gap between "seeing" and "reasoning."

The models are not broken; they are just working in a way we didn't expect. They are like a detective who can solve a crime by noticing that the suspect always wears a red hat, without ever actually looking at the suspect's face. The "causal" parts of their brain are doing the work, but they aren't necessarily looking at the target.

The authors are now planning to look at even more models and different types of camera architectures to see if this is a universal rule for all modern AI or just a quirk of the ones they tested. For now, they suggest that we shouldn't assume that because an AI can describe an image perfectly, it is actually "looking" at the object in the way we think it is. The AI might be looking at the whole picture, but missing the specific spot we care about.