From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models
This paper introduces a unified iERF-centered framework that integrates local, global, and mechanistic interpretability through Sharing Ratio Decomposition, Concept-Anchored Feature Explanation, and Interlayer Concept Attribution to provide robust, evidence-backed explanations of how vision models transform pixels into decisions.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a super-smart robot that looks at a photo and instantly knows, "That's a dog!" or "That's a cat!" But if you ask the robot why it thinks that, it just says, "Because I said so." It's a black box.
This paper introduces a new way to open that black box. The authors built a unified system to answer three big questions about how these vision models work:
- Local: Where exactly is the robot looking?
- Global: What specific ideas or "concepts" has the robot learned?
- Mechanistic: How does the robot combine those small ideas to make a final decision?
To do this, they created a single "super-unit" called the PFV-iERF bundle. Think of this as a GPS-tagged memory card.
- PFV (Pointwise Feature Vector): This is the "memory card." It holds a specific piece of information the robot found (like "fur texture" or "pointy ears").
- iERF (Instance-specific Effective Receptive Field): This is the "GPS tag." It tells you exactly which pixels in the original photo triggered that memory.
Here is how their system works, broken down into three parts:
1. The Local View: "The Detective's Magnifying Glass"
The Problem: Old methods of explaining AI often give blurry heatmaps or get confused when the image is slightly changed (like a tiny bit of noise). They might say, "The robot is looking at the whole dog," but they can't pinpoint the exact pixels that mattered.
The Solution (SRD): The authors use a method called Sharing Ratio Decomposition (SRD).
- The Analogy: Imagine a relay race. The robot passes a baton (information) from one runner to the next. SRD tracks exactly how much of the baton's speed came from the previous runner.
- How it works: Instead of guessing, it mathematically breaks down the robot's internal math to see exactly how much each pixel contributed to the final decision.
- The Result: It creates a sharp, high-definition map showing exactly which pixels the robot used. It's so precise that even if you try to trick the robot with tiny, invisible changes to the image, the map stays honest and doesn't get confused.
2. The Global View: "The Dictionary of Ideas"
The Problem: Sometimes, the robot learns concepts that are scattered. For example, a concept like "yawning" might be triggered by a mouth in one corner of the image and a tongue in another. Old methods try to find these concepts by looking at the "loudest" part of the image, which fails when the concept is spread out (non-localized). This is especially true in modern models (Transformers) that mix information from everywhere at once.
The Solution (CAFE): They use Concept-Anchored Feature Explanation (CAFE).
- The Analogy: Imagine the robot has a dictionary of abstract words (concepts). Old methods try to define a word like "Despair" by showing you the picture where the word appeared the most. But what if "Despair" is triggered by a spilled pill bottle in the corner, while the robot's "attention" is focused on the background? The old method gets it wrong.
- How it works: CAFE uses the "GPS tag" (iERF) to look behind the scenes. It ignores where the robot's attention is currently focused and instead asks, "Which specific pixels actually caused this concept to light up?"
- The Result: It can correctly identify that a concept like "Three" means "counting three objects," even if those objects are scattered across the whole image. It anchors abstract ideas to real, verifiable pixels.
3. The Mechanistic View: "The Family Tree of Ideas"
The Problem: We know what the robot sees and where it looks, but we don't know how it builds its final answer. How does "fur" + "ears" + "tail" become "Dog"?
The Solution (ICG & ICAT): They built an Interlayer Concept Graph.
- The Analogy: Think of the robot's brain as a multi-story building. The ground floor sees simple things (lines, colors). The top floor makes the final decision. This graph draws a family tree connecting the ideas from the ground floor to the top floor.
- How it works: They trace the "ancestry" of an idea. They ask, "Did the 'Dog' idea on the top floor come from the 'Fur' idea on the bottom floor?" They use a method called ICAT (Interlayer Concept Attribution) to measure the strength of these connections.
- The Result: They found that Integrated Gradients is the best tool for drawing this family tree.
- Success Story: They showed how the robot correctly identifies a "House Finch" by tracing a path from a tiny "red" pixel all the way up to the final decision.
- Failure Story: They showed how a robot gets tricked. In one case, a "Tiger Cat" was mistaken for a "Boxer Dog" because a "striped face" concept from the cat's face accidentally triggered a "dog" path. In another, a "Border Collie" was hacked to look like a "Sea Slug" because a tiny "wrinkle" concept was amplified to override the dog's identity.
Summary
The paper claims that by using this PFV-iERF unit (the GPS-tagged memory card), they can:
- Draw sharper, more honest maps of where the robot is looking (Local).
- Correctly define abstract concepts even when they are scattered across the image (Global).
- Trace the exact path of how small visual clues combine to form big decisions, revealing why the robot makes mistakes or gets hacked (Mechanistic).
They tested this on various models (like ResNet and Vision Transformers) and found it works better and is more robust against tricks than previous methods. It provides a clear, evidence-backed story from the raw pixels all the way to the final decision.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.