💻 computer science

Metonymy in vision models undermines attention-based interpretability

This paper reveals that modern pretrained vision transformers violate the locality assumption by exhibiting "intra-object leakage" (visual metonymy), which undermines the faithfulness of attention-based interpretability methods for part-based reasoning, and demonstrates that a two-stage approach can effectively mitigate this issue to improve attribute-driven part discovery.

Original authors: Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Massimiliano Mancini, Diego Marcos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Cheating" Detective

Imagine you are trying to teach a computer to recognize a bird by looking at its parts. You want the computer to say, "I know this is a blue jay because I see a blue wing," or "This is a robin because I see a red breast."

To make this work, researchers use a special tool called an attention mechanism. Think of this like a detective's magnifying glass. When the computer looks at a bird, the magnifying glass highlights the specific part it's looking at (like the wing) and ignores the rest of the bird. The idea is: If the computer focuses on the wing, it should only use information from the wing to make its decision.

The Paper's Discovery:
The authors found that modern AI models are "cheating." Even when the magnifying glass is focused strictly on the wing, the computer is actually "smelling" the rest of the bird through the walls.

In linguistics, this is called Metonymy. It's like saying "The White House issued a statement" when you really mean "The President issued a statement." The part (the building) stands in for the whole (the administration).

In these AI models, the "wing" part doesn't just contain information about feathers; it secretly contains information about the "head," the "legs," and the "beak."

The Analogy: Imagine you are trying to guess a person's favorite food by looking only at their left hand. In a normal world, your hand tells you nothing about their stomach. But in these AI models, the hand is magically connected to the stomach. If the person likes pizza, their left hand glows with "pizza energy," even though you aren't looking at their mouth.

Why This Matters: The "Faithful" Lie

The paper argues that this makes current "interpretable" AI untrustworthy.

Many researchers believe their models are "interpretable" because they can show you a heatmap (a glowing map) that says, "Look, I'm looking at the beak!"

The Reality: The model is looking at the beak, but it's making its decision based on the legs because the "beak" representation is contaminated with "leg" information.
The Result: The explanation looks honest, but the reasoning is a lie. The model isn't actually reasoning about the part it's pointing to; it's using a shortcut.

The Solution: The "Soundproof Room"

The authors propose a new way to build these models to stop the cheating. They call it a Two-Stage Approach.

Stage 1: The Detective Finds the Rooms.
First, the model looks at the whole bird and draws outlines around the different parts (head, wing, leg). It's like a real estate agent identifying the rooms in a house.

Stage 2: The Soundproof Rooms.
Instead of looking at the whole house at once, the model puts each room into a separate, soundproof box.

It takes the "Wing" box and puts it in a room where no sound from the "Head" or "Leg" can get in.
It forces the computer to make a guess about the wing using only the information inside that soundproof box.

The Analogy:
Think of a group project where everyone is in the same room. If you ask the "Wing" student for their opinion, they might just repeat what the "Head" student said because they can hear them.
The authors' solution is to put every student in a separate, soundproof booth. Now, if you ask the "Wing" student, they have to come up with an answer based only on what they know about wings. They can't cheat by listening to the others.

The Results: Does It Work?

The authors tested this on three different types of "objects":

Birds (CUB dataset): Checking if the model could identify wing colors or beak shapes.
Faces (CelebA dataset): Checking if the model could identify eye color or if someone was wearing a hat.
Chest X-rays (CheXpert dataset): Checking if the model could spot lung issues.

What they found:

Old Way (Late Masking): The "Wing" student could easily guess the "Leg" color. The model was confused and unfaithful.
New Way (Two-Stage/Soundproof): The "Wing" student could only guess wing things. The model became much better at actually understanding the specific parts.
The Trade-off: Building these soundproof rooms takes more computer power (about twice as much), but it makes the model's reasoning honest and accurate.

The Takeaway

The paper concludes that if you want an AI to explain why it made a decision based on a specific part of an image, you cannot just use standard models. They are too "noisy" and leak information from other parts. You need to build them with architectural isolation (the soundproof rooms) to ensure that when the AI points to a part, it is actually thinking about that part, and not just guessing based on the whole object.

In short: The paper proves that current AI models are lying about what they are looking at, and offers a "soundproof" construction method to force them to tell the truth.

Technical Summary: Metonymy in Vision Models Undermines Attention-Based Interpretability

1. Problem Definition

The paper addresses a critical flaw in the interpretability of modern deep learning vision models, specifically those relying on part-based reasoning and attention mechanisms. Many interpretable-by-design models, such as Concept Bottleneck Models (CBMs) and part-centric attention networks, operate under a locality assumption: the latent representation of a specific object part (e.g., a bird's wing) is assumed to encode primarily information about that specific image region.

The authors argue that this assumption is frequently violated by modern pretrained Vision Transformers (ViTs). Due to the large receptive fields of these architectures, the latent representation of a single part often encodes information from the entire object. The authors term this phenomenon intra-object leakage or visual metonymy, where a part comes to stand for the whole. Consequently, when an attention map highlights a specific part, the underlying features used for prediction may actually contain "leaked" information from other parts. This renders the model's reasoning unfaithful; the model appears to focus on the correct region but is actually relying on correlated features from elsewhere, undermining the trustworthiness of attention-based explanations, particularly in high-stakes domains like medical imaging.

2. Methodology

2.1 Benchmarking Intra-Object Leakage

To quantify this phenomenon, the authors propose two evaluation metrics using datasets with part-level attribute annotations (CUB-200-2011, CelebA, and CheXpert):

Part Specificity (PS): Measures the difference in attribute classification performance when using features from the "correct" part versus features from "incorrect" parts. A low PS indicates that features from non-matching parts are just as predictive as those from the matching part, signaling leakage.
Most Predictive Part Overlap (MPPO): Evaluates whether the discovered part with the highest prediction confidence for a specific attribute spatially overlaps with the ground-truth region of that attribute.

The authors compare Late Masking (standard approach: extracting features from a frozen backbone and applying masks post-hoc) against Early Masking (applying masks directly to the input image tokens before feature extraction). Early masking serves as an upper bound that prevents leakage by design.

2.2 Proposed Solution: Two-Stage Architecture

To mitigate intra-object leakage, the authors propose a two-stage transformer architecture:

Stage 1 (Attribute-Driven Part Discovery): Uses a frozen backbone (e.g., DINOv3) to generate attention masks that localize semantic parts. This stage employs a novel de-correlated orthogonality loss ( $L_{\perp}$ ) to ensure parts are distinct while allowing them to share common background features, avoiding the suppression of shared signals found in previous orthogonality losses.
Stage 2 (Strict Parallel Processing): Processes each discovered part independently in parallel streams.
- Mechanism: The model replicates prefix tokens (e.g., [CLS]) for each part and applies a block-diagonal attention mask. This ensures that the $k$ -th prefix token attends only to image patches within the $k$ -th part mask, strictly isolating features.
- Gradient Estimation: To enable end-to-end training where Stage 2 guides Stage 1, the authors introduce a Straight-Through Estimator (STE). The forward pass uses hard (binary) masks to guarantee zero leakage, while the backward pass uses soft (continuous) masks to allow gradients to flow and refine the part discovery masks.

3. Key Contributions

A Benchmark for Intra-Object Leakage: The introduction of PS and MPPO metrics to systematically measure the violation of locality in vision backbones.
Empirical Evidence of Widespread Leakage: A comprehensive evaluation showing that state-of-the-art pretrained backbones (DINOv1, DINOv2, DINOv3, CLIP, MAE) exhibit strong intra-object leakage, violating the locality assumption required for faithful interpretability.
A Mitigation Strategy: A two-stage architecture that enforces strict spatial isolation by design, demonstrating that preventing leakage improves both the faithfulness of attention maps and the quality of part discovery.

4. Results

4.1 Quantitative Findings

Prevalence of Leakage: Across CUB and CelebA, Early Masking consistently yields significantly higher Part Specificity (PS) scores than Late Masking (e.g., ~3x improvement on CUB, ~10x on CelebA for DINOv3). This confirms that standard backbones rely heavily on leaked information.
Medical Imaging (CheXpert): In the CheXpert dataset, standard Late Masking results in a Most Predictive Part Overlap (MPPO) of ~0.53, indicating predictions often stem from outside the pathology region. Enforcing Early Masking improves MPPO to ~0.62.
Part Discovery Improvement: The proposed two-stage STE variant significantly outperforms single-stage baselines and existing state-of-the-art methods (PDiscoFormer, MPAE) in clustering metrics (ARI and NMI).
- On CUB, STE achieves an ARI of 73.93 vs. 50.27 for the single-stage baseline.
- On CelebA, STE achieves an ARI of 97.51 vs. 49.46 for the baseline.
Predictive Performance: While enforcing strict locality (Stage 2 only) can slightly reduce raw attribute prediction accuracy (as the model can no longer exploit correlated shortcuts), ensembling the global context (Stage 1) with the strictly local evidence (Stage 2) yields the highest overall performance across all datasets, surpassing the single-stage baseline.

4.2 Qualitative Observations

Mask Expansion: Single-stage models and soft-masking variants tend to produce "lazy" masks that cover only small, highly discriminative patches, relying on leakage for the rest of the context. The STE variant produces more spatially expanded masks that cover the full semantic region (e.g., covering both lungs in chest X-rays rather than just one).
Faithfulness: The two-stage approach ensures that attribute predictions are grounded in the correct anatomical regions, whereas single-stage models often fail to respect spatial boundaries.

5. Significance and Claims

The paper claims that attention-based interpretability is fundamentally compromised if the underlying latent representations already entangle information across spatial regions. Merely visualizing attention maps is insufficient if the features themselves are contaminated by metonymy.

The authors argue that:

Locality is a Prerequisite: Faithful, compositional part-based reasoning requires representation-level solutions that enforce locality, not just post-hoc attention mechanisms.
Two-Stage Approaches are Promising: By applying attention masks directly to the input image (Early Masking) rather than deep features, it is possible to mitigate leakage and improve both interpretability and part discovery quality.
Trade-offs are Manageable: While the two-stage approach incurs a computational overhead (approximately 2.1x slower inference), the ability to ensemble global and local signals allows for models that are both highly accurate and strictly faithful to local evidence.

The work highlights a neglected issue in the interpretability of part-based representations, suggesting that future research must address representation-level entanglement to build truly trustworthy vision models.