💻 computer science

Discovery and Spatial Characterisation of Multiple Shortcut Groups for Auditing Vision Model Bias

This paper introduces a method to group per-image attribution maps into recurring spatial shortcut patterns using clustering techniques, enabling the identification of specific image subsets with high error rates and the development of targeted interventions that reduce performance disparities in vision models.

Original authors: Akshit Achara, Vishnunarayan Manickam, Thomas Day, Esther Puyol Anton, Alexander Hammers, Andrew P. King

Published 2026-08-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Akshit Achara, Vishnunarayan Manickam, Thomas Day, Esther Puyol Anton, Alexander Hammers, Andrew P. King

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a child to recognize different types of animals. You show them a picture of a cow in a green field, and they learn that "cow" means "animal with spots in a field." Then you show them a cow in a snowy barn, and they get confused because the "field" part is missing. This is exactly what happens with some artificial intelligence (AI) models. They are incredibly smart at spotting patterns, but sometimes they get lazy. Instead of learning the real features of the thing they are supposed to identify (like the shape of a cow), they grab onto easy, accidental clues (like the green grass) that happen to appear in their training photos. In the world of computer science, this is called "shortcut learning." It's like a student who memorizes the answer key's font style instead of actually studying the math. The problem is, these shortcuts work great until the test changes—like when the cow appears in a barn. When that happens, the AI fails, and if those failures happen more often for certain groups of people (like patients from a specific hospital), the AI becomes unfair or biased.

This paper is about a new way to catch these lazy shortcuts before they cause trouble. The researchers built a tool that acts like a "spatial detective" for AI. Instead of just looking at the final answer, they peek inside the model's brain to see exactly where in the image the AI is looking. They found that AI models often have multiple, distinct "shortcuts" they use for different pictures. Some pictures might trigger a shortcut based on the background, while others might rely on a weird artifact from the camera. By grouping these different shortcut patterns together, the researchers can spot exactly which images are most likely to fail and even fix the model on the fly by telling it to ignore the shortcuts and focus on the real clues.

The Detective Work: Finding the Shortcuts

The authors, a team from King's College London, set out to solve a specific mystery: existing methods could tell you that an AI was using shortcuts, but they couldn't tell you how or where it was using shortcuts in a way that helped fix specific groups of errors. Imagine trying to find a thief in a city by looking at a blurry, averaged map of all crimes. You'd know crime happens, but you wouldn't know if the thief is a pickpocket in the park or a burglar in the suburbs. The old methods were like that blurry map; they mashed all the images together, hiding the fact that different pictures had different kinds of shortcuts.

The team developed a method to break this down. First, they looked at three different "views" of the image for every single photo the AI saw:

The Task View: What the AI thinks is important for the job (e.g., identifying a tumor).
The Shortcut View: What the AI thinks is important for a sensitive trait (e.g., the hospital the photo came from, or the person's gender).
The Fair View: A reference model trained to be fair, ignoring the sensitive traits.

By comparing these views, they created a "contribution map" for every single image. This map highlights the specific pixels the AI used to make its decision. If the AI was using shortcuts, the map would light up in the background or on a specific artifact rather than the actual object.

Grouping the Patterns

Here is where the magic happens. The researchers took these thousands of individual maps and used a mathematical sorting technique (called K-means and Non-negative Matrix Factorization) to group them into "Shortcut Groups." Think of this like sorting a massive pile of mixed-up puzzle pieces into distinct boxes based on the patterns on the pieces.

They discovered that the shortcuts weren't just one big blob of bad behavior. Instead, there were several distinct "personalities" of shortcuts. For example, on a dataset of bird photos, they found one group of shortcuts where the AI looked at the water background, and another group where it looked at the land background. On a dataset of chest X-rays, they found shortcuts that focused on the breast shadow or the lower parts of the lung, while the "good" task-focused areas were right in the middle of the lungs.

These groups were incredibly useful. The researchers found that if they looked at just the top 20% of images that belonged to the "high-risk" shortcut groups, they could find a massive chunk of the model's total errors. In some cases, like with the bird dataset, this small group contained nearly 74% of all the mistakes the model made. This means auditors don't have to check every single image; they can just check the specific "shortcut groups" to find the problems quickly.

The "Do-Over" Button: Fixing the Model

The most exciting part of the paper is that they didn't just find the problems; they tried to fix them in real-time without retraining the whole model. They tested two types of "interventions" on the images:

Input Masking: They literally covered up the parts of the image the AI was using as a shortcut.
Feature Space Intervention: They went inside the model's internal processing and told it to "turn down the volume" on the shortcut signals and "turn up the volume" on the real task signals.

The results were fascinating. When they masked the "shortcut" areas, the model's performance often stayed the same or even got slightly better, proving those areas weren't actually necessary for the job. However, when they masked the "task" areas (the real clues), the model crashed, confirming those were the important parts.

The real win came from a combined approach: suppressing the shortcuts while amplifying the real task. When they did this, the gap in performance between different groups (like different genders or hospitals) shrank significantly. For instance, on the CelebA face dataset, this method reduced the performance gap by about 7.8 percentage points. On the CheXpert chest X-ray dataset, it reduced the gap by a whopping 20.3 percentage points.

What This Means

The paper suggests that by grouping these spatial shortcuts, we can create a much clearer picture of where AI models are failing. It's not just about saying "this model is biased"; it's about saying "this model is using shortcuts by looking at the background in these specific 8 types of patterns."

The researchers showed that these groups are stable and reliable. Even if they retrained the helper models used to detect the shortcuts, the groups remained consistent. They also tested this on different types of AI architectures (like ResNet and ViT) and different datasets (from skin lesions to bird photos), and the method worked across the board.

However, the authors are careful to note that this isn't a magic wand that solves everything. In some complex medical cases, like histopathology slides where the "shortcut" is a subtle staining difference from a specific lab, simply suppressing the spatial area wasn't enough to fix the bias. But for many other cases, this method of finding, grouping, and suppressing spatial shortcuts offers a powerful new way to audit and improve AI, making it fairer and more reliable without needing to start from scratch. It turns the blurry map of AI bias into a detailed, actionable guide for fixing the mistakes.

Technical Summary: Discovery and Spatial Characterisation of Multiple Shortcut Groups for Auditing Vision Model Bias

Problem Statement

Deep learning models often achieve high average accuracy by relying on "shortcut" features—spurious correlations present in the training distribution that do not generalize to test time, particularly within specific subpopulations. While out-of-distribution testing can reveal performance disparities, it fails to localize the specific image regions associated with these shortcuts. Existing attribution-based auditing methods, such as OSCAR, typically aggregate attribution maps across the entire dataset to produce a single dataset-level contribution map. This aggregation assumes spatial consistency of shortcut features across all images, potentially masking recurring spatial patterns that occur only in subsets of images or interact differently with task-relevant evidence in different subgroups. Consequently, auditors lack actionable spatial evidence to identify high-risk image subsets or understand the specific nature of shortcut reliance in fixed, trained models.

Methodology

The proposed framework, Spatial Shortcut Group Discovery, addresses these limitations by computing image-level contribution maps and grouping them to reveal recurring spatial patterns. The method proceeds in four main steps:

Image-Level Attribution and Ranking:
Three models are utilized: the audited task model ( $f_{TS}$ ), a bias-reduced baseline model ( $f_{BA}$ ) trained on an attribute-balanced dataset, and a sensitive attribute model ( $f_{SA}$ ). Attribution maps (e.g., LRP, AttnLRP) are generated for each model, partitioned into a grid of $R$ mutually exclusive regions, and converted into low-resolution spatial vectors. Crucially, these vectors are ranked within each image to preserve image-to-image variation.
Conditional Alignment and Contribution Maps:
Using partial correlation, the method computes image-level alignment scores:
- Shortcut Alignment ( $\rho_{sc}$ ): Measures the alignment between $f_{TS}$ and $f_{SA}$ after removing the linear effect of $f_{BA}$ . High scores indicate the task model uses regions associated with the sensitive attribute.
- Task Alignment ( $\rho_{task}$ ): Measures the alignment between $f_{TS}$ and $f_{BA}$ after removing the effect of $f_{SA}$ .
  Region-level contribution maps ( $m_{sc}$ and $m_{task}$ ) are derived by decomposing these alignment scores, retaining only positive contributions that support the respective alignments.
Spatial Pattern Grouping:
The joint shortcut and task contribution maps for each image are concatenated and normalized. These representations are then grouped into recurring spatial patterns using two approaches:
- K-means: Assigns each image to a single "shortcut group."
- Non-negative Matrix Factorization (NMF): Allows images to express multiple patterns simultaneously, providing soft membership weights.
  The result is a set of shortcut group prototypes (membership-weighted averages) and representative example images for each group.
Test-Time Spatial Interventions:
To validate the utility of these maps, the paper employs input masking and internal feature-space interventions.
- Input Masking: Occluding regions with high shortcut or task contributions to observe performance degradation.
- Feature-Space Interventions: Proposing shortcut suppression (removing task components from shortcut maps before suppression) and task amplification (removing shortcut components from task maps before amplification). A combined approach applies both simultaneously to reduce subgroup disparities while preserving accuracy.

Key Contributions

Spatial Shortcut Pattern Discovery: The paper introduces a method to compute image-level shortcut and task contribution maps and group them using K-means and NMF. This reveals that multiple coherent spatial shortcut patterns, with distinct subgroup compositions and error rates, can coexist within a single dataset, a nuance lost in dataset-level aggregation.
Targeted Failure Auditing: The framework demonstrates that shortcut group membership can be used to construct risk scores that identify subsets of images containing a disproportionately large fraction of the model's total test errors. This enables auditors to focus inspection efforts on high-risk subsets rather than random sampling.
Actionable Test-Time Interventions: The study validates that the identified spatial maps contain actionable evidence. Masking task-contribution regions degrades performance, while a combined intervention of shortcut suppression and task amplification generally reduces subgroup performance disparities (measured by the Largest Performance Gap, LPG) across various datasets and architectures.

Experimental Results

The method was evaluated across five datasets (CelebA, CheXpert, Waterbirds, Camelyon17, ISIC2019) and two architectures (ResNet50, ViT-B/16).

Pattern Discovery: Visual inspection of shortcut group prototypes revealed distinct spatial structures. For instance, in CelebA, some groups showed shortcut reliance concentrated in hair regions (task-relevant) while others focused on background/clothing. In CheXpert, shortcut groups highlighted non-lung regions (e.g., breast shadows), whereas task groups focused on lung regions.
Risk Identification: Shortcut group-based risk scores successfully identified subsets containing significantly more errors than random baselines. For example, on the CelebA ViT model, the top 20% of images ranked by risk score contained approximately 63.5% of total held-out errors (compared to the 20% expected by chance). NMF-based grouping generally outperformed K-means in error capture and stability across independently trained models.
Intervention Efficacy:
- Input Masking: Masking high task-contribution regions caused severe performance drops, confirming these regions are critical for prediction. Masking shortcut regions had variable effects, often preserving or slightly improving accuracy, suggesting these cues are non-essential or spurious.
- Feature Interventions: Combined shortcut suppression and task amplification consistently reduced the Largest Performance Gap (LPG) across most datasets. For instance, on CheXpert with ViT, the combined intervention reduced LPG by 12.8 percentage points while maintaining or slightly improving accuracy.
- Iterative Interventions: Repeated application of interventions further reduced shortcut alignment scores and subgroup disparities, demonstrating the maps' utility for iterative auditing.

Significance and Claims

The paper claims that spatial shortcut group discovery bridges the gap between visual inspection, targeted auditing, and test-time intervention. By moving beyond dataset-level aggregation, the framework provides a compact, inspectable summary of how shortcut reliance varies across images and subgroups.

The authors emphasize that their approach does not require retraining the model, making it applicable to fixed, deployed vision systems. The resulting prototypes and risk scores offer a practical workflow for auditors to:

Visualize recurring spatial patterns of bias that are otherwise hidden.
Prioritize subsets of data for manual review based on high error rates.
Intervene at test-time to mitigate disparities by suppressing spurious cues and amplifying task-relevant evidence.

The paper remains modest regarding generalization, noting that while the method handles single sensitive attributes effectively, real-world scenarios may involve multiple interacting attributes. Furthermore, the grid-based partitioning assumes some degree of spatial alignment, which may limit applicability to datasets with high spatial variability without further adaptation. The work establishes a foundation for using spatial contribution patterns to guide actionable interpretability in vision models.