🤖 AI

From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios

This paper introduces the Normalised Sensitivity Ratio (NSR), a post-hoc, model-agnostic diagnostic that identifies causal versus spurious features in trained models by leveraging sensitivity stability across structured environmental shifts, achieving exact identification under specific conditions and demonstrating high accuracy on both synthetic and real-world datasets.

Original authors: Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz, Sotirios Tsaftaris

Published 2026-07-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz, Sotirios Tsaftaris

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out why a magic crystal ball predicts the future. You know the crystal ball works, but you don't know how. Did it learn the real laws of physics, or did it just get lucky by noticing that the sky is always blue on Tuesdays? This is the world of machine learning, where computers learn to make predictions by finding patterns in data. Sometimes, they find the right patterns (like "rain makes grass wet"), but often they find "spurious" patterns—coincidences that look real but break as soon as the situation changes (like "Tuesdays cause rain").

The big problem is that once a computer model is trained, it's like a black box. We can't easily peek inside to see if it's relying on the truth or just a lucky guess. Previous tools to fix this required us to go back to the drawing board and retrain the model from scratch, which is slow and expensive. This paper introduces a new way to peek inside a finished model without changing a single thing about how it was built. It's like having a special flashlight that can tell you, just by watching how the model reacts to different rooms, whether it's looking at the real furniture or just the shadows on the wall.

The Detective's New Flashlight: NSR

The authors of this paper, Athanasios Vlontzos and his team, have invented a tool called the Normalised Sensitivity Ratio (NSR). Think of a trained AI model as a chef who has perfected a soup recipe. The chef knows that if they add a pinch of salt (a "causal" ingredient), the soup tastes better, no matter where they are cooking. But if the chef learned that "whenever it's raining outside, the soup tastes salty" (a "spurious" correlation), that rule only works in the kitchen where it rains. If the chef moves to a sunny kitchen, the soup might taste terrible because the rain isn't there to trigger the "saltiness."

The NSR tool works by asking the chef to taste the soup in different kitchens (environments).

The Causal Test: If the chef adds salt, the taste changes the same amount in every kitchen. The sensitivity is constant.
The Spurious Test: If the chef relies on the "rain" rule, the taste changes wildly depending on whether it's raining in that specific kitchen. The sensitivity shifts with the environment.

The NSR measures this "wobble." If the model's reaction to a feature is steady across different environments, NSR says, "This is a real, causal feature!" If the reaction jumps around, NSR says, "This is a spurious coincidence!"

How the Magic Works (Without the Magic)

The paper proves that if you have at least three different environments (like three different hospitals or three different batches of data) where the "spurious" things (like equipment noise or weather) change, but the "real" causes stay the same, the NSR tool can perfectly separate the truth from the lies.

The authors ran this idea through a gauntlet of tests:

Perfect Detection: In their computer simulations, when the conditions were right—specifically with at least 5 environments and a shift magnitude of 1 or greater—the tool achieved a perfect score. Under these specific regime conditions, it identified every single causal feature and every single spurious one with 100% accuracy (an AUROC of 1.000). It was like finding a needle in a haystack without ever touching the hay.
The "No-Go" Zones: The paper is very honest about when the tool fails. If you only have two environments, the tool goes blind. If the changes in the environments are too symmetrical (like a perfect mirror image), the tool gets confused. They also found that if the "shift" between environments is tiny compared to the noise, the tool can't see it. It's like trying to hear a whisper in a hurricane; the signal gets lost.
Real-World Test: They tried it on real data, specifically a bike-sharing dataset from a city. They knew which factors (like temperature and time of day) actually caused people to rent bikes, and which were just coincidences. Without retraining the model at all, the NSR tool correctly identified 6 out of 8 of the true causal factors. It did much better than other standard methods that just look at the size of the numbers in the model.

Why This Matters

The most exciting part of this discovery is that it works after the model is already trained. Usually, if you suspect a model is cheating by using spurious patterns, you have to throw it away and start over. With NSR, you can just take the finished model, shine the flashlight, and say, "Ah, I see you're relying on the day of the week instead of the actual medical symptoms."

The authors show that this method is a "post-hoc" diagnostic, meaning it's a check-up you can give to a model that's already on the job. It doesn't need to know the model's internal code, and it doesn't need to touch the training process. It just watches how the model behaves when the world changes around it.

However, the paper also draws a clear line in the sand. This tool works best when the "real" causes (like biology or physics) stay stable, and only the "noise" (like different machines or locations) changes. If the real causes themselves change from place to place (like if the definition of "sickness" changes between hospitals), the tool might get confused and think a real cause is just a coincidence. The authors are careful to say this is a limitation, not a bug.

In short, this paper gives us a new way to trust our AI. It provides a mathematical guarantee that if the world shifts in a specific way, we can spot the liars in the model's logic. It turns the black box into a glass box, letting us see exactly what the model is relying on, so we can be sure it's not just guessing based on the weather.

Technical Summary: Post-Hoc Causal Feature Identification via Sensitivity Ratios

Problem Statement

The paper addresses a critical gap in machine learning interpretability: determining which features a pre-trained model relies on causally versus spuriously, without access to the training procedure, the model class, or the underlying causal graph.

Existing methods for causal feature identification, such as Invariant Risk Minimization (IRM) and Invariant Causal Prediction (ICP), require intervention during the training phase (e.g., modifying loss functions or performing exhaustive subset searches). Consequently, they cannot diagnose models that have already been deployed. The authors focus on the structured-shift regime, a scenario common in multi-site clinical data or multi-batch genomics, where environments differ primarily in the mean of spurious features (e.g., equipment offsets, batch effects) while the causal mechanism and the distribution of causal features remain stable. In this setting, a model relying on spurious features will exhibit environment-dependent sensitivity, whereas a model relying on causal features will show constant sensitivity.

Methodology: Normalised Sensitivity Ratio (NSR)

The authors propose the Normalised Sensitivity Ratio (NSR), a post-hoc, model-agnostic diagnostic metric.

Core Concept

NSR quantifies the stability of a model's sensitivity to a specific feature across different environments.

Causal Sensitivity Ratio (CSR): For a feature $j$ and environments $e$ and a reference $e_{ref}$ , the CSR measures the average shift in the model's output when feature $j$ is replaced by a draw from the reference environment's marginal distribution, holding other features fixed.
Normalization: NSR is defined as the squared coefficient of variation ( $CV^2$ ) of the CSR values across environments, normalized by the mean CSR.
$\text{NSR}_j(e_{ref}) := \frac{\text{Var}_{e \neq e_{ref}}[\text{CSR}_j(e, e_{ref})]}{(\mathbb{E}_{e \neq e_{ref}}[\text{CSR}_j(e, e_{ref})])^2}$

Key Properties

Weight Invariance: The normalization cancels out the learned model weights ( $\hat{w}_j^2$ ). This ensures that spurious features with small coefficients (which might be missed by weight-based analysis) are still detected if their sensitivity varies across environments.
Diagnostic Signal: Under the structured-shift regime:
- Causal Features: Induce constant sensitivity across environments $\implies \text{NSR}_j = 0$ .
- Spurious Features: Induce sensitivity that tracks the environmental shift $\implies \text{NSR}_j > 0$ .
Implementation: NSR requires only a trained black-box model, environment labels, and held-out data. It can be computed using forward passes, SHAP values, or permutation importance as drop-in estimators for the sensitivity signal.

Key Contributions

1. Exact Identification Theorem

Under a linear Structural Causal Model (SCM) in the structured-shift regime with $K \ge 3$ non-degenerate environments, NSR achieves exact identification.

Theorem 1: For any linear predictor with non-zero weights, $\text{NSR}_j = 0 \iff j$ is a causal feature.
The proof demonstrates that the identification signal arises from the geometry of the environment shifts, not the model weights. This theoretical result extends empirically to non-linear models.

2. Complete Characterization of Failure Modes

The paper provides precise quantitative criteria for when NSR fails, allowing practitioners to assess regime validity:

Weak Shifts: NSR collapses as $O(\epsilon^4)$ when the shift magnitude $\epsilon$ is small relative to within-environment noise.
Degenerate Geometry: NSR is identically zero if $K < 3$ or if the absolute shifts from the reference environment are symmetric (e.g., shifts of $+1, -1, 0$ ).
Proxy Attenuation: For features mixing causal and spurious signals ( $X_j = \alpha X_C + (1-\alpha)X_S$ ), NSR degrades as $O((1-\alpha)^4)$ .

3. Finite-Sample Convergence Rates

The paper establishes a rate gap between causal and spurious features:

Under the null hypothesis (causal feature, $\text{NSR}=0$ ), the empirical estimator converges at $O_p(n^{-1})$ .
Under the alternative (spurious feature, $\text{NSR}>0$ ), it converges at $O_p(n^{-1/2})$ .
This gap enables permutation-based thresholding without distributional assumptions, as the null distribution shrinks to zero faster than the alternative separates.

4. Incomparability with IRM and ICP

The authors prove that NSR, IRM, and ICP operate under incomparable conditions regarding shift schedules.

There exist shift configurations (e.g., symmetric shifts) where IRM and ICP detect spurious features but NSR does not.
Conversely, there are configurations (e.g., asymmetric shifts where squared variances are zero) where NSR detects spurious features but IRM does not.
The methods are complementary; the choice depends on the specific geometry of the available environmental shifts.

Experimental Results

Synthetic Data Validation

Regime Verification: Experiments confirm the theoretical phase transition. With $K \ge 5$ and shift magnitude $\epsilon \ge 1$ , NSR achieves an AUROC of 1.000 with zero variance across 30 seeds.
Failure Modes: The experiments validate the sharpness of failure conditions. Symmetric shifts (degenerate geometry) result in chance-level performance (AUROC $\approx$ 0.5), while breaking symmetry restores perfect detection.
Convergence: Empirical convergence rates match the theoretical $O(n^{-1})$ and $O(n^{-1/2})$ predictions within 2-8% error.

Model Agnosticism

NSR was tested across five model families (OLS, Ridge, Random Forest, XGBoost, MLP). Rankings of features were consistent across families (Kendall $\tau \ge 0.529$ ), confirming the signal resides in the environment structure rather than the specific model class.

Real-World Applications

Bike-Sharing Data: Using month-quintile partitions, NSR recovered 6 of 8 causal features (Precision@7 = 0.75), outperforming LASSO (0.58) without modifying the trained model.
Wine Quality Data: NSR identified causal drivers in a setting with batch-level technical variation.
Practical Insight: The experiments highlight that NSR requires the base model to generalize within environments (not overfit). Adaptive regularization (targeting $R^2 \approx 0.75$ ) was necessary to prevent environment-specific noise from obscuring the stability signal.

Significance and Scope

The paper claims NSR provides a theoretically grounded, post-hoc audit tool for deployed models. Its significance lies in:

Operational Utility: It allows practitioners to diagnose spurious reliance in existing models without retraining or access to training data.
Explicit Boundaries: Unlike many "black box" interpretability tools, NSR comes with a complete map of its operating regime and failure modes, providing quantitative checks (e.g., verifying shift magnitude and environment count) before application.
Complementarity: It fills a niche distinct from IRM and ICP, specifically targeting scenarios where the shift geometry is asymmetric and the model is already trained.

Limitations Acknowledged by the Authors:

Marginal Causal Stability: NSR assumes the distribution of causal features ( $P(X_C)$ ) is stable across environments. If causal covariates shift (e.g., demographic changes), NSR may misclassify them as spurious.
Shift Structure: The current formulation assumes a uniform scalar shift across spurious features. Feature-specific or covariance shifts are not directly captured without subspace extensions.
Regime Dependency: The method is strictly a diagnostic for the structured-shift regime; it is not a general-purpose causal discovery tool for arbitrary data distributions.