🤖 machine learning

Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect

This paper introduces UNIVERSE, a novel framework that leverages Rashomon sets to provide robust bounds on variable importance in the presence of unobserved confounding and model multiplicity, thereby addressing critical limitations in standard feature selection and hypothesis generation.

Original authors: Jon Donnelly, Srikar Katta, Emanuele Borgonovo, Cynthia Rudin

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jon Donnelly, Srikar Katta, Emanuele Borgonovo, Cynthia Rudin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of data science, researchers often try to understand what drives a specific outcome, such as whether a loan will be repaid or if a patient will recover. To do this, they build mathematical models that look at various factors, or features, like income, credit history, or age. A standard part of this process is determining which of these factors are truly important. Scientists have long relied on methods that assign a score to each feature based on how much the model's performance drops if that feature is removed or shuffled. This helps them decide which variables to collect in the future and which to ignore. However, this standard approach has a hidden flaw: it usually assumes that the model being analyzed is the only good one. In reality, many different models can explain the same data equally well, a phenomenon known as the Rashomon effect. Furthermore, real-world data is often incomplete; crucial information, such as a person's exact income or health history, might be missing from the dataset entirely. When these missing pieces exist, the standard methods can produce misleading conclusions, making a feature look important when it is not, or hiding a feature that actually matters.

A team of researchers from Duke University and Bocconi University has developed a new framework called UNIVERSE to solve these problems. Their work addresses the twin challenges of having multiple equally good models and having missing data that could skew the results. Instead of picking a single best model and analyzing it in isolation, the researchers look at the entire group of models that perform nearly as well as the best one. They call this group a Rashomon set. By examining the entire set, they can see how the importance of a feature changes across different valid models. But they go a step further. They also account for the fact that some important variables are missing from the data. They do this by creating a safety net of mathematical bounds. These bounds act like a range of possible values, guaranteeing that the true importance of a feature falls somewhere within that range, even if the researchers do not know exactly what the missing variables are or which specific model is the true one.

The researchers tested their method using a clever simulation technique. They took real-world datasets, such as records of criminal recidivism, student college dropout rates, and credit risk assessments, and artificially created a scenario where some data was hidden. They split the data into groups and trained different models on each group, effectively creating a situation where the "true" model depended on which group a person belonged to, a piece of information the main analysis did not see. They then applied their UNIVERSE framework to see if it could correctly identify the importance of features despite this hidden information. The results were clear: without their new adjustments, standard methods often failed to capture the true importance of variables, especially when the sample size was large or when the data was complex. In contrast, the UNIVERSE method consistently produced ranges that contained the true answer. In their tests, the method successfully captured the correct importance of variables in more than eighty percent of the trials, even when the data was missing key pieces of information.

One of the most compelling demonstrations of their work involved a credit risk dataset used by banks. In this scenario, a specific risk score called the External Risk Estimate was widely considered a critical factor for predicting loan defaults. Standard analysis suggested this score was vital. However, when the researchers applied their UNIVERSE framework, they found that the importance of this score was not as robust as it seemed. They showed that if there were unmeasured factors, such as a person's income, that were not included in the data, the External Risk Estimate might not be necessary at all. Their analysis suggested that other, more interpretable features could easily replace it. This finding is significant because it implies that banks might be relying on a complex, expensive metric that could be swapped for simpler, more transparent information, potentially leading to better feedback for loan applicants.

The strength of this new approach lies in its ability to handle uncertainty without making unrealistic assumptions. Traditional methods often require knowing exactly how the missing data is distributed or assuming a specific type of relationship between variables. The UNIVERSE framework does not need this. Instead, it asks the analyst to provide a reasonable estimate of how much the missing data might change the results. Based on this estimate, the method calculates a range of possible importance values. The researchers proved mathematically that these ranges are reliable, meaning that if the analyst's estimate of the missing data's impact is correct, the true importance of the feature will almost certainly fall within the calculated bounds. They also showed that these bounds become tighter and more precise as more data becomes available, making the method useful for both small and large datasets.

This work represents a shift in how scientists think about variable importance. Rather than seeking a single, definitive answer that depends on a specific model and a complete dataset, the researchers embrace the complexity of the real world. They acknowledge that multiple models can be right and that data is often incomplete. By providing a method that accounts for these realities, they offer a tool that is more honest about what the data can and cannot tell us. The framework does not claim to find the one true model or the one true importance score. Instead, it provides a reliable interval that captures the truth, ensuring that decisions based on data are not derailed by hidden variables or the illusion of a single best explanation. This approach offers a path toward more trustworthy and robust scientific conclusions in fields ranging from finance to public policy.

Technical Summary: UNIVERSE – Variable Importance with Unobserved Confounding and the Rashomon Effect

Problem Statement

Variable importance (VI) methods are standard tools for hypothesis generation, feature selection, and scientific validation. However, standard VI pipelines suffer from two critical limitations that can lead to misleading insights:

The Rashomon Effect: Multiple distinct models can explain a dataset equally well. A feature deemed important in one near-optimal model may be irrelevant in another. While recent work has addressed this by analyzing the "Rashomon set" (the set of all near-optimal models), these methods typically assume all relevant variables are observed.
Unobserved Confounding: In observational datasets, essential variables are often omitted. When unobserved confounders exist, the importance of a feature estimated on observed data may not reflect its true importance relative to the underlying conditional mean function.

Existing approaches to unobserved confounding (sensitivity analysis) often rely on restrictive assumptions (e.g., linearity) that do not hold for modern, non-linear VI metrics like SHAP or permutation importance. Conversely, existing Rashomon set methods do not account for omitted variables. This paper addresses the gap of performing robust variable importance inference in the presence of both the Rashomon effect and unobserved confounding.

Methodology: The UNIVERSE Framework

The authors introduce UNIVERSE (UNobservables and Inference for Variable importancE using Rashomon SEts), a framework that derives bounds on the true variable importance even when features are missing. The approach integrates three sources of uncertainty: the Rashomon effect, unobserved variables, and finite sample error.

1. Theoretical Foundation

The framework defines the true conditional mean function $g^*(x, u)$ which depends on both observed covariates $X$ and unobserved covariates $U$ . The goal is to bound the variable importance $\Phi_j(g^*, P_U)$ for this true function.

The method proceeds in three logical steps:

Step A: Bounding the True Submodels via the Rashomon Set.
The authors define a set of conditional submodels $S^* = \{f_u \mid u \in \mathcal{U}\}$ , where each $f_u$ is the optimal model given a specific value of the unobserved variable $U$ . They prove that $S^*$ is a subset of the population Rashomon set $R(\epsilon_{unobs})$ , where $\epsilon_{unobs}$ bounds the expected loss of these submodels on the observed data.
Step B: Finite-Sample Correction.
Since the population Rashomon set is unobservable, the method constructs an empirical Rashomon set $\hat{R}^{(n)}$ . Theorem 1 provides a probabilistic guarantee that an adjusted empirical Rashomon set (with a threshold corrected for finite sample size and regularization bias) contains the population Rashomon set with high probability. This ensures that all necessary submodels in $S^*$ are captured.
Step C: Accounting for Distribution Shift (VI-Drift).
Variable importance metrics often depend on the input distribution. The importance of a feature in a submodel $f_u$ evaluated on the conditional distribution $P_{XY|U=u}$ may differ from its importance on the marginal observed distribution $P_{XY}$ . The authors introduce a parameter $\tau_j$ (VI-drift) to bound the maximum shift in importance caused by conditioning on unobserved variables.

2. The Final Bound

By combining these components, Theorem 3 establishes that the true variable importance $\Phi_j(g^*, P_U)$ lies within an interval with high probability ( $1 - (\delta + \gamma)$ ):
$\Phi_j(g^*, P_U) \in \left[ \inf_{f \in \hat{R}} \Phi_j(f, D^{(n)}) - \tau_j - \alpha, \quad \sup_{f \in \hat{R}} \Phi_j(f, D^{(n)}) + \tau_j + \alpha \right]$
Where:

$\hat{R}$ is the adjusted empirical Rashomon set.
$\alpha$ accounts for finite-sample uncertainty in estimating the VI metric itself.
$\tau_j$ accounts for the distribution shift due to unobserved confounding.
The bounds are derived using subtractive model reliance (or other compatible metrics) and hold for any model class where the Rashomon set can be estimated.

Key Contributions

First Unified Framework: UNIVERSE is the first method to simultaneously address the Rashomon effect, unobserved confounding, and finite-sample uncertainty in variable importance estimation.
Theoretical Guarantees: The paper provides rigorous finite-sample bounds (Theorems 1–3) that guarantee the true variable importance is contained within the calculated interval with high probability, even under unobserved confounding.
Generality: Unlike sensitivity analysis frameworks restricted to linear estimands, UNIVERSE is compatible with a wide class of model-agnostic VI metrics (e.g., SHAP, permutation importance, model reliance).
Practical Implementation: The authors provide a method to estimate the size of the Rashomon set to tighten bounds and demonstrate the framework's utility in a credit risk case study.

Experimental Results

The authors evaluated UNIVERSE using "semi-synthetic" datasets derived from four real-world sources (COMPAS, Dropout, FICO, Wine Quality). In this setup, the "unobserved" variable was the partition of the data used to train a specific submodel, allowing the authors to know the ground truth.

Coverage Validation: Across varying sample sizes and datasets, the framework consistently achieved the target coverage rates (e.g., $\geq 80\%$ or $\geq 90\%$ ) only when all three adjustments (model uncertainty, VI estimation uncertainty, and VI drift) were applied.
Necessity of Adjustments: Experiments showed that omitting finite-sample corrections or VI-drift adjustments led to severe under-coverage, where the true importance fell outside the calculated bounds.
Case Study (FICO Credit Risk): In an analysis of the "External Risk Estimate" (ERE) feature, UNIVERSE demonstrated that ERE's apparent importance was not robust to unobserved confounding. The bounds suggested that other, potentially more interpretable features could replace ERE if they captured a specific level of signal, challenging the conclusion that ERE is indispensable.

Significance and Claims

The paper claims that UNIVERSE represents a substantial step forward in using variable importance in challenging, realistic settings where data is incomplete. By providing mathematically guaranteed bounds, the framework allows practitioners to:

Distinguish between features that are robustly important across all plausible models and unobserved scenarios versus those whose importance is an artifact of omitted variables.
Avoid misleading conclusions drawn from single-model analyses or analyses that ignore the Rashomon effect.
Quantify the "cost" of unobserved confounding in terms of the width of the importance interval.

The authors note that while finding the complete Rashomon set can be computationally intensive, the framework is general and applicable to any model class where the set can be approximated (e.g., decision trees, random forests). They acknowledge that future work is needed to extend these methods to continuous outcomes and more complex model classes like neural networks.