📊 statistics

Improving Discrepancy Measures for Global Sensitivity Analysis

This paper introduces a theoretically grounded, adjusted ersatz discrepancy measure that significantly improves agreement with Sobol' indices and outperforms existing zero-extra-cost estimators in identifying global sensitivity, particularly for non-smooth models where traditional methods fail.

Original authors: Samuele Lo Piano, Alessio Lachi, Razi Sheikholeslami, Arnald Puy, Pamphile Tupui Roy, Andrea Saltelli

Published 2026-07-31

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Samuele Lo Piano, Alessio Lachi, Razi Sheikholeslami, Arnald Puy, Pamphile Tupui Roy, Andrea Saltelli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery inside a giant, black-box machine. You can't see the gears or wires, but you can poke it with different dials (inputs) and watch what happens on the screen (output). Your goal is to figure out which dials actually matter and which ones are just noise. This is the world of Global Sensitivity Analysis. It's a crucial tool for scientists and engineers because real-world models—whether they predict the weather, design a bridge, or simulate a pandemic—are full of unknowns. If you don't know which variables are driving the results, you might waste time fixing the wrong things or make dangerous decisions based on shaky assumptions.

To solve this, scientists often use a method called Sobol' indices. Think of this as a high-precision, but very expensive, microscope. It tells you exactly how much each dial contributes to the final result, including how dials work together in complex teams. However, running this microscope takes a massive amount of computer power and time. Recently, a cheaper, faster alternative was proposed: a "discrepancy measure." Imagine this as a quick, low-res sketch of the data. Instead of calculating complex math, it just looks at a scatterplot (a cloud of dots) and counts how empty the spaces are. If the dots are spread out evenly, the dial doesn't matter. If they clump together or leave big gaps, the dial is important. The problem was that this quick sketch was a bit messy and didn't always match the high-precision microscope.

This paper introduces a clever fix to make that quick sketch much sharper. The authors, a team of researchers from universities and institutions across Europe and the Middle East, propose two simple tweaks to the "quick sketch" method. First, they rearrange the data so the dots are spread out perfectly evenly before they start counting, removing any visual clutter caused by the shape of the data itself. Second, they use a smart "fill-in" rule (like a game of cellular automata) to guess what should be in the empty spots on the grid, ensuring that hidden patterns aren't missed.

The result is a new tool called the adjusted ersatz discrepancy. The authors tested this new method against the expensive "microscope" and other fast methods using seven different mathematical puzzles and one real-world river model. They found that while the expensive microscope is still the best for smooth, predictable problems, the new adjusted sketch is the only method that got the ranking of importance perfectly right for the tricky, real-world river model. It successfully identified which factors mattered without needing the heavy computational cost of the traditional method. The paper also proves mathematically why this works, showing that it's a reliable way to spot important factors, even if it's not perfect at measuring exactly how important they are. Ultimately, this gives scientists a powerful, free-to-use tool to quickly separate the signal from the noise, even when the data is messy or the model is weird.

Technical Summary: Improving Discrepancy Measures for Global Sensitivity Analysis

Problem Statement
Global Sensitivity Analysis (GSA) aims to quantify how uncertainty in model inputs influences model outputs. Variance-based methods, specifically those relying on Sobol' total-order indices ( $T_i$ ), are the gold standard for identifying influential factors and interactions. However, computing $T_i$ via Monte Carlo integration is computationally expensive. While emulators like Polynomial Chaos Expansion (PCE) offer a cheaper alternative, they rely on smoothness assumptions that often fail for non-smooth, discrete, or highly interactive real-world models.

Recently, Puy et al. (2024) proposed an "ersatz" discrepancy measure ( $S_{\text{ersatz}}$ ) as a low-cost screening tool. This method quantifies deviations from uniformity in input–output scatterplots. While effective at ranking inputs, the original $S_{\text{ersatz}}$ lacked theoretical grounding and showed imperfect agreement with $T_i$ , particularly regarding magnitude estimation and specific dependency structures.

Methodology
The authors introduce an Adjusted Ersatz Discrepancy ( $S^{\text{adjusted}}_{\text{ersatz}}$ ) that refines the algorithm of Puy et al. (2024) through two primary modifications:

Rank Transformation (Uniformization): Before gridding the input–output plane $[x_i, Y]$ , the output $Y$ is transformed into a uniform variable $Y'$ via rank transformation ( $Y' = \text{rank}(Y)/N_s$ ). This converts the scatterplot into a sample from the empirical copula of $(x_i, Y)$ , removing the confounding effect of the output's marginal distribution.
Deterministic Imputation: The grid is analyzed for empty cells. To distinguish between genuine absence of points and sampling artifacts, a Moore-neighbourhood rule is applied. If at least 50% of the 8 neighboring cells of an empty cell are occupied, the empty cell is imputed (assigned a value of 1). This step is designed to capture localized clustering induced by interaction effects that might otherwise be missed.

Theoretical Contributions
The paper provides a copula-theoretic justification for these adjustments, establishing four key theoretical properties:

Zero Condition: If $T_i = 0$ (statistical independence), $S^{\text{adjusted}}_{\text{ersatz}}$ converges to 0 as sample size increases.
Monotone-Direction Property: Under concordance-ordered families of copulas, higher $T_i$ implies higher expected $S^{\text{adjusted}}_{\text{ersatz}}$ .
Full-Support Ceiling: For dependencies with full support (e.g., Gaussian copulas), the measure converges to 0 regardless of $T_i$ magnitude as $N_s \to \infty$ . This proves the method is a consistent screening statistic (identifying if an input matters) but not a consistent magnitude estimator (identifying how much it matters) for non-deterministic dependencies.
Interaction Detection: The imputation step is shown to extend sensitivity detection from first-order effects ( $S_i$ ) to total-order effects ( $T_i$ ) by detecting interaction-induced clustering.

The authors also identify a structural limitation: purely interaction-mediated dependencies (e.g., $Y = (x_i + x_k) \mod 1$ ) where the bivariate copula is independent will result in a zero score, a failure mode inherent to bivariate screening.

Experimental Design and Results
The adjusted method was benchmarked against three zero-extra-cost comparators (PCE-derived $T_i$ , PCE-derived Shapley effects, and a PAWN-type maximum Kolmogorov–Smirnov index) across:

Seven analytical benchmark functions (varying in smoothness, dimensionality, and interaction structure).
A Becker meta-function stress test (512 random configurations).
The HYMOD real-world hydrological model (non-smooth, bounded, left-skewed output).

Key Findings:

Smooth Functions: On smooth, well-specified functions (e.g., Bratley 1988, Oakley–O'Hagan), PCE and Shapley effects outperform the adjusted ersatz in magnitude accuracy, achieving near-perfect rank correlation.
Non-Smooth/Discrete Functions: On functions violating PCE assumptions (Ishigami, Multiverse, HYMOD), PCE and Shapley suffer from systematic Type II errors (missing important inputs) or catastrophic ranking failures. The adjusted ersatz and PAWN(max-KS) remain robust.
HYMOD Case Study: The adjusted ersatz is the only estimator achieving perfect rank agreement ( $\rho = 1.000$ ) with reference $T_i$ on the HYMOD model, maintaining this performance across sample sizes where PCE plateaus due to basis misspecification.
Parameter Sensitivity: A joint sensitivity analysis of the screening method's own parameters revealed that grid resolution is the dominant driver of performance variability. The imputation threshold and sampling method (quasi-random vs. pseudo-random) have negligible influence.

Significance and Practical Implications
The paper positions the adjusted ersatz not as a universal replacement for $T_i$ , but as a robust, theoretically grounded screening tool.

Cost-Efficiency: It achieves screening performance comparable to expensive Monte Carlo $T_i$ calculations at a fraction of the computational cost and without the need for model evaluations beyond the initial sample.
Robustness: It offers a balanced trade-off between Type I and Type II errors across diverse model structures, particularly excelling where polynomial-based methods fail.
Interpretability: The method retains the intuitive appeal of scatterplot analysis, making it accessible to non-specialist stakeholders while resting on rigorous statistical foundations (copula theory).

The authors conclude that practitioners should use PCE/Shapley for smooth, continuous models but rely on the adjusted ersatz for early-stage development, models with unknown functional forms, or those involving discrete or non-smooth inputs.

More like this