📊 statistics

Interpretable AI with Local Distillation

This paper introduces "local distillation," a framework that achieves high accuracy and interpretability by training regularized linear "student" models guided by black-box "teacher" models at each query point, utilizing a novel randomization technique to ensure stable feature selection and reveal heterogeneous patient subgroups that global or black-box models miss.

Original authors: Erin Craig, Yiling Huang, Snigdha Panigrahi

Published 2026-08-25

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Erin Craig, Yiling Huang, Snigdha Panigrahi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of artificial intelligence, powerful computer models can make predictions with startling accuracy, often outperforming traditional statistical methods in fields ranging from finance to medicine. These "black box" systems, such as deep learning networks and complex ensembles, process vast amounts of data to find patterns that humans might miss. However, their greatest strength is also their greatest weakness: they offer little insight into why they made a specific prediction. For a doctor deciding on a treatment or a judge weighing evidence, knowing the outcome is not enough; they need to understand the reasoning behind it. This creates a tension between accuracy and transparency. Scientists have long sought a way to have both, hoping to find a method that is as precise as the most advanced AI but as clear and logical as a simple equation.

A team of researchers at the University of Michigan has proposed a solution called "local distillation," a technique that bridges this gap by teaching a simple, transparent model to mimic a complex, powerful one, but only for specific situations. The core idea relies on a fundamental mathematical truth: while a complex curve might twist and turn wildly across a large landscape, it looks almost perfectly straight if you zoom in close enough to a single point. By focusing on a small, relevant neighborhood of data around a specific question, a simple linear model can capture the behavior of a sophisticated black box without losing its clarity. The researchers demonstrated that this approach not only matches the accuracy of the complex model but also reveals hidden patterns in the data that neither the simple nor the complex model could see on their own.

The researchers built their method around a "teacher-student" dynamic. The teacher is a high-performing, opaque AI model that has already learned to predict outcomes well. The student is a simple, transparent linear model that the researchers want to use for decision-making. Instead of trying to force the student to learn the entire complex behavior of the teacher at once, the researchers let the teacher guide the student only when a specific question is asked. When a new data point arrives—say, a specific car's fuel economy or a patient's gene expression—the teacher first identifies which past examples are most similar to this new case. It does this not by looking at the raw features, like engine size or gene count, but by looking at what the teacher itself predicted for those past cases. This creates a "local neighborhood" of similar outcomes.

Once this neighborhood is defined, the teacher provides two crucial pieces of help to the student. First, it tells the student which past examples to pay attention to, giving more weight to those with similar predicted outcomes. Second, it offers a specific prediction for the current case, acting as an anchor. The student then fits a simple line through this weighted, localized data, pulling its own prediction toward the teacher's anchor. This process ensures the student stays close to the teacher's high accuracy while remaining a simple, understandable equation. If the teacher is not actually better than a standard global model, the system simply defaults to the standard model, ensuring it never performs worse.

To make these local models trustworthy, the researchers added a layer of stability testing. In statistics, a model is only as good as its reliability; if a tiny change in the data causes the model to pick completely different factors as important, the conclusion is shaky. To test this, the researchers ran the local model hundreds of times, each time adding a tiny, random amount of noise to the calculation. They then watched to see which factors the model consistently chose to include. If a factor, such as "engine displacement" for a car, appeared in the model 95% of the time despite the noise, it was deemed a stable, reliable predictor. If it appeared only 50% of the time, it was flagged as uncertain. This process allowed them to distinguish between real signals and random fluctuations, providing a clear measure of confidence for every prediction.

The researchers tested this approach on 17 different real-world datasets, including data on car fuel economy, student performance, and wine quality. In every case, the local distillation method nearly matched the predictive accuracy of the powerful teacher models, such as TabPFN and XGBoost, while producing a sparse, easy-to-read linear equation for each individual case. In a test with car data, the method improved prediction accuracy by 48% compared to a standard global linear model, while staying almost as accurate as the complex teacher. More importantly, the local models revealed that the rules for predicting fuel economy change depending on the car. For inefficient cars, the number of cylinders was the key factor, but for efficient cars, engine displacement mattered more. A single global model would have averaged these differences out, missing the nuance entirely.

The power of this method became even more apparent in a high-dimensional example involving breast cancer gene expression. The researchers analyzed data from 536 patients, looking at over 17,000 genes to predict the expression of a specific gene linked to cancer risk. A standard global model selected 123 genes as important, but the local distillation method found that different groups of patients relied on different sets of genes. By clustering the patients based on the stability of their local models, the researchers identified distinct subgroups. In one group, a specific gene was a strong negative predictor, while in another group, a completely different gene played that role. These distinct biological patterns were invisible to the global model and difficult to extract from the black-box teacher. The local distillation approach successfully surfaced these hidden subgroups, showing that the relationship between genes and disease is not uniform across all patients.

The study also established theoretical guarantees that this method is stable. The researchers proved mathematically that the selection of important factors remains consistent even when the training data is slightly perturbed, provided the randomization is tuned correctly. This means that the conclusions drawn from the model are not just artifacts of a specific dataset but reflect genuine relationships in the data. The method works by balancing the influence of the complex teacher with the simplicity of the linear student, using a data-driven rule to decide how much to trust the teacher. If the teacher is not an improvement, the system reverts to a standard fit, ensuring safety and reliability.

Ultimately, this work offers a path forward for using artificial intelligence in high-stakes decisions where reasoning is as important as the result. It suggests that we do not need to choose between the raw power of modern AI and the clarity of classical statistics. By letting a complex model guide a simple one locally, and by rigorously testing the stability of the results, researchers can build systems that are both accurate and interpretable. The findings show that the "local" view of data often holds the key to understanding complex systems, revealing heterogeneity and structure that global averages obscure. This approach provides a framework for making AI not just a tool for prediction, but a partner in reasoning, capable of explaining its own logic in terms that humans can understand and trust.

Technical Summary: Interpretable AI with Local Distillation

Problem Statement

Modern artificial intelligence models, including tabular foundation models and gradient-boosted ensembles, frequently outperform classical statistical methods in predictive accuracy, even in small-sample tabular settings. However, these "black-box" models lack inherent interpretability, offering little basis for decision-makers to reason about predictions. While post-hoc explanation tools (e.g., LIME, SHAP) exist, they are constructed separately from the model, often lack fidelity to the underlying predictor, and can be unstable due to external choices in perturbation schemes or reference distributions.

The core challenge is to develop a predictive framework that is both highly accurate and interpretable by design. The authors posit that while global regression functions may be complex and nonlinear, they are locally well-approximated by linear functions. The difficulty lies in defining what constitutes "local" in a supervised, tuning-free manner and developing statistical tools to ensure the resulting local interpretations are stable and trustworthy.

Methodology: Local Distillation

The authors propose Local Distillation, a method where a high-performing black-box "teacher" model guides the construction of a sparse, regularized linear "student" model at each specific query point. The method operates in three primary steps:

Defining Locality via Supervised Weights:
Unlike classical local regression (e.g., LOESS) which uses unsupervised input-space similarity, local distillation defines locality based on the teacher's predicted outcomes. For a query point $x^*$ , training observations $j$ are weighted by the similarity of the teacher's prediction $\hat{\phi}(x_j)$ to the teacher's prediction at the query point $\hat{\phi}(x^*)$ . This collapses the $p$ -dimensional feature space into a single axis of predicted risk or outcome, identifying a local neighborhood of similar patients or data points.
Anchoring the Fit:
The student model is anchored to the teacher's prediction at the query point. This is achieved by treating the teacher's prediction as a "pseudo-observation" added to the training data. The weight of this pseudo-observation is estimated from the data as the cross-validated ratio of the student's loss to the teacher's loss ( $\hat{\mu}$ ).
- If the teacher outperforms the student globally ( $\hat{\mu} > 1$ ), the teacher's influence increases.
- If the local neighborhood is sparse (small effective sample size $\hat{n}_{eff}$ ), the teacher's influence is strengthened to stabilize the fit.
- If the teacher offers no improvement ( $\hat{\mu} \le 1$ ), the method reverts to a global linear fit.
Optimization:
For each query point, a regularized linear model (e.g., Lasso or Elastic Net) is fit by minimizing a weighted loss function that includes:
- The weighted squared error of the training data (weighted by similarity).
- A penalty term anchoring the prediction to the teacher's output.
- A regularization penalty (e.g., L1/L2) to ensure sparsity and interpretability.

Interpretability through Randomized Stability

To address the instability inherent in feature selection (e.g., Lasso coefficients changing with minor data perturbations), the authors introduce Randomized Local Distillation.

Mechanism: A small amount of Gaussian randomization is added directly to the optimization objective of the local fit. This perturbs the loss function without altering the locality definition (weights and teacher predictions remain fixed).
Selection Frequencies: By refitting the model multiple times with different randomization draws, the method calculates the empirical selection frequency for each feature. Features selected in a high proportion of refits are deemed stable and reliable; those with low frequencies are flagged as uncertain.
Aggregate Interpretation: The randomized local coefficients are used to cluster observations into stable subgroups. This reveals heterogeneity in feature-outcome relationships across the dataset that a global linear model would average out and a black-box model might obscure.

Theoretical Contributions

The paper provides theoretical guarantees for the stability of feature-selection probabilities under the Lasso penalty.

Smoothness: The authors prove that adding Gaussian randomization smooths the feature-selection probability function.
Sensitivity Bounds: They establish a bound on the sensitivity of these probabilities to perturbations in the training responses. The bound depends on the effective sample size ( $\hat{n}_{eff}$ ) and the smoothness of the teacher's prediction function.
Stability Guarantee: Under specific assumptions (bounded design, general position), the feature-selection probabilities are shown to be stable under small perturbations of the training data, providing a theoretical basis for trusting the selection frequencies.

Results

The method was evaluated across 17 benchmark regression datasets and a high-dimensional gene expression case study.

Predictive Performance: Across the 17 datasets, local distillation nearly matched the predictive accuracy (measured by $R^2$ and PSE) of its black-box teacher (TabPFN or XGBoost), significantly outperforming global linear models (Lasso/Ridge). In the Auto MPG example, local distillation reduced prediction squared error by 48% compared to a global Lasso while matching the TabPFN teacher.
Interpretability and Sparsity: The method produced sparse linear models at each test point. In the Auto MPG example, local models revealed that "cylinders" predicted fuel economy for inefficient cars, while "displacement" was more predictive for efficient cars—a heterogeneity invisible to the global Lasso (which assigned zero coefficients to both).
High-Dimensional Example: In a breast cancer gene expression study ( $p=17,322$ ), local distillation matched the teacher's accuracy ( $R^2=0.670$ ) while identifying patient subgroups with distinct gene drivers. A global Lasso selected 123 genes, whereas the stable local models (filtered by selection frequency) retained a median of only 15 genes, revealing specific tumor suppressors (e.g., FAM107A, KLF14) active only in specific subgroups.

Significance and Claims

The authors claim that local distillation occupies a "sweet spot" between interpretable modeling and modern AI.

Built-in Interpretability: Unlike post-hoc tools, the model is interpretable by construction, providing a transparent linear explanation for every prediction.
Trustworthy Conclusions: By leveraging randomized stability, the framework distinguishes between reliable feature contributions and artifacts of a single fit, addressing the "trust" gap in AI.
Handling Heterogeneity: The method explicitly models and reveals heterogeneity in feature-outcome relationships, identifying stable subgroups that global models miss and black-box models obscure.
Robustness: The method is robust to the choice of teacher; if the teacher does not improve upon the student, the framework automatically defaults to a global linear fit, ensuring no degradation in performance.

The paper concludes that this framework offers a path toward predictive, interpretable, and trustworthy statistical modeling, particularly in high-stakes domains like personalized medicine.