🔬 materials science

Uncertainty quantification design principles for machine learning interatomic potentials: lessons learned from hierarchical Bayesian inference

This paper benchmarks uncertainty quantification strategies for machine learning interatomic potentials using argon coupled-cluster data and proposes a hierarchical Bayesian Gaussian process design that effectively balances accuracy, calibration, and error discrimination while explicitly dissecting uncertainty sources for reliable applications.

Original authors: Brennon L. Shanks, George Simmons, Albert P. Bartók, James. R. Kermode

Published 2026-09-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Brennon L. Shanks, George Simmons, Albert P. Bartók, James. R. Kermode

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of materials science, researchers often try to predict how atoms will behave by simulating them on computers. For decades, these simulations have relied on rules of thumb or simplified physics to describe how atoms push and pull on one another. While useful, these simplified rules often break down when scientists try to model complex new materials or extreme conditions. To get around this, a newer generation of tools has emerged: machine learning interatomic potentials. These are computer programs trained on high-level quantum physics calculations to learn the rules of atomic interaction directly from data. They are incredibly fast and accurate, acting as a bridge between the slow, precise world of quantum mechanics and the fast, large-scale world of engineering. However, a major problem remains: these programs are often too confident. They will make a prediction and give a number, but they cannot reliably tell you when that number is likely to be wrong. In fields like designing new batteries or nuclear materials, knowing when a prediction is uncertain is just as important as the prediction itself. Without a way to measure this doubt, these powerful tools cannot be safely used for critical decisions.

This is the challenge that a team of researchers set out to solve. They focused on a specific type of machine learning model that uses a statistical framework known as a Gaussian process. Think of this framework not as a rigid set of equations, but as a flexible way of learning that naturally keeps track of how much it knows and how much it is guessing. The researchers wanted to see if they could build a system that didn't just predict energy and force between atoms, but also provided a honest, reliable measure of its own uncertainty. To test this, they chose argon, a noble gas with very simple interactions, as a training ground. By using the most accurate quantum physics calculations available as a reference, they could see exactly how well their new method worked.

The team developed a sophisticated approach that combines known physics with statistical learning. Instead of starting from scratch, they fed the computer a basic, well-understood description of how argon atoms interact. The machine learning model then learned only the small corrections needed to make that basic description perfect. Crucially, they did not just pick the single "best" set of settings for the model. Instead, they allowed the computer to explore a wide range of possible settings, understanding that there is no single perfect answer when data is limited. This process, called hierarchical Bayesian inference, lets the model carry forward the uncertainty about its own settings into its final predictions. The result is a system that admits when it is unsure, rather than pretending to know more than it does.

When the researchers compared their new method against other popular machine learning techniques, the difference was stark. Other methods, including advanced neural networks, were very good at getting the right answer on average. However, they were dangerously overconfident. When these models were wrong, they often gave a very small uncertainty estimate, suggesting they were sure of a wrong answer. In contrast, the new hierarchical model was slightly more conservative, often predicting a wider range of possible outcomes. But this caution was a feature, not a bug. The model's uncertainty estimates were honest: when the prediction was likely to be far off, the model signaled a large uncertainty. When the prediction was likely to be accurate, the uncertainty was small. This ability to distinguish between easy and hard cases is vital for active learning, a process where computers decide which new experiments to run next. If a model cannot tell you which of its predictions are shaky, you cannot use it to guide research efficiently.

The study also revealed that the way the model learns matters deeply. Simply having a lot of data is not enough; the model needs to understand the underlying physics. When the researchers removed the basic physical description from the model and asked it to learn everything from scratch, the results were poor, especially when data was scarce. By anchoring the model with a known physical law, the machine learning part only had to learn the small deviations, which made the whole system much more robust. Furthermore, they found that simply picking the single best set of settings for the model was a mistake. While this method gave slightly more accurate point predictions, it destroyed the model's ability to tell you when it was uncertain. The only way to get a reliable measure of doubt was to let the model explore many different possibilities and average them together.

In the end, the researchers demonstrated that it is possible to build machine learning models for atoms that are not only accurate but also trustworthy. By combining physical knowledge with a statistical method that respects uncertainty, they created a tool that knows its own limits. This is a significant step forward for the field. It suggests that in the future, scientists will be able to use these powerful simulations with a clear understanding of where the predictions are solid and where they are shaky. This clarity is essential for moving from simple simulations to the design of real-world materials where failure is not an option. The work shows that the path to reliable artificial intelligence in science is not just about making models smarter, but about teaching them to be humble about what they do not yet know.

Technical Summary: Uncertainty Quantification Design Principles for Machine Learning Interatomic Potentials

Problem Statement
Reliable uncertainty quantification (UQ) remains a critical bottleneck for Machine Learning Interatomic Potentials (MLIPs). While multiscale materials modeling relies on the robust propagation of uncertainties to derive validated properties at larger scales, current MLIP approaches often lack well-defined probabilistic interpretations for downstream UQ. Existing strategies, such as ensemble methods based on random initializations or data subsets, provide empirical measures of disagreement but fail to offer a hierarchical probabilistic model where distinct sources of predictive uncertainty (e.g., model form, hyperparameters) are explicitly represented and propagated. Furthermore, Gaussian Process (GP)-based potentials, which theoretically offer rigorous posterior variance estimates, are underutilized due to tendencies toward overestimation of errors and the computational complexity of optimizing or sampling hyperparameters.

Methodology
The authors benchmark UQ strategies using argon dimers and trimers as a test case, with reference data generated at the coupled-cluster [CCSD(T)] level. The study focuses on a hierarchical Gaussian Process (GP) framework that integrates physics-informed priors with Bayesian propagation of hyperparameter uncertainty.

Key methodological components include:

Physics-Informed Priors: Instead of a zero-mean prior, the model employs a flexible $(\lambda-6)$ Mie potential as the explicit mean function. This encodes known physical behaviors (short-range repulsion and long-range dispersion) into the prior, allowing the GP to model only the residual deviations.
Non-Stationary Kernels: The residual covariance is modeled using an amplitude-modulated Matérn kernel ( $\nu=3/2$ ), which includes a spatially varying standard deviation envelope. This envelope accounts for physical decay at long ranges and increased uncertainty in data-sparse regions.
Hierarchical Bayesian Inference: The framework treats hyperparameters (defining the mean, covariance, and noise) as random variables. A generalised posterior is constructed using either the marginal likelihood or a leave-one-out cross-validation (LOO-CV) objective.
Sampling and Propagation: Hyperparameter uncertainty is sampled using Hamiltonian Monte Carlo with a No-U-Turn Sampler (HMC-NUTS). The final predictive distribution is a mixture of Gaussians obtained by marginalizing over the sampled hyperparameters, explicitly separating uncertainty arising from the conditional GP (at fixed hyperparameters) from uncertainty due to hyperparameter variation.
Benchmarks: The proposed hierarchical GP is compared against:
- MACE foundation model committees (fine-tuned from different pretrained backbones).
- MACE seed ensembles (random initializations).
- Variants of the GP model (zero-mean vs. Mie-prior; MAP estimation vs. Laplace approximation vs. full HMC sampling).

Key Results
The study reveals distinct trade-offs between accuracy, calibration, sharpness, and error discrimination:

Accuracy vs. Calibration:
- MACE Ensembles: While achieving competitive Root Mean Square Errors (RMSE) and Continuous Ranked Probability Scores (CRPS), MACE ensembles exhibit severe overconfidence. Their empirical coverage (the percentage of true values falling within the predicted confidence interval) is low (e.g., 3.5% for seed ensembles with energy+force training), indicating they fail to capture true uncertainty.
- Hierarchical GP: The proposed GP model is the most accurate (lowest RMSE and CRPS on energy+force sets) but is "conservative," achieving 100% empirical coverage. It tends to overestimate errors, particularly in low-error regimes, but avoids the dangerous underestimation seen in neural network ensembles.
Error Discrimination:
- The hierarchical GP maintains a strong positive correlation between predicted uncertainty and realized error (Spearman $\rho \approx 0.62-0.65$ ).
- In contrast, MACE ensembles show weak or even negative correlations, meaning their ensemble spread is a poor signal for identifying configurations with large errors—a critical failure for active learning applications.
Variance Decomposition:
- The total predictive variance in the hierarchical GP is decomposed into conditional GP variance and hyperparameter variance.
- In the high-error regime, the dominant source of uncertainty is the variation among plausible hyperparameter settings (epistemic uncertainty), not the local distance to training data. This explains why the model remains conservative even when test points are far from training data; it reflects the uncertainty in the model form itself.
Design Principles:
- Physics-Informed Priors: In low-data regimes (e.g., 20 configurations), a physics-informed mean (Mie potential) is essential. It reduces RMSE by an order of magnitude compared to zero-mean GPs by allowing the model to learn residuals rather than the full interaction.
- Hyperparameter Propagation: Simply optimizing hyperparameters (Maximum A Posteriori, MAP) improves point prediction accuracy but destroys error discrimination (correlation drops to negative values). Full propagation of hyperparameter uncertainty (via HMC) is necessary to retain informative uncertainty estimates.
- Laplace Approximation: A Laplace approximation of the hyperposterior offers a promising middle ground, significantly improving error correlation compared to MAP while avoiding the computational cost of full HMC sampling, though it may struggle with non-Gaussian posteriors.

Significance and Claims
The paper claims to provide practical design principles for MLIP UQ in uncertainty-critical applications. The primary contribution is demonstrating that a hierarchical Bayesian framework with physics-informed priors and full hyperparameter propagation produces uncertainty estimates that are:

Conservative rather than overconfident, ensuring safety in predictive modeling.
Informative of realized errors, allowing the model to effectively rank configurations by their likelihood of failure.
Explicitly dissectable, enabling researchers to distinguish between uncertainty due to data sparsity and uncertainty due to model form.

The authors emphasize that while ensemble methods (like MACE committees) can yield accurate point predictions, they often lack the rigorous probabilistic structure required for reliable uncertainty propagation. The work suggests that for applications where the cost of error is high, the "conservative" nature of the hierarchical GP is a feature, not a bug, provided it is paired with physics-informed priors to ensure accuracy in data-sparse regimes.

Technical Summary: Uncertainty Quantification Design Principles for Machine Learning Interatomic Potentials

More like this