← Latest papers
🔬 materials science

Jackknife-conformal uncertainty for linear ACE potentials: calibrated forces and silicon properties compared with Bayesian and POPS baselines

This paper introduces a jackknife-conformal uncertainty quantification method for linear atomic cluster expansion potentials that provides calibrated force uncertainties and accurate property predictions for silicon, outperforming Bayesian and POPS baselines while identifying and addressing specific limitations regarding shared training biases in relaxed point defects.

Original author: James R. Kermode

Published 2026-11-09
📖 6 min read🧠 Deep dive

Original author: James R. Kermode

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

In the microscopic world of materials science, scientists rely on computer models to predict how atoms behave. These models act as virtual laboratories, simulating the forces that hold atoms together and the energy they possess. For decades, researchers have used these simulations to design new materials, from stronger metals to more efficient batteries. However, these computer models are never perfect; they are simplified versions of reality. Because they are simplified, they inevitably make mistakes. The critical challenge for scientists is not just knowing what the model predicts, but knowing how much to trust that prediction. If a model says a material will be strong, engineers need to know if there is a hidden chance it might fail. Without a reliable way to measure this uncertainty, the predictions are like a map with no scale bar: you can see the terrain, but you cannot tell how far you are from the edge of the known world.

A researcher at the University of Warwick has developed a new method to draw that missing scale bar for a specific type of computer model used to study silicon, the material at the heart of modern electronics. Their work focuses on a technique called the "jackknife," a statistical tool that tests a model's stability by seeing how much its answer changes when a small piece of the training data is removed. By combining this with a method called conformal prediction, which guarantees a certain level of accuracy, they created a system that can tell scientists exactly how confident they should be in a prediction. They tested this system on twenty different properties of silicon, ranging from how stiff the material is to how much energy it takes to create a tiny defect inside it. The result is a set of uncertainty estimates that are far more reliable than previous methods, providing a clear, calibrated range of possibilities for every prediction the model makes.

The researcher focused on a specific family of computer models known as linear atomic cluster expansion potentials. These models are designed to learn the rules of atomic interaction from a large database of highly accurate calculations. The problem is that even the best models are "misspecified," meaning they are built on assumptions that do not perfectly match the complex reality of atoms. Because of this mismatch, traditional methods for estimating error often fail. They tend to be overly confident, suggesting a prediction is precise when it is actually quite uncertain. The researcher realized that to fix this, they needed a way to measure the error that accounts for the model's inherent flaws, rather than just the noise in the data.

To solve this, they used a clever two-step approach. First, they looked at how the model's internal settings shifted when they removed one entire configuration of atoms from the training data. This "delete-one" test revealed the shape of the uncertainty: it showed which parts of the model were fragile and which were robust. This shape was not a simple circle of error but a complex, multi-dimensional cloud that stretched and squeezed depending on the local environment of the atoms. Second, they calibrated the size of this cloud. They took a separate set of data that the model had never seen before and adjusted the cloud until it covered the actual errors at a specific, guaranteed rate. This ensured that when the model made a prediction, the associated uncertainty range was neither too tight nor too loose, but just right to capture the truth most of the time.

The researcher applied this calibrated system to twenty different properties of silicon. These properties included the material's elastic constants, which describe how it bends and stretches, and the energy required to form various types of defects, such as missing atoms or extra atoms squeezed into the crystal structure. They compared their new method against two existing approaches: one based on standard Bayesian statistics and another that uses a set of optimal parameter sets. The results were clear. The new method provided uncertainty ranges that successfully captured the true values for the vast majority of the properties tested. For the more accurate of the two models they studied, the uncertainty intervals were about four times tighter than those of the less accurate model, yet they still maintained the correct coverage. This means the method successfully shrinks the margin of error as the model improves, a crucial feature for practical applications.

However, the researcher also discovered where their method hits a wall. They found that for certain types of defects, specifically those involving missing or extra atoms, the model consistently made the same kind of mistake. This happened because every training example for these defects in their database came from a "hot" snapshot of atoms in motion, while the real-world properties they wanted to predict were for "cold," relaxed structures. Because the error was shared by every single training example, removing one at a time did not reveal the problem. The model was blind to a bias that affected all its data equally. To fix this, the researcher devised a new step: they measured the size of this hidden bias directly from the training data and added it as a correction term. Once this adjustment was made, the uncertainty ranges for these difficult defect properties expanded just enough to include the true values, bringing the success rate back to nearly perfect.

The study also tested how well the method worked on materials and structures that were completely absent from the training data, such as cracks and twisted atomic lines known as dislocations. For the larger, more complex model, the method remained robust, accurately covering the errors even in these extreme environments. For the smaller, cheaper model, it struggled slightly with the twisted dislocations, showing that while the method is powerful, the quality of the underlying model still matters. The researcher also showed that their approach was numerically stable and consistent, unlike some alternative methods that produced wildly different results depending on how the calculations were set up.

Ultimately, this work provides a practical toolkit for scientists who use computer models to predict material behavior. It moves the field away from guessing how much to trust a simulation and toward knowing. By providing calibrated, reliable uncertainty estimates, the method allows researchers to distinguish between a prediction that is merely a guess and one that is a solid, quantifiable insight. The approach is implemented in open-source software, making it available for others to use in their own studies of silicon and potentially other materials. The key takeaway is that while no model can ever be perfect, we can now measure its imperfections with a precision that was previously out of reach, turning vague uncertainty into a clear, usable guide for discovery.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →