Are Scaling Laws of Universal MLIPs Universal?
This study demonstrates that while increasing dataset size improves machine-learned interatomic potential (MLIP) performance, the scaling behavior and sample efficiency are not universal but critically depend on the interplay between model architecture, target properties, data sampling strategies, and the chemical distribution of the test set.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine trying to predict how a new material will behave by watching how its atoms move. To do this accurately, scientists usually rely on a powerful but incredibly slow computer method called density functional theory. It solves the complex quantum mechanics of every electron in a structure, but the time it takes grows so fast that simulating even a modest amount of material can take days or weeks. To speed this up, researchers have developed "machine-learned interatomic potentials." Think of these as smart shortcuts: a computer program trained on thousands of expensive quantum calculations that learns to guess the energy and forces between atoms almost instantly. These shortcuts allow scientists to simulate materials for hours or days, opening the door to discovering new batteries, superconductors, and stronger alloys.
However, training these shortcuts is expensive. Every piece of data used to teach the model requires a slow, costly quantum calculation. This raises a critical question for the field: if you have a limited budget to generate training data, how should you spend it? Should you simply grab a random mix of structures, or is there a smarter way to pick them? For years, the assumption has been that more data is always better, and that picking diverse examples—perhaps by ensuring a wide variety of chemical elements or crystal shapes—would make the model learn faster and more efficiently. This idea, known as a "scaling law," suggests that performance improves predictably as you add more training examples, much like how a student might improve their test scores by studying more practice problems.
A team of researchers set out to test whether this scaling law is truly universal for these atomic models. They took three different types of machine-learned potentials and trained them on a massive pool of over one million atomic structures. Instead of just using the whole pool, they created smaller training sets ranging from 10,000 to 640,000 structures. Crucially, they did not just pick these structures randomly. They tried several different strategies: some picked structures to ensure a balanced mix of chemical elements, others tried to balance different crystal formulas, and some used complex mathematical maps of the data to pick the most "diverse" or "far apart" examples. They then tested how well these models performed on two different challenges: one that looked like the data they were trained on, and another that represented a slightly different, shifted type of material.
The results challenged the common belief that more diversity automatically leads to better models. The researchers found that the idea of a single, universal rule for how these models improve with more data does not hold up. The speed at which the models got better depended heavily on which specific model architecture was used, which physical property was being predicted, and, most surprisingly, which test set was used to measure success. In many cases, the simple strategy of picking structures at random actually performed just as well as, or better than, the sophisticated methods designed to maximize diversity.
Perhaps the most striking finding was that the "best" way to pick data changed depending on where the model was being tested. A strategy that improved the model's performance on one type of material could actually make it worse on another. For instance, a method that carefully balanced the chemical elements in the training set helped the model predict the properties of the shifted material, but it slightly hurt its performance on the original material. Conversely, methods that tried to pick the most mathematically diverse structures often performed worse than random selection. The study showed that the key to a good training set is not just having a large number of examples, but having examples that match the specific chemical world where the model will eventually be used.
The researchers also discovered that not all predictions get easier at the same speed. As the models saw more data, their ability to predict the total energy of a structure improved much faster than their ability to predict the forces pushing on individual atoms or the stress within the material. This suggests that if a scientist only looks at how well the model predicts energy, they might get an overly optimistic view of how quickly the model is learning to handle the more difficult tasks of predicting forces and stresses.
Ultimately, the study concludes that there is no single "magic bullet" for selecting training data. The value of adding more expensive calculations depends entirely on the context. If a model is intended for a specific application, the training data must align with the chemical and structural environment of that application. Simply throwing more data at the problem, or trying to force maximum diversity without regard for the target use case, may not yield the expected improvements. The path to better materials discovery lies not just in gathering more data, but in gathering the right data for the job at hand.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.