Scalable Data-Driven Basis Selection for Linear Machine Learning Interatomic Potentials

This paper introduces active set algorithms for automated, data-driven basis selection within the Atomic Cluster Expansion framework, demonstrating that the resulting sparse linear machine learning interatomic potentials offer superior computational efficiency, generalization accuracy, and interpretability compared to dense models.

The Gist

Imagine you are trying to teach a computer to predict how atoms behave in a new material, like a super-strong metal or a new type of battery. To do this, the computer needs a "rulebook" (a mathematical model) that describes how every atom pushes and pulls on its neighbors.

This paper is about building the perfect, lean, and efficient rulebook without wasting time or energy.

Here is the story of how they did it, broken down into simple concepts:

1. The Problem: The "Kitchen Sink" Approach

Traditionally, scientists tried to build these rulebooks by throwing everything at the wall to see what sticks. They would include thousands of potential rules (mathematical terms) describing every possible way atoms could interact.

• The Analogy: Imagine trying to write a recipe for a perfect cake. Instead of just listing flour, sugar, and eggs, you write down every ingredient in the universe: "Add 1 gram of flour, 0.5 grams of salt, 2 grams of glitter, 3 grams of sand..."
• The Result: The recipe becomes huge, slow to read, and confusing. The computer gets "distracted" by the useless ingredients (like sand and glitter) and might even start guessing wrong because it's trying to fit the noise. This is called overfitting.

2. The Solution: The "Smart Detective" (Sparse Selection)

The authors propose a new way to build these models. Instead of guessing which rules are important, they use a "detective" algorithm to hunt down the exact few rules that matter and ignore the rest.

• The Analogy: Instead of listing every ingredient in the universe, you hire a smart detective. The detective tastes the batter, looks at the chemistry, and says, "Okay, we only need flour, sugar, eggs, and vanilla. Forget the glitter and the sand."
• The Result: You get a tiny, super-fast recipe that tastes just as good (or better) than the giant one, but it's much easier to understand and faster to cook.

3. The Tools: Two Specialized Algorithms

The paper tests two specific "detectives" (algorithms) to do this selection:

• OMP (Orthogonal Matching Pursuit): Think of this as a greedy hiker. The hiker looks around, picks the single best path forward, takes a step, looks again, and picks the next best path. It's fast and gets the job done quickly.
• ASP (Active Set Pursuit): Think of this as a careful cartographer. This detective draws a continuous map of all possible paths, slowly refining the route to find the absolute smoothest, most efficient trail. It takes a bit more time but is very precise.

4. The Results: Why It Matters

The authors tested these methods on three different "worlds" (datasets):

1. Simple Metals (like Copper and Nickel): The detective found that the "standard" huge models were bloated. The new sparse models were smaller, faster, and actually more accurate at predicting how the metal would behave.
2. Silicon (The Computer Chip Material): They built a model for silicon that was nearly as good as the most famous, complex models out there, but it used less than half the number of rules. It was like building a Ferrari engine with half the parts.
3. Water (Liquid): This is tricky because water molecules are constantly moving and bonding in complex ways. The detective successfully identified that the most important interactions were between Hydrogen and Oxygen (the "O-H" bonds), ignoring the less important ones. It figured out the physics of water just by looking at the data, without needing a human to tell it what to look for.

5. The Big Win: No More "Tuning"

Usually, to make these models work, scientists have to spend weeks manually tweaking knobs and dials (hyperparameters) to see what works best. It's like trying to tune a radio by turning the dial blindly.

• The Paper's Magic: Their new method automates this. The "detective" adjusts the knobs itself as it builds the model. It finds the perfect balance between "simple" and "accurate" automatically.

Summary

This paper introduces a way to build atomic rulebooks that are:

• Smaller: They use fewer mathematical rules.
• Faster: Computers can run simulations much quicker.
• Smarter: They generalize better to new situations because they aren't confused by useless data.
• Automatic: They don't require humans to spend months tweaking settings.

In short, they taught the computer to ignore the noise and focus on the signal, creating a more efficient and reliable way to simulate the building blocks of our universe.

Technical Summary

1. Problem Statement

Machine Learning Interatomic Potentials (MLIPs) bridge the gap between the high accuracy of ab initio methods (like Density Functional Theory, DFT) and the computational efficiency of empirical models. While the Atomic Cluster Expansion (ACE) framework provides a systematic way to construct linear MLIPs, it faces significant challenges:

• Feature Selection Complexity: Traditional approaches rely on a priori feature selection or manual hyperparameter tuning (e.g., cutoff radius, correlation order, polynomial degree). This often leads to models that are either too complex (high computational cost, poor generalization) or too simple (low accuracy).
• The "Dense" Limitation: Standard least-squares solvers produce "dense" models where many basis functions have non-zero coefficients, even if they contribute negligibly to the physics. This results in unnecessary computational overhead and reduced interpretability.
• Need for Automation: There is a lack of scalable, automated methods to identify the minimal, most informative subset of basis functions without extensive manual intervention or cross-validation.

2. Methodology

The authors propose a framework that integrates active set algorithms for automated, data-driven sparse feature selection directly into the ACE framework.

A. The ACE Framework

The potential energy is decomposed into site energies expressed as a linear combination of invariant basis functions ($B$):
$$\epsilon_i = \sum_{B \in \mathcal{B}} c_B B(\{x_{ij}\})$$
The goal is to determine the coefficients $c$ such that the model fits DFT data (energies, forces, virials) while selecting a sparse subset of the candidate basis set $\mathcal{B}$.

B. Sparse Optimization Solvers

Instead of standard dense regression, the paper employs two specific sparse recovery methods to solve the regularized least-squares problem:

1. Active Set Pursuit (ASP): A homotopy solver for the Basis Pursuit Denoising (BPDN) problem. It traces the entire solution path as the regularization parameter varies, automatically adjusting the active set of basis functions. It is based on a dual active-set approach suitable for large-scale problems.
2. Orthogonal Matching Pursuit (OMP): A greedy algorithm that iteratively selects the basis function most correlated with the current residual, updates the solution via least squares, and orthogonalizes the residual.

Key Technical Innovations:

• Path Tracing: Both ASP and OMP generate a "path" of models with increasing complexity (number of active basis functions) in a single run. This eliminates the need for repeated cross-validation to find the optimal regularization parameter.
• De-biasing (TSVD): Since $\ell_1$-regularization (used in ASP) tends to shrink coefficients toward zero, the authors apply a post-processing step using Truncated Singular Value Decomposition (TSVD) on the selected active set. This refines the coefficients, improving accuracy and numerical stability.
• Implementation: The methods are implemented in the Julia package ActiveSetPursuit.jl, integrated with ACEpotentials.jl.

3. Key Contributions

• Automated Basis Selection: Demonstrated that data-driven sparse selection outperforms heuristic or pre-defined basis selection strategies. The selected features often follow non-intuitive patterns that cannot be predicted by simple rules (e.g., total degree).
• Superior Solvers: Showed that existing Julia packages for LASSO paths (LARS.jl, Lasso.jl) lack robustness and efficiency for MLIPs, whereas the proposed ASP and OMP implementations are stable and scalable.
• Generalization and Stability: Proved that sparse models achieve better generalization (lower test errors) and maintain physical stability in long Molecular Dynamics (MD) simulations compared to dense models.
• Interpretability: The sparse selection process reveals which physical interactions (e.g., specific bond types or angular correlations) are most critical for a given material, enhancing the interpretability of the potential.

4. Results

The authors validated their approach on three benchmark datasets:

A. Limited Diversity Materials (Li, Mo, Ni, Cu, Si, Ge)

• Performance: ASP and OMP consistently achieved lower Mean Absolute Errors (MAE) for energies and forces compared to dense solvers (RRQR) and Bayesian methods (ARD).
• Efficiency: Sparse models with ~300–1000 basis functions outperformed dense models with ~5000+ basis functions.
• Pattern: The selected basis functions showed no predictable pattern based on standard selection rules, confirming the necessity of data-driven selection.

B. Silicon (PRX 2018 Dataset)

• Generalization: The dataset included diverse phases (diamond, hcp, liquid, amorphous). Sparse models (ASP/OMP) achieved accuracy comparable to the state-of-the-art Gaussian Approximation Potential (GAP) and dense ACE models trained with Bayesian Linear Regression (BLR).
• Parameter Efficiency: Sparse models reached high accuracy with <50% of the basis functions required by the dense BLR model.
• Property Prediction: Predictions for point defects, elastic constants, and surface energies were highly accurate and matched DFT reference values closely.

C. Liquid Water

• Interaction Prioritization: In a multi-component system (H and O), the sparse solver autonomously prioritized O-H-H three-body interactions over O-O-O or O-O-H, aligning with chemical intuition regarding hydrogen bonding.
• Comparison with Non-linear Models: The linear sparse ACE model (OMP) outperformed the dense linear ACE model and achieved competitive accuracy against the non-linear Cartesian ACE (CACE) model, but with significantly fewer parameters (e.g., 12,000 parameters vs. 69,000 for CACE).
• MD Stability: All fitted potentials remained stable during 300 ps NVT MD simulations at 300 K, accurately predicting diffusivity.

5. Significance and Conclusion

This work establishes sparse regression as a critical component for scalable MLIP development.

• Reduced Human Bias: It removes the need for manual hyperparameter tuning and heuristic basis selection, allowing the data to dictate the model structure.
• Computational Efficiency: By reducing the number of active basis functions, the method significantly lowers the computational cost of energy/force evaluations during MD simulations without sacrificing accuracy.
• Scalability: The approach is applicable to single-element systems and can be extended to complex alloys and compounds, provided the feature space is managed (e.g., via latent space embeddings for many species).
• Future Outlook: While OMP is computationally faster (greedy), ASP offers a more principled path-tracing approach. The authors suggest that these methods pave the way for fully automated, interpretable, and highly efficient interatomic potentials.

In summary, the paper demonstrates that active set algorithms provide a robust, scalable, and automated solution for constructing high-fidelity, sparse linear interatomic potentials, outperforming traditional dense regression and heuristic selection methods across diverse material systems.