Evaluation of Foundational Machine Learned Interatomic Potentials for Migration Barrier Predictions

This study benchmarks five foundational machine learned interatomic potentials against DFT-NEB calculations to evaluate their accuracy in predicting ionic migration barriers, revealing that models like MACE-MP-0 and Orb-v3 excel in barrier prediction and high-throughput screening despite a lack of correlation with local geometry accuracy.

The Gist

Imagine you are trying to design the ultimate battery for your electric car or your phone. The secret to a great battery isn't just how much energy it holds, but how fast the ions (tiny charged particles) can zip through the material to charge and discharge.

Think of these ions as marbles trying to roll through a maze.

• The walls of the maze represent the energy barriers the ions must climb over to move.
• The height of the walls is called the Migration Barrier ($E_m$).
• If the walls are low, the marbles roll fast (great battery). If the walls are high, the marbles get stuck (slow battery).

The Problem: The "Super-Computer" Bottleneck

To figure out the height of these walls, scientists usually use a method called DFT-NEB.

• The Analogy: Imagine trying to find the exact path a marble takes through a complex, 3D maze made of invisible, shifting walls. To do this with perfect accuracy, you need a super-computer to simulate every single step.
• The Issue: It's incredibly slow and expensive. It's like hiring a team of architects to hand-calculate the perfect route for every single marble in a million different mazes. You can't do this fast enough to find the best new battery materials.

The Solution: The "AI Guessers" (MLIPs)

Enter Machine Learned Interatomic Potentials (MLIPs). These are AI models trained on millions of existing chemical structures.

• The Analogy: Instead of hiring architects to calculate every route from scratch, you hire AI assistants who have seen thousands of mazes. They can instantly guess the path and the wall heights.
• The Goal: The authors of this paper wanted to test five of these top-tier AI assistants to see:
  1. Do they guess the wall height correctly?
  2. Do they guess the shape of the maze correctly?
  3. Can they help us find the best battery materials faster?

The Race: Who Won?

The researchers put five AI models through a gauntlet of 574 different battery materials (the "mazes") and compared their guesses against the "gold standard" (the slow, expensive super-computer calculations).

Here is how the runners finished:

1. The All-Rounder (MACE-MP-0): This model was the most consistent. It didn't make huge mistakes and gave the best average score across the board. It's like the reliable veteran who finishes every race in a solid time.
2. The Specialist (Orb-v3): This model was the star when the conditions were right. If the maze wasn't too weird, Orb-v3 gave the most precise guesses. However, it sometimes struggled to "find its footing" in very complex mazes (it had trouble converging on some difficult structures).
3. The Classifiers (Orb-v3 & SevenNet): These two were the best at a specific job: Sorting. If you just want to know, "Is this a good battery material or a bad one?" (without needing the exact wall height), these two got it right 82-85% of the time. They are perfect for quickly screening thousands of materials to find the winners.
4. The Under-estimators (CHGNet & M3GNet): These models tended to be overly optimistic. They often guessed the walls were lower than they actually were. While they were good at finding easy mazes, they got confused by the hard ones.

The Big Surprise: The "Good Guess, Bad Map" Paradox

The most fascinating discovery in the paper is a bit counter-intuitive.

• The Expectation: You'd think that if an AI guesses the wall height perfectly, it must have also drawn the maze map perfectly.
• The Reality: Nope.
  • Sometimes, an AI guessed the wall height perfectly but drew a completely wrong maze map.
  • Sometimes, it drew a perfect map but guessed the wall height wrong.

Why?

• Low Walls (Easy Mazes): If the walls are flat and low, it doesn't matter if the map is slightly wobbly; the marble still rolls fast. The AI gets the "speed" right even if the "shape" is wrong.
• High Walls (Hard Mazes): If the walls are steep and deep, even a tiny error in the map (a slightly wrong angle) can make the AI think the wall is huge or tiny. Here, the shape matters a lot, but the AI often fails to get the height right even if the shape is okay.

The Practical Takeaway: How This Helps You

This paper is like a user manual for AI tools in battery research.

1. Speed Up Discovery: We don't need to run the slow super-computer for every single material anymore. We can use Orb-v3 or SevenNet to quickly filter out the bad materials and keep only the promising ones.
2. Better Starting Points: Even when the AI isn't 100% perfect, the "maps" (geometries) it generates are often better starting points than random guesses. This means if we do need to run the slow super-computer later, the AI gets it 90% of the way there, saving massive amounts of time.
3. No Magic Bullet: There is no single AI that does everything perfectly. You have to pick the right tool for the job (e.g., use Orb-v3 for sorting, MACE-MP-0 for general accuracy).

In short: This research proves that AI can act as a powerful "co-pilot" for battery scientists. It won't replace the final, precise calculations, but it will help us fly through the search for the next generation of super-batteries much faster.

Technical Summary

1. Problem Statement

The development of next-generation battery materials requires the rapid identification of ionic conductors with high diffusivity ($D$). Ionic diffusivity is exponentially dependent on the migration barrier ($E_m$), the energy an ion must overcome to hop between lattice sites.

• Current Bottleneck: Accurate calculation of $E_m$ typically relies on Density Functional Theory (DFT) combined with the Nudged Elastic Band (NEB) method. While precise, DFT-NEB is computationally expensive, particularly because it requires an accurate initial guess for the Minimum Energy Path (MEP). Linear interpolation (LI) of coordinates often yields poor initial guesses, leading to convergence difficulties and high computational costs.
• The Gap: While foundational Machine Learned Interatomic Potentials (MLIPs) have emerged as powerful tools for general materials property prediction, their specific efficacy in predicting $E_m$ and generating accurate MEP initial guesses for high-throughput screening has not been systematically benchmarked across diverse battery-relevant chemistries.

2. Methodology

The authors benchmarked five state-of-the-art foundational MLIPs (pre-trained on large, diverse datasets) integrated with the NEB framework:

1. MACE-MP-0: E(3)-equivariant GNN capturing many-body interactions.
2. SevenNet-MF-ompa: Equivariant GNN with multifidelity learning.
3. Orb-v3: Roto-equivariant GNN with analytical gradients and infinite neighbor lists.
4. CHGNet: GNN incorporating magnetic moments and atomic charges.
5. M3GNet: GNN including three-body interactions.

Datasets:

• Dataset-1 (60 systems): Used for detailed geometry analysis. Contains full DFT-NEB relaxed structures of intermediate images.
• Dataset-2 (574 systems): A curated subset of literature data spanning diverse battery chemistries (layered, spinels, olivines, perovskites, etc.) with $E_m$ values between 0.06 eV and 2.5 eV.

Workflow:

• NEB Setup: The authors compared standard Linear Interpolation (LI) and Image Dependent Pair Potential (IDPP) interpolation as initial guesses. They found IDPP marginally superior.
• MLIP-NEB: All models were used to relax intermediate images without further DFT refinement.
• Metrics:
  • Barrier Accuracy: Mean Absolute Error (MAE) against DFT-NEB values.
  • Classification: Ability to distinguish "good" ($E_m < 500$ meV) vs. "bad" conductors.
  • Geometry Similarity ($\theta$): A novel metric comparing local geometric features (pairwise distances and solid angles) of MLIP-relaxed structures against DFT-NEB ground truth versus simple LI.
  • Correlation: Analysis of the relationship between $E_m$ prediction accuracy and geometry prediction accuracy.

3. Key Contributions

• First Comprehensive Benchmark: Provides the first systematic evaluation of foundational MLIPs specifically for ionic migration barrier predictions across a wide range of battery materials.
• Geometry-Barrier Decoupling: Introduces a rigorous geometric similarity metric ($\theta$) and demonstrates a counter-intuitive lack of correlation between accurate geometry prediction and accurate barrier prediction.
• High-Throughput Screening Protocol: Validates the use of MLIP-NEB as a pre-screening tool to generate superior initial guesses for DFT-NEB, potentially accelerating the discovery of novel ionic conductors.

4. Key Results

A. Barrier Prediction Accuracy

• Overall Performance: MACE-MP-0 achieved the lowest overall MAE (0.310 eV) across the full dataset.
• Outlier Exclusion: When excluding common outliers (errors > 1 eV), Orb-v3 showed the best performance with an MAE of 0.245 eV, followed closely by MACE-MP-0 (0.239 eV).
• Bias Analysis:
  • M3GNet and CHGNet exhibited a systematic bias toward underestimating $E_m$ (73–78% of predictions were underestimates).
  • MACE-MP-0, SevenNet, and Orb-v3 showed balanced distributions of over- and under-estimation.
• Range Dependency:
  • All models struggled with high barriers ($>1.3$ eV), with accuracy dropping significantly.
  • Orb-v3 maintained the most robust performance across a wide range of $E_m$ values.
  • Simpler models (CHGNet, M3GNet) performed well only in low-barrier regimes but degraded rapidly as complexity increased.

B. Classification Performance

• Using a threshold of 500 meV to classify conductors:
  • Orb-v3 achieved the highest accuracy (84.8%).
  • SevenNet followed closely (82.9%).
  • Both models are deemed highly reliable for high-throughput screening to filter out non-conductive materials.

C. Geometry Prediction & Initial Guesses

• Superiority over LI: MLIP-NEB relaxed structures provided better initial guesses for the MEP than simple Linear Interpolation in >71% of cases (specifically for MACE-MP-0 and SevenNet).
• Best Geometry Models: SevenNet (71.9% "good" geometries) and MACE-MP-0 (lowest "bad" geometry fraction at 19.0%) outperformed others in generating accurate local geometries.
• IDPP vs. MLIP: While IDPP is better than LI in some cases, MLIP relaxation generally provided significantly better initial guesses than IDPP alone.

D. The Geometry-Barrier Correlation Paradox

• Key Finding: There is no positive correlation between accurate $E_m$ prediction and accurate geometry prediction.
• Inverse Relationship:
  • Models often achieved their best geometry predictions for systems with high $E_m$ (where they simultaneously had the worst barrier predictions).
  • Conversely, for low $E_m$ systems, models often predicted barriers accurately despite poor geometry predictions.
• Explanation: Low $E_m$ systems often have "flat" potential energy surfaces where small geometric errors do not significantly alter the energy barrier. High $E_m$ systems have "deep" minima where small geometric errors lead to large energy deviations, yet the models still managed to predict the geometry well (likely due to the stability of the minima).

5. Significance and Impact

• Accelerated Discovery: The study establishes that Orb-v3 and SevenNet are the most suitable candidates for high-throughput screening of battery materials due to their high classification accuracy (>82%) and robustness across barrier ranges.
• Workflow Optimization: Using MACE-MP-0 or SevenNet to generate initial NEB paths can significantly reduce the computational cost of subsequent DFT-NEB relaxations by providing better starting geometries in >70% of cases.
• Theoretical Insight: The finding that accurate barriers do not require accurate local geometries challenges the assumption that geometric fidelity is a prerequisite for energetic fidelity in MLIPs. This suggests that for screening purposes, MLIPs can be trusted for energy predictions even if local structural details are imperfect.
• Limitations: The study highlights that all models struggle with very high migration barriers ($>1.3$ eV), indicating a need for specialized training or transfer learning for difficult ionic transport cases.

In conclusion, this work provides a critical roadmap for integrating foundational MLIPs into battery materials discovery pipelines, validating their utility for both rapid screening and accelerating high-fidelity DFT calculations.