Technical Summary: ADAPT - Lightweight, Long-Range Machine Learning Force Fields Without Graphs

1. Problem Statement

Point defects are central to the properties of materials, yet modeling them presents significant computational challenges. First-principles methods like Density Functional Theory (DFT) are accurate but prohibitively expensive for high-throughput screening or structural relaxation of large supercells (often containing hundreds to thousands of atoms).

While Machine Learning Force Fields (MLFFs) offer a computationally efficient alternative, state-of-the-art approaches based on Graph Neural Networks (GNNs) face specific limitations when applied to point defects:

• Oversmoothing: GNNs often struggle to represent long-range interactions, which are critical for the structural formation of defects.
• Local Bias: GNN architectures inherently focus on local interactions, requiring deep message-passing layers to propagate information across large supercells. This can lead to "over-squashing" and the degradation of long-range signals.
• Computational Cost: Message-passing algorithms in GNNs are computationally demanding for large periodic supercells, limiting their scalability.

2. Methodology: The ADAPT Architecture

The authors introduce the Accelerated Deep Atomic Potential Transformer (ADAPT), an MLFF that abandons graph representations in favor of a direct coordinate-based formulation using Transformer architectures.

2.1 Core Design Principles

• Tokenization: Atoms are treated as "tokens." A structure with $n$ atoms is represented as a sequence of $n$ tokens.
• Input Features: Each token is a 12-dimensional vector containing Cartesian coordinates ($x, y, z$) and atomic descriptors (group, period, electronegativity, covalent radius, valence electrons, ionization energy, electron affinity, atomic radius, molar volume).
• No Graph Inductive Biases: The model eschews graph topologies and explicit geometric priors (like bond definitions), relying instead on the Transformer's ability to learn relationships from raw coordinates.

2.2 Architecture Components

The system employs two distinct networks: one for force prediction and one for energy prediction.

A. Force Prediction (Transformer Encoder)

1. Embedding: A Multi-Layer Perceptron (MLP) maps the 12-dimensional input tokens into a higher-dimensional latent space ($d_{model}$).
2. Transformer Encoder: The sequence passes through $k$ encoder blocks, each containing:
   • Layer Normalization (LN): Ensures training stability.
   • Multi-Head Scaled Dot-Product Attention: The core mechanism where tokens interact. Unlike GNNs, this uses full, unmasked self-attention, allowing every atom to interact with every other atom in a single step, capturing long-range dependencies without depth-based propagation.
   • Feed-Forward Network (FFN): Applies non-linear transformations to individual tokens.
3. Force Projection: A linear projection maps the encoded tokens to 3D force vectors ($f_x, f_y, f_z$).

B. Energy Prediction (MLP + Residual)

• A separate model predicts the formation energy of the entire structure.
• The authors tested three architectures (Decoder, MLP, MLP+Residual) and found that an MLP with residual connections performed best.
• Input Handling: Since MLPs require fixed-length inputs, structures are padded to a maximum of 220 atoms (based on the dataset's 64-atom Si lattice plus dopants).

2.3 Loss Function and Training Strategy

• Importance-Weighted MSE: To address the scale imbalance between forces in the defect core (high magnitude) and the bulk lattice (near zero), the authors introduce a specialized loss function.
  • An importance mask $m_i$ is applied, weighting atoms based on their proximity to defect locations ($D$).
  • The weight decays as a power law of the distance from the defect: $m_i = \prod_{j \in D} (1 + \frac{\lambda_1}{\|r_i - r_j\|^2 + \lambda_2})$.
  • This ensures the model prioritizes accurate predictions in the critical defect regions rather than minimizing error on the trivial bulk atoms.
• Separate Models: Forces and energies are predicted by separate models. This allows for modular deployment and independent retraining, though the authors note this results in a non-conservative force field (forces are not guaranteed to be the gradient of the energy surface).

3. Key Contributions

1. Graph-Free MLFF: ADAPT demonstrates that high-accuracy force fields can be built without graph representations, utilizing Transformers to handle global atomic interactions directly.
2. Explicit Long-Range Modeling: By using full self-attention, ADAPT explicitly models all pairwise atomic interactions, overcoming the oversmoothing and long-range signal degradation issues inherent in GNNs.
3. Computational Efficiency: The architecture leverages dense linear algebra, which is highly optimized on modern hardware (GPUs), avoiding the sparse operations that limit GNN performance.
4. Specialized Loss Function: The introduction of an importance-weighted loss function specifically tailored for defect simulations improves practical utility over standard Mean Squared Error (MSE).

4. Results

The model was evaluated on a dataset of 6,082 silicon defect DFT trajectories (containing simple and complex defects).

• Accuracy:
  • Forces: ADAPT achieved a Mean Absolute Error (MAE) of 0.0126 eV/Å (Small configuration), representing a ~33% reduction in error compared to a retrained MACE model. It significantly outperformed pretrained universal models like MatterSim 5M.
  • Energy: The ADAPT energy predictor achieved an MAE of 0.5782 eV, outperforming retrained MACE and MatterSim 5M.
• Computational Cost:
  • Training Time: Training the ADAPT force model required 3 compute hours (80 epochs on 1 NVIDIA A100).
  • Comparison: Retraining MACE required 680 compute hours (300 epochs on 16 NVIDIA A100s). ADAPT is >227× faster to train for force prediction and >138× faster overall when including the energy model.
• Hardware Accessibility: The compact design allows training on commodity hardware, including workstations and consumer-grade laptops, whereas GNN-based training typically requires large-scale clusters.

5. Significance and Claims

The paper claims that ADAPT offers a paradigm shift in modeling crystalline defects by:

• Bypassing DFT Relaxations: With force prediction accuracy comparable to DFT stopping criteria (~0.01 eV/Å), ADAPT serves as a viable surrogate for accelerating structural relaxation pipelines.
• Scalability: The ability to handle large supercells efficiently without the computational overhead of message-passing makes it suitable for high-throughput defect databases.
• Performance over Inductive Bias: The results challenge the necessity of strong geometric inductive biases (graphs) for this specific domain, suggesting that with sufficient data and global attention mechanisms, Transformers can learn the necessary physical invariances (permutation, translation, rotation) directly from coordinates.

Limitations Acknowledged:

• The model is currently non-conservative (forces and energies are decoupled), limiting its use in molecular dynamics simulations requiring strict energy conservation.
• It relies on large datasets; performance may degrade with limited training data compared to GNNs.
• Current validation is specific to silicon defects; applicability to diverse bulk structures remains an open question.