🤖 machine learning

Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation

This paper demonstrates that atom-averaged features from pretrained Machine-Learning Interatomic Potentials (MLIPs), such as MACE, serve as effective coarse coordinates for both evaluating material generative models via a novel Coarse-Fine Transport Distance (CFTD) metric and guiding the generation process itself.

Original authors: Paul Hagemann, Katharina Ueltzen, Simon Müller, Janine George, Philipp Benner

Published 2026-08-03

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Paul Hagemann, Katharina Ueltzen, Simon Müller, Janine George, Philipp Benner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an architect trying to design a new, super-strong building material. You could spend years digging through a massive library of existing blueprints, hoping to find a pattern you can tweak. But what if you had a magical robot that could dream up entirely new blueprints from scratch? This is the promise of generative artificial intelligence in materials science. Instead of just copying old designs, these AI models try to invent new crystal structures—tiny, repeating patterns of atoms that could make better batteries, faster computers, or stronger bridges.

However, there's a big catch: how do you know if the AI is actually inventing something new and useful, or if it's just copying its homework? If the robot memorizes the library and spits out the same old blueprints with slightly different colors, that's not a discovery; it's plagiarism. Scientists have been struggling to build a "report card" that can tell the difference between a brilliant new invention and a lazy copycat. They need a way to measure two things at once: Quality (is the structure stable and physically possible?) and Novelty (is it actually new, or just a recycled version of something we already know?).

This is where a new paper comes in, offering a clever solution to this grading problem. The authors, a team of researchers, realized that previous methods were like trying to judge a painting by only looking at the frame or only looking at the colors, but never both together. They introduce a new scoring system called Coarse-Fine Transport Distance (CFTD). Think of it as a two-lens camera: one lens zooms in to check if the AI is copying (the "Fine" lens), while the other zooms out to check if the structure is physically sound and stable (the "Coarse" lens).

The paper doesn't just stop at grading; it also shows how to use these lenses to teach the AI better. By feeding the AI a "coarse" summary of what a good material looks like (derived from a powerful physics simulator called MACE), they can guide the robot to dream up structures that are not only new but also stable. The results suggest that this dual-lens approach is much better at spotting when an AI is "mode collapsing" (stuck in a rut, repeating the same few ideas) compared to older methods. It's a step forward in making sure that when AI discovers a new material, it's a genuine breakthrough, not just a clever forgery.

The Two-Lens Camera for AI Dreams

Imagine you are a teacher grading a student's art project. The student claims to have painted a brand-new, never-before-seen landscape. How do you check?

The "Fine" Lens (The Copy Detector): You look closely at the brushstrokes. Did the student just trace a famous painting from a textbook? In the world of crystals, this is checking if the AI just memorized a structure from its training data. The authors use a "contrastive" AI (a smart detector trained to spot similarities) to see if the new crystal is just a slightly tweaked version of an old one. If it's too similar, it's a copy.
The "Coarse" Lens (The Stability Check): You step back and look at the whole picture. Does the landscape make sense? Are the mountains floating? Is the water flowing uphill? In crystals, this means checking if the atoms are arranged in a way that is physically stable. The authors use a pre-trained "Machine-Learning Interatomic Potential" (MLIP), specifically a model called MACE, which acts like a physics expert. They take the "hidden thoughts" (features) of this expert model to see if the new crystal feels stable.

The paper introduces CFTD to combine these two checks into one score. The name comes from the idea of "Coarse" (the big-picture stability from MACE) and "Fine" (the detailed identity check from the contrastive model).

Why Old Grading Systems Failed

Before this paper, scientists mostly used a system called SUN metrics (Stability, Uniqueness, Novelty). The problem with SUN is that it checks each crystal one by one, like a teacher checking homework one page at a time. If an AI generates 1,000 copies of the same "good" crystal, the SUN metric might still give it a high score because every single copy is unique compared to the others in the pile, and they are all stable. It misses the bigger picture: the AI is stuck in a rut, repeating the same idea over and over. This is called mode collapse.

The authors argue that to catch this behavior, you need to look at the distribution of the results—how the whole pile of generated crystals looks compared to the whole pile of training data. They also found that previous attempts to measure quality and novelty together (like their earlier "TNovD" metric) had a flaw: if you made the AI generate more stable crystals, the metric would accidentally think they were less novel, and vice versa. It was a zero-sum game where improving one hurt the other.

The "Goldilocks" Solution

The authors' big idea is to stop trying to measure quality and novelty with the same ruler. Instead, they use two different rulers (featurizers) and create a "Goldilocks Zone."

Imagine a hallway with three zones:

The "Too Close" Zone (Memorization): If a new crystal is too similar to a training crystal (detected by the Fine lens), it's a copy.
The "Too Far" Zone (Bad Quality): If a new crystal is too different in terms of physics (detected by the Coarse lens), it's unstable and likely won't work.
The "Just Right" Zone (The Goldilocks Zone): This is the sweet spot where the crystal is different enough to be new, but stable enough to be real.

The CFTD metric counts how many crystals fall outside this Goldilocks zone. If the AI is repeating patterns (memorizing) or failing (making unstable junk), the score goes up. If the AI is doing a great job, the score goes down.

What They Found

The team tested this new metric on several existing AI models that generate crystals. Here is what they discovered:

It Detects Repetition: When they simulated an AI that was "mode collapsing" (generating only a tiny subset of possible structures), the CFTD metric immediately flagged it as bad. Older metrics like continuous SUN missed this because they were looking at individual crystals rather than the whole group.
It Detects Overfitting: They trained a model on a tiny dataset (only 2,000 materials). As the model trained longer, it started memorizing the data. The CFTD score went up, showing that the model was overfitting. Interestingly, this was only visible after the crystals were "relaxed" (optimized for stability), proving that the metric works best when combined with physics simulations.
It Guides the AI: The authors didn't just use the metric to grade; they used it to teach. They built a new AI generator that was "conditioned" on the coarse MACE features. They fed the AI a "summary" of a stable material and asked it to build a crystal that matched that summary.
- The result? The AI successfully followed the instructions. When they gave it a summary of a stable material, it generated stable materials. When they gave it a summary of a different material, it generated that instead.
- However, because they used summaries from the training data, the AI did memorize a bit more. This suggests that while the "coarse" features are a powerful guide, the next step is to teach the AI to generate new summaries, not just copy old ones.

The Takeaway

This paper suggests that using the "hidden knowledge" of physics simulators (like MACE) as a guide and a grader is a game-changer for materials science. By separating the check for "is it a copy?" from the check for "is it stable?", the new CFTD metric gives a much clearer picture of whether an AI is truly inventing new materials or just pretending to.

The authors show that these "coarse" features carry a lot of useful information about a material's stability and composition, even when compressed. While they didn't solve the problem of generating perfectly novel materials yet, they provided a much better map for navigating the vast landscape of possible crystals, ensuring that the next generation of AI discoveries are real, stable, and truly new.

=== DRAFT ===
Here is a detailed technical summary of the paper "Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation."

Problem Statement

The field of inorganic crystal structure generation is increasingly utilizing generative machine learning. However, evaluating these models remains a significant challenge. Existing evaluation metrics, such as the continuous SUN (Stability, Uniqueness, Novelty) metrics, often operate at the instance level (structure-by-structure). This approach struggles to distinguish between a model that genuinely captures the data distribution and one that merely produces novel but unstable structures or suffers from mode collapse. Furthermore, traditional distributional metrics like the Fréchet Inception Distance (FID) are difficult to apply directly because they typically minimize distance when the generated distribution exactly matches the training distribution, failing to penalize a lack of novelty.

A core difficulty lies in the "coverage-novelty tradeoff." Metrics that capture both quality (coverage of the training distribution) and novelty (distance from the training distribution) often force a binary choice: improving one metric inherently degrades the other if a single feature representation is used. Additionally, determining whether a generated structure is a true novelty or merely a memorized training sample (or a simple chemical substitution of one) is non-trivial.

Methodology

The authors propose a dual-featurizer framework called the Coarse-Fine Transport Distance (CFTD). This metric extends the previously introduced Transport Novelty Distance (TNovD) by decoupling the evaluation of "quality" and "novelty" into two distinct geometric spaces using different feature extractors.

1. The Dual-Featurizer Framework:

The "Fine" Featurizer (Novelty/Memorization): This component utilizes a contrastive Graph Neural Network (GNN) trained with an InfoNCE loss. It is designed to detect structural identity and memorization. The training augmentation strategy includes rotation, translation, Gaussian noise on fractional coordinates, and crucially, substitution with chemically similar elements (based on the modified Pettifor scale). This ensures that structures differing only by chemically similar substitutions are treated as "identical" (i.e., not novel), preventing models from gaining novelty scores by simply swapping elements.
The "Coarse" Featurizer (Quality/Stability): This component utilizes hidden features from a pretrained Machine-Learning Interatomic Potential (MLIP), specifically MACE-MP-0b3. Instead of using the final energy or force outputs, the authors extract invariant "node features" from the interaction blocks of the MACE model. These features are mean-pooled to create a fixed-size vector representing the entire crystal, followed by a random projection to reduce dimensionality. The authors argue these features encode "implicit stability" (structural, chemical, and physical information) rather than explicit thermodynamic quantities like energy above the hull.

2. The CFTD Metric:
The metric calculates an Optimal Transport (OT) distance between the training distribution and the generated distribution.

A "Goldilocks zone" is established using two thresholds: $\tau_{mem}$ (for memorization) and $\tau_{qual}$ (for quality).
The cost function for the OT plan is a weighted average of the distances in the "Fine" space ( $C_{id}$ ) and the "Coarse" space ( $C_{mace}$ ).
The final score penalizes generated samples that are either too close to training data in the Fine space (memorization) or too far in the Coarse space (low quality/instability). Samples falling within the Goldilocks zone are not penalized.

3. MLIP-Conditional Generation:
To demonstrate the utility of the coarse features beyond evaluation, the authors designed a conditional generative model. They treat the material as a tuple including a coarse MACE feature vector ( $h$ ). The model generates atom types, lattice parameters, and fractional coordinates conditioned on a sampled pair of atom count ( $n$ ) and MACE features ( $h$ ). This tests whether the hidden MACE features can serve as a guidance signal for generation.

Key Contributions

CFTD Metric: Introduction of the Coarse-Fine Transport Distance, which resolves the intrinsic quality-novelty tradeoff of single-featurizer metrics by using a dual-featurizer geometry.
MACE Features for Evaluation: Demonstration that mean-pooled hidden features from pretrained MLIPs (MACE) serve as effective "coarse" coordinates that capture implicit stability and chemical composition, outperforming previous contrastive-only approaches in detecting stability errors.
Robustness to Mode Collapse: Showing that CFTD is more sensitive to distributional shifts (mode collapse) than instance-based metrics like continuous SUN.
Conditional Generation: Proof that MACE features can act as a conditioning signal for material generators, guiding the model toward more stable structures, albeit at the cost of reduced novelty when using empirical training features.

Results

Evaluation of Generative Models:
The authors benchmarked sets of generated crystal structures from various models (MatterGen, ADiT, Chemeleon, DiffCSP, etc.), totaling 10,000 structures across all models, using CFTD.

Relaxation Effects: Relaxing generated structures with MACE generally improved quality (lower CFTD) but increased memorization, suggesting generative models often push existing materials slightly away from energy minima rather than creating truly novel stable structures.
Comparison with cSUN: CFTD ranked models differently than continuous SUN (cSUN). Notably, Chemeleon2, which ranked highest in cSUN, scored the lowest on the CFTD metric. The authors attribute this to Chemeleon2's reinforcement learning strategy, which overweights rare events and shifts the distribution away from the training data (a form of mode collapse or distributional shift) that CFTD penalizes but instance-level metrics miss.
Perturbation Tests: Experiments involving lattice strain, Gaussian noise, and element substitutions showed that CFTD is highly sensitive to structural degradation (quality loss) and chemical similarity, correctly penalizing unstable structures and identifying memorization.

Conditional Generation Experiments:

Overfitting Detection: When trained on a small subset (2,000 samples), the model overfit. CFTD detected this overfitting more effectively than validation loss, particularly after structural relaxation revealed that "near-miss" structures were actually memorized copies.
Guidance Capability: A model trained with MACE feature conditioning on the full dataset produced materials with higher validity and stability (LeMat-GenBench metrics) compared to an unconditioned baseline, though with lower novelty. The generated materials' features closely matched the conditioning inputs, confirming the model faithfully obeyed the MACE guidance.

Significance and Claims

The paper claims that CFTD provides a more natural, distributional approach to evaluating material generative models compared to instance-based metrics. By separating the concerns of "is this structure stable?" (Coarse/MACE) and "is this structure a copy?" (Fine/Contrastive), the metric avoids the false tradeoffs inherent in previous methods.

The authors assert that hidden features from pretrained MLIPs contain rich structural and chemical information that can be leveraged not just for evaluation, but as a guidance signal for generation. This opens a pathway for hierarchical generative models where a coarse representation (MACE features) is generated first to ensure stability and property adherence, followed by the generation of specific atomic coordinates.

The paper concludes modestly, noting that while CFTD is a powerful tool, the learnability and identifiability of the MACE feature space for hierarchical generation require further investigation. It also highlights that the current implementation uses empirical MACE features, and future work should focus on learning a generative model for the MACE features themselves to decouple stability from the specific training distribution.