Uncertainty-aware phase fraction prediction and active-learning-guided out-of-domain discovery of refractory multi-principal element alloys

This study introduces an uncertainty-aware deep learning framework using Mixture Density Networks to predict phase fractions in refractory multi-principal element alloys, identifies a minimal feature set for robust predictions, and employs an active learning strategy to guide the discovery of novel alloys with unseen elements.

The Gist

Imagine you are a master chef trying to invent the perfect new recipe for a super-strong, heat-resistant metal soup. You have a pantry full of 9 different ingredients (like Titanium, Iron, Aluminum, etc.), and you can mix them in millions of different ways. Your goal is to find the specific mix that creates a "super-metal" that won't melt or break in a jet engine.

The problem? There are too many combinations to test one by one in a real lab. It would take centuries. So, scientists use computer programs (Machine Learning) to guess which recipes will work.

The Problem with Old Computers
In the past, these computer programs acted like overconfident fortune tellers. If you asked them, "Will this mix make a strong metal?" they would say, "Yes!" or "No!" with 100% certainty.

But here's the catch: Sometimes, two very different recipes look almost identical to the computer, yet they produce totally different results. The old computers didn't know this. They would guess confidently even when they were actually clueless. This is dangerous because if you build a real engine based on a confident-but-wrong guess, it could fail.

The New Solution: The "Weather Forecast" Approach
The authors of this paper built a new kind of computer brain called a Mixture Density Network (MDN). Instead of acting like a fortune teller, this new brain acts like a weather forecaster.

• Old Way: "It will rain tomorrow." (100% certainty).
• New Way: "There is a 70% chance of rain, but there's also a 30% chance it might be sunny, and we aren't 100% sure because the data is fuzzy."

This new system doesn't just give you a single answer; it gives you a probability and a confidence score. It tells you, "I think this recipe will work, but I'm only 80% sure because I've never seen this exact mix before."

Three Big Breakthroughs

1. Knowing What You Don't Know (Aleatoric Uncertainty)
Sometimes, the "fuzziness" comes from the data itself. Imagine trying to guess the height of a person based only on their shoe size. Two people with the same shoe size might be very different heights. The computer knows this ambiguity. It says, "I can't be precise here because the input is naturally messy." This helps researchers avoid wasting time on recipes that are too risky.

2. Knowing What You're Missing (Epistemic Uncertainty)
Sometimes, the computer is unsure because it's missing key information. Imagine trying to bake a cake but you forgot to tell the computer how much sugar to use. The computer might guess, but it will be very unsure.
The authors tested this by removing some of the "ingredients" (data features) the computer used. They found that if they cut the list of ingredients down too much, the computer's confidence dropped, and its guesses got worse. They figured out the minimum list of 12 key ingredients needed to make a perfect prediction. This is like finding the essential spices you must have to make the dish taste right.

3. The "Exploration vs. Exploitation" Game
The most exciting part is how they used this to find new metals that the computer had never seen before (Out-of-Distribution discovery).

They set up a game with two strategies:

• The "Safe Bet" Strategy (Exploitation): The computer only suggests recipes that it is very confident about.
  • Result: It finds good recipes quickly, but it stays in a small, safe neighborhood of the design space. It never discovers anything truly new.
• The "Daredevil" Strategy (Exploration): The computer suggests recipes where it is very unsure.
  • Result: It makes more mistakes at first, but by testing these risky, unknown areas, it learns faster. Eventually, it finds amazing new recipes that the "Safe Bet" strategy would have missed.

The Analogy of the Treasure Map
Think of the design space as a giant, foggy island with buried treasure (the perfect metal).

• Old AI would walk in a straight line, confident it was going the right way, but it might walk right off a cliff because it didn't know the map was incomplete.
• This New AI is like a smart explorer with a compass that vibrates when it's near a cliff.
  • If the compass vibrates a little (low uncertainty), it walks confidently toward the treasure.
  • If the compass vibrates wildly (high uncertainty), it knows it's in uncharted territory. It can choose to stay safe, or it can take a risk to explore that foggy area, which might lead to a better treasure than anyone expected.

Why This Matters
This research gives scientists a powerful new tool. It allows them to design new, super-strong metals for extreme environments (like space travel or nuclear reactors) much faster and with much less risk. It stops them from wasting money on experiments that are likely to fail and guides them toward the most promising, yet unexplored, possibilities.

In short: They taught the computer to say, "I'm not sure," and used that uncertainty to guide the search for the next generation of super-materials.

Technical Summary

1. Problem Statement

Refractory Multi-Principal Element Alloys (RMPEAs) offer exceptional mechanical performance in extreme environments but possess a vast compositional design space that is difficult to navigate due to sparse experimental data and complex composition-property relationships.

• Limitations of Current ML: Existing machine learning (ML) models for RMPEA phase prediction typically rely on deterministic mappings (e.g., Multilayer Perceptrons) that output a single phase label or fraction. They fail to account for aleatoric uncertainty (inherent data variability where different compositions map to similar feature spaces but yield different phases) and epistemic uncertainty (uncertainty arising from incomplete knowledge of the optimal input features).
• Out-of-Distribution (OOD) Challenge: Conventional models often produce overconfident predictions when extrapolating to novel elemental combinations (OOD), making them unreliable for discovering new materials outside the training domain.
• Feature Selection Inconsistency: Previous studies use varying combinations of input descriptors, leading to inconsistencies and potential model robustness issues due to missing critical information.

2. Methodology

A. Dataset Generation

• Composition Space: Generated 70,000 unique compositions using random sampling from a design space containing 9 elements (Ti, Fe, Al, V, Ni, Nb, Zr, Mn, Co) with 3–5 elements per alloy.
• Labeling: Used Thermo-Calc (TCHEA6 database) to calculate phase fractions at seven temperatures ranging from 850 K to $0.8 T_m$.
• Target Phases: Focused on FCC, BCC/B2 (grouped), Laves, Sigma, Heusler, and Liquid phases. HCP was excluded due to data scarcity.
• Data Engineering:
  • Started with 51 physicochemical features (e.g., mixing enthalpy, entropy, atomic size difference, electronegativity, etc.).
  • Reduced to 41 features by removing highly correlated pairs ($|r| > 0.9$) and filtering outliers.
  • Oversampling: Applied a custom binning strategy to address severe class imbalance in phase fractions, ensuring underrepresented phases had sufficient training samples.
• Final Dataset: 484,065 compositions after removing failed Thermo-Calc solutions and outliers.

B. Deep Learning Framework: Mixture Density Networks (MDN)

Instead of predicting a single deterministic value, the authors employed Mixture Density Networks (MDNs) to output a probability density function (PDF) for phase fractions.

• Architecture: A neural network with hyperbolic tangent (tanh) activation, optimized via Bayesian optimization. It outputs mixture coefficients ($\pi$), means ($\mu$), and standard deviations ($\sigma$) for a Gaussian mixture model.
• Training: Trained six separate MDN models (one for each target phase) by minimizing the Negative Log-Likelihood (NLL).
• Uncertainty Quantification:
  • Aleatoric Uncertainty: Captured by the variance ($\sigma$) of the predicted Gaussian mixture, reflecting intrinsic data noise.
  • Epistemic Uncertainty: Quantified by analyzing how model performance degrades when the input feature set is reduced.

C. Feature Importance Analysis

• Conducted a perturbation-based feature importance analysis on the BCC phase model.
• Systematically reduced the input feature set from 41 down to the top-ranked features to determine the "minimally sufficient" set required for accurate prediction and low epistemic uncertainty.

D. Active Learning for OOD Discovery

• Scenario: A model trained on Ti-free alloys was tasked with discovering Ti-containing BCC alloys (an OOD scenario).
• Strategy: Implemented a two-stage active learning loop:
  1. Initial Stage: Select candidates based on high predicted BCC fraction.
  2. Iterative Stage: Compare two acquisition strategies:
     • Low-Uncertainty Route (Exploitation): Select candidates where the model is most confident.
     • High-Uncertainty Route (Exploration): Select candidates where the model is most uncertain.
• Process: Selected candidates were added to the training set, and the model was retrained iteratively to improve extrapolation capabilities.

3. Key Results

A. Predictive Performance

• The MDN models achieved high accuracy across all six phases, with $R^2$ values ranging from 0.89 (Sigma) to 0.99 (Laves).
• The models successfully captured the probabilistic nature of phase formation, providing confidence intervals rather than point estimates.

B. Feature Importance and Epistemic Uncertainty

• Top Features: The most important features identified were Covalent radius, Crystal radius, MB electronegativity, $\Omega$, and atomic size mismatch ($\delta$).
• Optimal Subset: Reducing the feature set from 41 to the top 12 features maintained high predictive accuracy ($R^2 \approx 0.98$) and low epistemic uncertainty.
• Critical Threshold: Reducing further to only the top 5 features caused a significant drop in performance ($R^2 \approx 0.63$) and a massive increase in epistemic uncertainty, proving that a minimal but sufficient feature set is crucial for robustness.

C. Active Learning Outcomes

• Exploitation (Low-Uncertainty Route): Rapidly increased the BCC fraction of selected candidates from 0.68 to **0.90** within three cycles. However, it showed minimal improvement in the F1 score (Recall), as it only explored regions the model already understood well.
• Exploration (High-Uncertainty Route): Selected candidates had lower initial BCC fractions (~0.60) but led to a steady increase in Recall and F1 score over cycles. This route expanded the model's knowledge of the Ti-alloy design space.
• Acquisition Size: An acquisition size of 100 or larger was necessary to stabilize the discovery process; smaller sizes (30–50) resulted in fluctuating performance due to insufficient statistical coverage.
• Diversity: The discovered alloys were broadly distributed across the design space, not confined to a narrow region.

4. Key Contributions

1. Uncertainty-Aware Framework: Developed the first deep learning framework for RMPEAs that explicitly quantifies both aleatoric (data-driven) and epistemic (model-knowledge-driven) uncertainties using Mixture Density Networks.
2. Feature Optimization: Identified a minimally sufficient set of 12 input features that balances predictive accuracy with model interpretability, resolving inconsistencies in feature selection found in prior literature.
3. Active Learning Strategy: Demonstrated a practical trade-off between exploitation (high accuracy, low discovery rate) and exploration (lower immediate accuracy, higher long-term discovery potential) in OOD material discovery.
4. Robust OOD Generalization: Showed that uncertainty-guided active learning can successfully bridge the gap between training data (Ti-free) and target discovery space (Ti-containing), overcoming the overconfidence issues of standard deterministic models.

5. Significance

This study provides a robust, transferable tool for accelerated materials discovery. By moving beyond deterministic predictions, the framework allows researchers to:

• Trust predictions: Distinguish between high-confidence predictions and unreliable guesses in unexplored chemical spaces.
• Optimize resources: Guide experimental efforts toward compositions that are either highly likely to succeed (exploitation) or likely to reveal new physics (exploration).
• Enhance reliability: Reduce the risk of wasted experimental effort on "hallucinated" materials by explicitly modeling uncertainty.

The approach is broadly applicable to other complex materials systems where data is sparse and the cost of experimental failure is high.