Out-of-distribution transfer of PDE foundation models to material dynamics under extreme loading

This paper benchmarks the out-of-distribution transfer capabilities of pretrained PDE foundation models (POSEIDON and MORPH) to extreme-loading material dynamics by evaluating their sample efficiency in predicting terminal states for shock-driven multi-material interfaces and dynamic fracture, comparing fine-tuning against training from scratch.

The Gist

Imagine you are trying to teach a super-smart robot how to predict the future of physical objects.

For a long time, scientists have been training these robots (called PDE Foundation Models) using "calm" data. Think of this like teaching a student only by showing them videos of smooth, flowing rivers and gentle wind. The student becomes an expert at predicting how water ripples or how air moves around a wing.

But what happens when you ask that same student to predict what happens when a meteorite hits a spaceship, or when a metal beam shatters under extreme pressure? The physics change completely. Instead of smooth flows, you get shockwaves, sudden cracks, and chaotic explosions.

This paper is a report card on how well these "river experts" can handle "meteorite impacts."

The Big Experiment: From Rivers to Explosions

The researchers from Los Alamos National Laboratory took two of the best "river expert" robots available (named POSEIDON and MORPH) and tried to teach them two new, very dangerous jobs:

1. The "Shockwave" Job (PLI): Imagine two different liquids (like oil and water) stacked on top of each other. Now, hit them with a massive shockwave. The boundary between them gets messy, turbulent, and chaotic. The robot has to predict what the mess looks like after the shockwave has passed.
2. The "Shattering" Job (FRAC): Imagine a block of metal. Now, hit it until it cracks. The cracks spread like lightning, branching out in unpredictable ways. The robot has to predict the final pattern of the broken pieces.

The Challenge: The "Out-of-Distribution" Problem

In machine learning, "Out-of-Distribution" (OOD) is a fancy way of saying: "You've never seen anything like this before."

It's like taking a chef who only knows how to bake perfect, fluffy cakes and asking them to cook a steak that's on fire. The chef knows about heat and ingredients, but the situation is totally different.

The researchers wanted to see:

• Can these robots learn the new job quickly (Sample Efficiency)?
• Do they need to be retrained from scratch, or can they just "fine-tune" their existing knowledge?

The Results: Who Won?

The results were a mix of success and "not quite there yet."

1. The "Shockwave" Test (PLI):

• Winner: MORPH.
• The Analogy: MORPH was like a student who had studied a bit of everything (different types of fluids, different dimensions). When faced with the chaotic shockwave, MORPH adapted better. It could see the big picture of the messy interface.
• POSEIDON (the specialist in smooth fluid flows) struggled a bit more, getting confused by the sudden, violent jumps in the data.

2. The "Shattering" Test (FRAC):

• Winner: POSEIDON (by a tiny margin).
• The Analogy: Here, the specialist actually did slightly better. It seems the way POSEIDON was trained on fluid dynamics helped it understand how cracks propagate, even though it wasn't trained on cracks specifically.
• MORPH did okay, but it didn't have the same edge here.

The "Data Diet" Experiment

The researchers also tested how much "food" (training data) the robots needed to learn.

• The "Starving" Phase (Little Data): When they gave the robots very few examples to learn from, the pre-trained robots (the ones who knew about rivers) were much better than robots trained from zero. They had a head start, like a student who already knows how to read, trying to learn a new language.
• The "Feast" Phase (Lots of Data): As they gave the robots more and more examples of the new jobs, the advantage of being pre-trained disappeared. If you show a river-expert enough pictures of meteorites, they can eventually learn the meteorite rules just as well as someone who started with meteorites.

The Big Takeaway

The main lesson of this paper is: You can't just teach a robot about rivers and expect it to be an expert on explosions.

While the pre-trained models were helpful (especially when data was scarce), they weren't perfect. The "smooth river" knowledge didn't transfer perfectly to "chaotic explosion" knowledge.

The Future:
To make truly robust AI for engineering and safety (like designing better spacecraft or predicting earthquakes), we need to stop training these models only on smooth, calm data. We need to feed them more "extreme" data—shocks, fractures, and explosions—right from the start.

Think of it as upgrading the curriculum: We need to stop teaching our AI students just about calm lakes and start teaching them about tsunamis and earthquakes so they are ready for the real world.

Technical Summary

1. Problem Statement

The paper addresses a critical gap in the application of PDE Foundation Models (PDE-FMs). While these models have shown promise in fluid dynamics and smooth-field regimes (e.g., Navier-Stokes, advection-diffusion), their utility in extreme-loading material dynamics remains unverified.

• The Challenge: Extreme-loading scenarios (e.g., hypersonic impact, explosive shock, dynamic fracture) are characterized by:
  • Highly non-smooth fields dominated by shocks and steep gradients.
  • Evolving multi-material interfaces and discontinuities (cracks, fractures).
  • Rate-dependent inelastic responses and strong sensitivity to early-time perturbations.
• The Gap: Most existing PDE foundation models are pretrained on fluid-centric datasets. It is unclear if the inductive biases learned from smooth fluid flows transfer effectively to discontinuity-dominated solid mechanics.
• The Task: The authors formulate the downstream task as terminal-state prediction. Instead of autoregressive step-by-step rollout, the model must learn a long-horizon operator mapping directly from the first snapshot (initial state) to the final snapshot (terminal state) without intermediate supervision. This is crucial for design optimization and uncertainty quantification in safety-critical systems.

2. Methodology

Datasets

The study introduces a unified evaluation protocol using two complementary, open-source, discontinuity-dominated datasets:

1. Perturbed Layered Interface (PLI):
   • Physics: 2D axisymmetric multi-material simulations involving shock propagation through complex targets (copper, aluminum, steel, polymers, explosives).
   • Characteristics: 5,293 simulations, 38 channels, 100 time steps ($1120 \times 400$ resolution). Focuses on Richtmyer-Meshkov instabilities and interface dynamics.
2. Dynamic Fracture/Failure Evolution (FRAC):
   • Physics: Fracture initiation and propagation in tungsten under horizontal boundary conditions.
   • Characteristics: ~200,000 simulations (subset used), generated via Phase-Field and Finite-Discrete Element Methods (FDEM/HOSS). Focuses on crack network evolution and damage localization.

Models Evaluated

Two open-source, pretrained PDE foundation models were benchmarked:

1. POSEIDON: Based on the scalable Operator Transformer (scOT). Pretrained on fluid dynamics (Navier-Stokes and Euler equations) from PDEGym. It uses time-conditioned layer normalization and hierarchical multiscale vision transformers.
2. MORPH: A modality-agnostic model designed for heterogeneous scientific data. It combines component-wise convolutions, field-wise cross-attention, and axial attention. Pretrained on diverse datasets from PDEBench and The Well (including 1D-3D scalar/vector fields).

Experimental Protocol

• Fine-tuning Strategy: Both models were fine-tuned for terminal-state prediction (First Frame $\to$ Last Frame).
• Baselines: Performance was compared against training the same architectures from scratch (random initialization).
• Scaling Study: Experiments varied training-set sizes to quantify sample efficiency under distribution shift.
• Metrics: Mean Squared Error (MSE) on held-out test sets, evaluated at native resolution.

3. Key Results

Terminal-State Prediction Accuracy

• On PLI (Shock/Interface Dynamics): MORPH significantly outperformed POSEIDON (MSE: 0.054 vs. 0.139). MORPH's architecture appears better suited for capturing the complex interface displacements and shock-driven dynamics.
• On FRAC (Fracture Dynamics): POSEIDON achieved a modest advantage over MORPH (MSE: 0.037 vs. 0.041). Both models recovered the overall damage field, with errors concentrated near crack tips and junctions.

Sample Efficiency (Data-Level Scaling)

• Low-Data Regime: Pretraining generally provided benefits, but the magnitude varied by model and dataset.
  • PLI: Fine-tuned MORPH was superior to fine-tuned POSEIDON across all data sizes. Notably, MORPH trained from scratch outperformed fine-tuned POSEIDON, suggesting POSEIDON's fluid-centric pretraining was less transferable to PLI physics.
  • FRAC: POSEIDON showed consistent gains from pretraining in the low-data regime compared to training from scratch. Conversely, MORPH trained from scratch performed better than fine-tuned MORPH, indicating a potential negative transfer or mismatch in MORPH's pretraining for this specific fracture task.
• High-Data Regime: As training data increased, the performance gap between fine-tuning and training from scratch narrowed, indicating that sufficient in-domain data can overcome the lack of relevant pretraining.

Transfer Gain Magnitude

The "transfer gain" (ratio of loss from scratch vs. fine-tuned) peaked at only $\approx 2\times$ in the low-data regime. This is significantly lower than gains reported in previous PDE-FM literature (which often span coding or broader multitask knowledge), suggesting that fluid-centric pretraining alone is insufficient for extreme-loading mechanics.

4. Key Contributions

1. New Benchmarking Protocol: Established a rigorous evaluation framework for OOD transfer in extreme-loading material dynamics, focusing on terminal-state prediction rather than autoregressive rollout.
2. Dataset Introduction: Introduced and utilized two large-scale, discontinuity-dominated datasets (PLI and FRAC) specifically designed to test PDE surrogates on shocks, interfaces, and fractures.
3. Empirical Analysis of Foundation Models: Provided the first comparative analysis of POSEIDON and MORPH on extreme-loading physics, revealing that:
   • Model performance is highly regime-dependent (MORPH for interfaces, POSEIDON for fracture).
   • Fluid-centric pretraining offers limited transfer benefits for solid mechanics with strong discontinuities.
4. Sample Efficiency Quantification: Demonstrated that while pretraining helps in low-data regimes, the benefits diminish rapidly with more in-domain data, and the "inductive bias" of fluid models may not align with shock physics.

5. Significance and Future Directions

• Limitation of Current FMs: The study highlights that current PDE foundation models, trained primarily on smooth fluid flows, lack the necessary inductive biases to efficiently learn shock-driven and fracture-dominated dynamics.
• Need for Diverse Pretraining: To achieve substantial transfer gains, future PDE foundation models must be pretrained on diverse extreme-loading datasets that include shocks, multi-material interfaces, and fracture mechanics.
• Continual Learning: The authors suggest that continual learning strategies could be a viable pathway to integrate these new, challenging physical regimes into foundation models without requiring full retraining, allowing models to progressively adapt to new material behaviors.

In summary, this paper serves as a critical reality check for the PDE foundation model community, demonstrating that while these models show promise, their current fluid-centric training limits their direct applicability to the high-stakes, discontinuity-dominated physics of extreme material loading.