💻 computer science

Missing Physics Discovery through Fully Differentiable Finite Element-Based Machine Learning

This paper introduces FEML, a fully differentiable finite element-based machine learning framework that discovers configuration-independent constitutive laws for missing physics by embedding PDE solvers into training, enabling zero-shot transfer across geometries and boundary conditions while recovering interpretable physical models from both synthetic and real-world data.

Original authors: Ado Farsi, Nacime Bouziani, David A Ham

Published 2026-07-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ado Farsi, Nacime Bouziani, David A Ham

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to bake the perfect cake, but you only have half the recipe. You know exactly how the oven works (the heat, the timer, the pan shape), and you know the ingredients you do have. But the secret ingredient—the specific way the flour reacts to the sugar to make the cake rise just right—is a mystery. In the world of physics and engineering, scientists face this exact problem every day. They have powerful mathematical rules called "Partial Differential Equations" (PDEs) that describe how things move, heat up, or bend. These rules are like the oven instructions. However, the specific "laws" that describe how a material behaves inside that oven—like how a piece of metal stretches or how heat moves through a rock—are often missing or too complicated to write down. Without these missing laws, scientists can't predict if a bridge will hold or if a spacecraft will survive re-entry. Traditionally, they've had to guess these laws or run endless expensive experiments to figure them out.

This is where a new approach called FEML (Finite Element-Based Machine Learning) comes in, acting like a super-smart detective that can solve the mystery of the missing recipe without ever tasting the cake. Instead of just guessing the final result, FEML combines a rigorous physics simulator with a learning computer program. It treats the unknown physical law as a "black box" that it tries to figure out by watching how the whole system reacts. If the system is a bridge, FEML watches how the bridge bends under weight and works backward to discover the hidden rule that tells the steel how to stretch. The beauty of this method is that it doesn't just learn for one specific bridge; it learns the rule itself. Once it figures out how steel behaves, that rule can be applied to a completely different shape, size, or material, even if the computer has never seen that specific setup before. It's like learning the grammar of a language so well that you can instantly understand a new sentence you've never heard, rather than just memorizing a list of phrases.

The researchers behind this study, Ado Farsi, Nacime Bouziani, and David A. Ham, introduced this framework to tackle the problem of "missing physics." They built a system that couples a high-end physics solver (which calculates how structures and heat behave) directly with a machine learning model. The key innovation is that the entire system is "differentiable," meaning the computer can calculate exactly how a tiny change in the hidden rule affects the final result, allowing it to learn the rule by trial and error, guided by real-world measurements.

In their experiments, they treated the discovery process like a video game with levels of increasing difficulty. First, they tackled a simple scenario: a material that gets softer when squeezed (like a foam). They fed the system data from a simple squeeze test where they knew the force applied and the resulting movement. The system successfully learned the hidden rule describing how the material's stiffness changed, even though it never directly measured the internal stress. Next, they moved to a harder level: a material that behaves like plastic, which bends permanently after a certain point. Here, they used a "progressive discovery" strategy. They took the rule they had already learned for the elastic part and "froze" it, locking it in place. Then, they asked the system to figure out only the new part: how the material hardens as it stretches further. By breaking the problem into steps, the system learned a complex plastic-hardening law with high accuracy.

The real magic happened when they tested if these learned rules could travel. They took the two rules they had discovered (the elastic softening and the plastic hardening) and combined them into a single "foundation model." They then dropped this model into a completely new, three-dimensional simulation of a metal rod with a weird keyhole-shaped slot being twisted. This was a totally different shape and a different type of stress than what the model had been trained on. Remarkably, the model predicted the rod's behavior with almost perfect accuracy, without needing any retraining. This "zero-shot" transfer proves that the system learned the actual physics, not just a trick for one specific shape.

To show this works in the real world, not just in simulations, they applied FEML to real experimental data from a benchmark test on a titanium alloy. In this test, the material was sheared until it broke, a process involving both plastic hardening and "ductile damage" (the material getting weaker as it cracks). The system learned both the hardening rule and the damage rule directly from the noisy, messy data of the actual experiment. It successfully reproduced the entire curve of the test, including the tricky part where the material starts to soften right before it snaps, matching the real-world measurements within the natural variation of the samples.

Finally, the team showed that they could take the complex neural network they had trained and translate it back into a simple, human-readable math formula using a technique called "symbolic regression." In a heat transfer experiment, the system learned a rule for how thermal conductivity changes with temperature. When they asked the computer to find a simple equation for this, it discovered a power-law formula that was nearly identical to the true physical law, even though the computer was never told what the formula should look like.

In short, this paper demonstrates a powerful new way to discover the hidden laws of nature. By embedding machine learning directly inside physics simulations, the researchers created a tool that can learn missing physical relationships from indirect measurements, generalize those rules to new situations, and even express them as clear mathematical formulas. It suggests a future where we can uncover the secrets of complex materials and systems by letting computers learn the rules of the game directly from the data, bridging the gap between what we know and what we need to know to build safer, more efficient technologies.

Technical Summary: Missing Physics Discovery through Fully Differentiable Finite Element-Based Machine Learning

Problem Statement

Modelling complex physical systems often relies on partial differential equations (PDEs) where the governing physics are partially known, but specific relationships—such as constitutive laws, thermal properties, or damage mechanisms—remain unknown or cannot be expressed in closed mathematical form. Existing data-driven approaches face significant limitations: purely data-driven models require massive datasets and lack physical interpretability, while current physics-informed surrogate models (e.g., FNO, PINO) typically learn the solution map for a specific configuration (geometry, boundary conditions, discretisation) rather than the underlying missing operator. Consequently, these surrogates cannot be transferred to new problem settings, and they often fail to recover the hidden physical laws themselves, especially when the quantities of interest (e.g., internal stress) are not directly measurable.

The core challenge addressed is the discovery of unknown internal operators within a PDE system using indirect, experimentally accessible measurements (e.g., displacement fields or boundary forces) without requiring direct observation of the internal state variables.

Methodology: The FEML Framework

The authors introduce FEML (Finite Element-Based Machine Learning), an end-to-end differentiable framework that couples a standard Finite Element Method (FEM) solver with trainable machine learning (ML) operators.

Core Architecture

Differentiable Coupling: The framework integrates the state-of-the-art FEM solver Firedrake with deep learning libraries (PyTorch and JAX). It establishes a bidirectional differentiable link where:
- ML models are embedded as symbolic operators within the variational formulation (UFL) of the PDE.
- The FEM solver is registered as a custom differentiable operator within the ML computational graph.
- Gradients are propagated end-to-end using the adjoint method (via pyadjoint) for the PDE solve and automatic differentiation for the ML components. This allows the system to compute gradients of a loss function (defined on observable data) with respect to the ML parameters, even when the operator's output is an internal, unmeasured quantity.
Structure-Preserving Operator Networks (SPONs): To model the missing physics, FEML employs SPONs. Unlike standard neural operators that learn pointwise mappings, SPONs operate on the degrees of freedom (DoFs) of finite element spaces. They follow an encode-process-decode architecture:
- Encoder: Projects input functions onto the basis of the finite element space.
- Processor: A learnable neural network operating on the DoF vectors.
- Decoder: Reconstructs the output function in the target finite element space.
- Benefits: SPONs preserve key continuous properties (e.g., frame indifference, isotropy) at the discrete level and enable zero-shot super-resolution, allowing a model trained on one mesh resolution or geometry to be evaluated on different meshes and discretisations without retraining.
Progressive Discovery Strategy: The framework supports a staged learning approach where previously discovered laws are "frozen" and treated as known physics in subsequent experiments. This allows the system to isolate and learn increasingly complex unknowns (e.g., learning elastic response first, then plastic hardening, then damage) by reducing the ambiguity of the inverse problem.

Key Contributions and Results

1. Progressive Discovery of Elastoplastic Laws (Synthetic Data)

The authors demonstrate the framework's ability to discover constitutive laws for a closed-cell polymeric foam through a sequence of synthetic experiments:

Nonlinear Elasticity: In a displacement-controlled uniaxial test, the framework learned a nonlinear elastic softening law ( $E(\kappa)$ ) from just six noisy force-displacement pairs, accurately reproducing the softening behavior despite the lack of direct stress data.
Plastic Hardening: In a load-controlled Brazilian disc test, the learned elastic operator was frozen. The framework then learned a monotone plastic hardening law ( $\sigma_y(p)$ ) using full-field displacement data. Monotonicity was enforced by construction via an integral neural network architecture, ensuring thermodynamic consistency.
Zero-Shot Transfer: The two pretrained operators were composed into a foundation constitutive model and deployed zero-shot on a 3D cylindrical rod with a keyhole slot under torsion. The model, trained only on 2D data, accurately predicted 3D displacement and stress fields without retraining, demonstrating successful transfer across geometry, dimensionality, and loading conditions.

2. Discovery from Real Experimental Data

The framework was applied to real data from the Second Sandia Fracture Challenge (SFC2) involving a titanium alloy shear-coupon test:

Objective: Learn both a plastic-hardening law and a ductile-damage law directly from measured shear load-displacement curves, without a known ground-truth constitutive model.
Method: A two-stage process was used. First, the hardening law was learned on the rising branch. Second, the hardening operator was frozen, and a nonlocal ductile-damage operator (parameterized by a rectified integral network) was learned to capture post-peak softening.
Outcome: The resulting model reproduced the full experimental response, including the peak load and post-peak softening, within the experimental specimen-to-specimen scatter (RMSE of ~1.9% of peak force). The learned operators correctly localized plastic strain and damage into the shear band.

3. Transient Thermodynamics and Symbolic Regression

Nonlinear Thermal Conductivity: In a transient heat conduction problem, the framework learned a temperature-dependent thermal conductivity function from noisy temperature field measurements. The learned operator accurately matched the ground truth nonlinear profile.
Closed-Form Recovery: The authors applied symbolic regression (using PySR) to the trained neural operator. The algorithm successfully extracted a closed-form power-law expression ( $k \propto T^{-\alpha}$ ) that matched the ground truth structure with high accuracy ( $R^2 > 0.99$ ), demonstrating that FEML can recover interpretable physical laws from indirect data.

Significance and Claims

The paper claims that FEML represents a distinct advancement over existing methods by shifting the focus from learning configuration-specific solution surrogates to discovering reusable, configuration-independent physical operators.

Generalizability: Unlike surrogates (FNO/PINO) that are tied to specific training distributions, FEML learns the hidden operator itself, enabling zero-shot transfer to new geometries, boundary conditions, and discretisations.
Indirect Learning: The framework uniquely enables the learning of internal operators (like stress-strain relations) from indirect measurements (displacements/forces) by embedding the ML model directly within the variational PDE solver.
Interpretability: By using SPONs and subsequent symbolic regression, the framework bridges the gap between black-box neural networks and human-interpretable physical laws.
Modularity: The progressive discovery strategy allows for the incremental construction of complex physical descriptions (e.g., combining elasticity, plasticity, and damage) by reusing previously learned components.

The authors acknowledge computational costs associated with end-to-end differentiation (requiring forward and adjoint solves per epoch) but note that the ML operator overhead is negligible compared to the PDE solve. They position FEML not as a competitor to solution-level surrogates for fast inference, but as a complementary tool for physics discovery and the creation of foundation models for scientific simulation.