🤖 machine learning

LLM-PDESR: Robust PDE Discovery via Subdomain Weighted Residuals and LLM-Guided Symbolic Hypothesis Generation

LLM-PDESR is a novel framework that combines Large Language Model-guided symbolic hypothesis generation with robust subdomain weighted residuals and C⁴-continuous spline differentiation to accurately discover governing partial differential equations from noisy data, significantly outperforming existing methods in structural recovery and noise resilience.

Original authors: Jinyang Du, Hao Ma, Xiaohu Shi, Bo Yang, Yanchun Liang, Heow Pueh Lee, Chunguo Wu

Published 2026-07-14

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Jinyang Du, Hao Ma, Xiaohu Shi, Bo Yang, Yanchun Liang, Heow Pueh Lee, Chunguo Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: a set of noisy, messy clues (data) left behind by a complex physical system, like a swirling storm or a flowing river. Your goal is to find the hidden "rulebook" (the mathematical equation) that explains how that system moves.

For a long time, detectives tried to solve this by guessing random combinations of math symbols, like a monkey typing on a keyboard hoping to write a novel. This is called Symbolic Regression. But here's the problem: the "noise" in the data (like static on a radio) acts like a prankster. When the detectives tried to calculate how fast things were changing (derivatives) using old-school methods, the noise would explode, making the clues look like a distorted, impossible map. The detectives would get lost, chasing fake leads that looked promising but were actually mathematical tricks.

Enter LLM-PDESR, a new detective team that combines a super-smart AI brain with a very careful, noise-proof magnifying glass.

The New Detective Team: How It Works

Instead of just guessing random math, this team uses a Large Language Model (LLM)—a type of AI trained on tons of science books—to act as the "Idea Generator." Think of the LLM as a brilliant physicist who can look at the messy data and say, "Hmm, maybe the rule involves a curve here and a slope there," proposing a list of possible equation skeletons.

But the LLM isn't perfect; it might suggest a math structure that looks cool but fails when you try to measure it against the noisy data. That's where the second part of the team comes in: the Noise-Proof Evaluator.

The Smoothie Maker (Quintic Splines): Instead of looking at individual, jittery data points (which makes the noise scream), this method blends the data into a smooth, continuous curve using something called quintic splines. It's like taking a bumpy, rocky road and smoothing it out into a perfect highway so you can measure the speed without the car shaking apart. This allows the team to calculate derivatives (rates of change) up to the 4th order without the noise ruining the math.
The Noise Filter (Subdomain Weighted Residuals): Instead of checking if the equation is right at every single tiny point (where noise is worst), the team checks the equation over small, overlapping neighborhoods. They use a special "test function" (a smooth wave shape) to weigh the errors. This acts like a low-pass filter, naturally blocking out the high-frequency static while keeping the true signal. It's like listening to a conversation in a noisy room by focusing on the general rhythm of the speech rather than every individual crackle of the microphone.
The Pareto Coach: Once the LLM proposes an equation and the Noise-Proof Evaluator tests it, the team uses a "Pareto-driven feedback loop." Imagine a coach who tells the LLM: "That equation is accurate, but it's too complicated and full of junk. Try to find a simpler one that's almost as good." This keeps the final rulebook clean and simple, avoiding "equation bloat" (adding unnecessary terms just to fit the noise).

The Big Test: Did It Work?

The team put their new detective method through a grueling series of tests to see if it could actually find the truth.

1. The Standard Drills (23 Canonical Equations)
They tested the system on 23 standard physics equations (like the famous Burgers equation) that are well-known in science. They added different levels of noise, from 0% to 20% (which is a lot of static).

The Result: The new method found the correct equation 100% of the time when there was no noise, and stayed incredibly robust even with 20% noise.
The Competition: Other methods, like SGA-PDE, DISCOVER, and WSINDy_PDE, started to fail badly as the noise increased. They either couldn't find the equation or found the wrong one. One method, EqGPT, did okay but only because it used extra steps to clean the data first; LLM-PDESR handled the raw, noisy data directly.

2. The "Never-Before-Seen" Challenge (5 Novel Equations)
To make sure the AI wasn't just memorizing answers from its training data, the researchers created 5 brand-new, weird equations that don't exist in any textbook. These included:

Morphogenesis: A model of how patterns form in biology with a tricky "rational fraction" (a fraction with variables on top and bottom).
Traffic Flow: A model of cars getting stuck at a bottleneck, involving sharp, sudden changes.
Predator-Prey: A system where predators hunt prey based on the prey's density gradient.
The Result:
- For Morphogenesis and the Predator-Prey system, LLM-PDESR found the exact mathematical structure.
- For the Traffic Flow model, the method hit a wall. The sharp, sudden changes (shockwaves) in the data were too "stiff" for the smoothing method to handle, and it failed to find the exact rule. This shows the method has limits with extremely jagged data.
- For the other two novel equations, it found a "partial" solution that captured the main physics, even if it missed a tiny detail.
- Crucially: All the other methods failed completely on these new equations. They either got stuck in loops, produced nonsense, or couldn't handle the multiple variables at once.

3. The Real-World Mission: The Atmosphere
Finally, they took the method out of the lab and into the real world, using ERA5 climate data (a massive dataset of Earth's weather). They wanted to find a simple rule to explain how wind moves in the atmosphere.

The Discovery: The AI found a new, interpretable equation called 1D-CACE. It wasn't just a random guess; it found terms that made physical sense, like how wind in one direction pushes wind in another, and even discovered special terms that act like "closure" for missing 3D effects (like the Earth's rotation).
The Proof: They tested this new rule on data from different places and times (like moving from the North Pacific to the South Indian Ocean, or from January to February).
- When they moved to the West Pacific, the rule still worked with an accuracy (R²) of 60.97%.
- When they moved to the South Indian Ocean (a completely different hemisphere and season), it bounced back to 72.54%.
- This proves the AI didn't just memorize the specific weather of one week; it found the underlying, unchanging laws of how the atmosphere moves.

What the Paper Rules Out

The authors are very clear about what this method is not:

It is not a magic wand that solves every problem. It explicitly fails on equations with extreme, near-discontinuous shockwaves (like the Traffic Flow example), where the math gets too "stiff" for the smoothing technique.
It is not a method that relies on "hand-crafted" lists of math terms (dictionaries) like older methods. Those lists are too rigid and can't handle the weird, new shapes the AI found.
It is not a black box. Unlike deep learning models that just predict numbers without explaining why, this method outputs a clear, readable equation.

How Sure Are We?

The paper doesn't just "suggest" this works; they measured it extensively.

They ran 23 standard tests and 5 novel tests with specific noise levels.
They compared their results against four other top-tier methods using strict metrics like "Structural Success Rate" (did they get the exact equation?) and "Relative Coefficient Error" (how close were the numbers?).
The results are simulated and measured on real-world data (ERA5), but the "perfect" recovery on the novel equations is based on synthetic data generated specifically for the test.
The claim that it "significantly outperforms" state-of-the-art methods is backed by the data: in the noise tests, the new method stayed near 100% success while others dropped to near 0%.

In short, LLM-PDESR is a powerful new tool that uses a smart AI to guess the shape of the rulebook and a careful math filter to check if the guess is real, even when the data is messy. It's a huge step forward, but it still struggles when the data gets too jagged and chaotic.

Technical Summary: LLM-PDESR

Problem Statement

The discovery of governing partial differential equations (PDEs) from noisy observational data remains a fundamental challenge in scientific machine learning. Traditional symbolic regression (SR) methods, ranging from sparse regression to genetic programming, face two critical bottlenecks:

Search Space Limitations: They struggle to scale across vast combinatorial hypothesis spaces and lack the architectural flexibility to incorporate domain-specific physical priors without relying on rigid, handcrafted function libraries.
Numerical Instability: The reliance on pointwise evaluations and discrete finite difference (FD) methods for computing derivatives inherently amplifies high-frequency measurement noise. This distortion creates deceptive fitness landscapes that derail optimization processes, preventing algorithms from distinguishing true physical mechanisms from mathematical artifacts.

While Large Language Models (LLMs) have emerged as promising engines for generating physically plausible structural hypotheses, extending them to complex, time-varying PDEs is currently bottlenecked by these same numerical instabilities. Feeding deceptive fitness scores back to an LLM causes evolutionary search to fail.

Methodology: LLM-PDESR

The authors propose LLM-PDESR (Large Language Model-guided Partial Differential Equation Symbolic Regression), an end-to-end framework that integrates the structural hypothesis generation of LLMs with a mathematically rigorous, noise-robust evaluation environment. The framework operates through a closed-loop pipeline comprising three core modules:

1. LLM-Driven Hypothesis Generation

The LLM acts as an automated structural architect. Given physical observations (e.g., spatial coordinates and field variables), the LLM generates a population of candidate executable equations represented as Python Abstract Syntax Trees (ASTs).

Constraints: To prevent syntactically or physically invalid structures, the search space is strictly bounded by predefined constraints (e.g., enforcing specific derivative operators like d_dx(u, order)).
Output: The LLM generates only symbolic skeletons, utilizing parameter placeholders (e.g., params[0]) for numeric coefficients, which are later optimized numerically.

2. Robust Parameter Identification

To resolve the derivative noise bottleneck, the framework replaces standard FD methods with a continuous, noise-robust numerical pipeline:

Quintic Spline Smoothing: The noisy spatial field is reconstructed using $C^4$ continuous quintic B-splines. This allows for analytical, noise-robust spatial differentiation up to the fourth order, avoiding the $O(\sigma/(\Delta x)^k)$ noise amplification inherent in discrete FDs.
Subdomain Weighted Residuals (SWR): Instead of pointwise Mean Squared Error (MSE), the framework evaluates the PDE residual by integrating against smooth, compactly supported test functions over localized sub-domains. This formulation acts as a natural low-pass filter, attenuating high-frequency noise before error calculation.
Optimization: Parameters are optimized using the quasi-Newton BFGS algorithm to minimize the integrated residual loss, ensuring reliable convergence on a smoothed fitness landscape.

3. Pareto-Driven Evaluation and Selection

The framework models equation discovery as a multi-objective optimization problem, balancing predictive accuracy (Score: $-MSE$) and structural parsimony (Complexity: AST node count).

Feedback Loop: A Pareto frontier is maintained to track the trade-off between accuracy and complexity. The top 3 elite solutions (highest score, lowest complexity, and balanced) are serialized and fed back into the LLM prompt to guide iterative refinement.
Automated Selection: The final equation is selected based on a logarithmic error tolerance, choosing the simplest structure within an order of magnitude of the global minimum error, effectively pruning unphysical bloat.

Key Contributions

Framework Integration: The proposal of LLM-PDESR, which bridges LLM structural generation with continuous numerical optimization and evolutionary search to automate PDE discovery.
Noise Resolution: The resolution of the derivative noise bottleneck via $C^4$ continuous quintic splines and SWR evaluations. This mitigates fitness landscape distortion, while a Pareto-driven feedback loop optimally balances accuracy with parsimony.
Rigorous Benchmarking: The introduction of a comprehensive benchmark comprising 23 canonical PDEs and five structurally novel equations. These novel equations (e.g., Topography-Constrained Chemotaxis, Cross-Chemotactic Predator-Prey) are explicitly designed to preclude dataset memorization and validate genuine symbolic discovery capabilities.
Real-World Application: The successful extraction of an interpretable 1D dynamical surrogate (1D-CACE) directly from noisy ERA5 climate reanalysis data, demonstrating the framework's ability to capture invariant dynamical mechanisms in complex, real-world systems.

Experimental Results

The framework was evaluated against state-of-the-art methodologies (SGA-PDE, DISCOVER, WSINDy_PDE, and EqGPT) across three regimes:

Canonical PDEs: On 23 benchmark PDEs with varying noise levels (0% to 20%), LLM-PDESR achieved a 100% structural success rate at 0% noise and maintained consistently high robustness as noise increased. Competitors suffered severe degradation, with many failing to converge under noisy conditions.
Structurally Novel Equations: On five equations designed to be out-of-distribution (e.g., non-polynomial rational fractions, high-order nonlinearities, coupled multivariate systems), traditional baselines universally failed or produced unphysical, overfitted expressions. LLM-PDESR successfully recovered the exact mathematical skeletons for Morphogenesis and the coupled Predator-Prey system, and achieved the lowest Mean Squared Error (MSE) across all five systems, even when exact symbolic recovery was partial.
Real-World Discovery (1D-CACE): Applied to hourly wind velocity data from the ERA5 dataset, the framework extracted a coupled atmospheric circulation surrogate.
- Generalization: The discovered structural skeleton demonstrated strong out-of-distribution (OOD) generalization. When frozen and re-optimized for distinct spatiotemporal domains (e.g., West Pacific, South Indian Ocean, and different months), the model maintained high predictive fidelity ( $R^2$ ranging from 58% to 76%), confirming it captured invariant dynamical mechanisms rather than localized statistical artifacts.
- Physical Interpretation: The discovered terms, including spatially dependent closure terms ($xv, xu$), were shown to act as mathematical proxies for unresolved macroscopic background gradients (analogous to the $\beta$ -plane approximation), rather than mere overfitting.

Significance and Claims

The paper claims that LLM-PDESR significantly outperforms existing state-of-the-art methodologies in structural recovery, noise resilience, and the avoidance of spurious complexity. By integrating the reasoning capabilities of LLMs with a continuous, noise-robust evaluation environment, the framework overcomes the limitations of discrete numerical differentiation that have historically plagued PDE discovery.

The authors emphasize that the framework's ability to discover structurally novel equations and generalize across distinct spatiotemporal regimes validates its capacity for genuine symbolic discovery rather than training data memorization. The successful application to chaotic, real-world climate data suggests a pathway for extracting interpretable, reduced-order models from complex physical systems where traditional methods fail.

Limitations Acknowledged:
The authors note that the framework currently struggles with PDEs exhibiting extreme numerical stiffness or near-discontinuous shock fronts (e.g., traffic flow with spatial bottlenecks), where steep gradients disrupt the SWR integration and BFGS optimization. Additionally, the reliance on iterative LLM inference introduces computational overhead, and the current implementation is limited to 1D systems on structured grids. Future work aims to extend the framework to 2D/3D unstructured grids and optimize computational efficiency.