🤖 machine learning

Robust data-driven discovery of fractional differential equations via weak formulations and Pareto-based subset selection

This paper introduces Weak-Pareto, a robust data-driven framework that combines adjoint-consistent weak formulations with Pareto-based subset selection to accurately discover fractional differential equations from noisy data by mitigating noise amplification and optimizing both discrete term structures and continuous fractional orders.

Original authors: Pongpisit Thanasutives, Yoshinobu Kawahara

Published 2026-08-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Pongpisit Thanasutives, Yoshinobu Kawahara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective's Dilemma: Finding Hidden Laws in a Noisy World

Imagine you are a detective trying to figure out the rules of a game just by watching the players. You see a ball rolling, a car turning, or a virus spreading, and you want to write down the exact mathematical formula that explains why it happens. This is the dream of "data-driven discovery": letting the data tell us the laws of nature instead of guessing them first.

However, real life is messy. The data we collect is never perfect; it's full of static, glitches, and random noise, like trying to hear a whisper in a rock concert. In the world of physics, some systems don't just react to what's happening right next to them; they have "long memories" or "ghostly connections" across space and time. Scientists call these "fractional" systems. Think of it like a crowd doing a wave: the person in the back doesn't just react to the person in front; they react to the whole wave pattern.

The problem is that when you try to find the math behind these ghostly connections using noisy data, the usual tools make things worse. Trying to calculate how fast something is changing (differentiation) on a messy dataset is like trying to sharpen a blurry photo by turning up the contrast—it just makes the fuzziness explode into a giant, unusable mess. This paper tackles the big question: How can we find the hidden, ghostly rules of these complex systems without letting the noise drown out the signal?

The Solution: Weak-Pareto, the Noise-Proof Detective

The researchers, Pongpisit Thanasutives and Yoshinobu Kawahara, have built a new tool called Weak-Pareto. It's a clever combination of two ideas that work together to solve the "noisy data" problem.

1. The "Smoothie" Trick (Weak Formulation)
Usually, when scientists try to find equations, they look at the data point-by-point, like checking the speed of a car at every single second. If the data is noisy, this is a disaster because the noise gets amplified, turning a tiny glitch into a massive error.

Weak-Pareto does something different. Instead of looking at individual points, it uses a "weak formulation." Imagine you have a bucket of muddy water (the noisy data) and you want to know how much dirt is in it. Instead of trying to count every single speck of dirt (which is impossible because they are too small and moving), you pour the whole bucket through a fine sieve (a smooth test function) and weigh what comes out.

In math terms, this "sieve" is a smooth, gentle curve. By multiplying the data by this smooth curve and adding everything up (integrating), the tool transfers the "roughness" from the messy data onto the smooth curve. The result? The noise gets averaged out and smoothed down, while the true signal remains clear. It's like listening to a song through a high-quality speaker system that filters out the static, rather than trying to hear it through a broken radio.

2. The "Smart Search" (Pareto-Based Subset Selection)
Once the data is smoothed out, the tool still has to guess which math terms are actually in the equation. There are millions of possible combinations of terms and numbers. A straightforward search would try every single one, but that's like trying to find a specific needle in a haystack by checking every single piece of hay one by one.

Weak-Pareto uses a "Pareto-based subset selection." Think of this as a smart shopping list. You want the best equation, but you also want it to be simple (parsimonious). If you add too many terms, the equation becomes too complicated and starts memorizing the noise instead of the rules. The tool searches for the "sweet spot"—the simplest equation that still explains the data perfectly. It balances "Loss" (how wrong the equation is) against "Complexity" (how many terms it has) to find the perfect elbow in the curve, just like finding the best price for a phone that isn't too expensive but still has all the features you need.

The Secret Sauce: Continuous Orders
Most tools have to guess the "order" of the equation from a fixed list, like choosing between 1, 1.5, or 2. But the real world doesn't always fit on a grid. The order might be 1.634. Weak-Pareto doesn't guess from a list; it searches for the exact number, treating the order as a continuous variable. This avoids the "discretization bias" where you force a round peg into a square hole.

What They Found

The team tested Weak-Pareto on several famous physics problems, including how particles spread (advection-diffusion) and how fluids move (Burgers' equation).

It works in the noise: When they added up to 20% noise (which is a lot of static) to the data, Weak-Pareto successfully found the correct equation in 5 out of 5 attempts for both the advection-diffusion and Burgers benchmarks.
The competition failed: They compared it to the old "strong-form" method (the point-by-point approach). When noise was introduced, the strong-form method failed completely, recovering the correct equation in 0 out of 5 runs. It was so overwhelmed by the noise that it couldn't find the right answer at all.
It handles 2D and 3D: They even showed it works in two dimensions, finding different rules for different directions (anisotropic models), which is crucial for things like how heat moves through soil or how materials stretch.
Speed: On the advection-diffusion benchmark, it was not only more accurate but also ran substantially faster than a neural network baseline.

The Catch and the Confidence

The authors are very careful about what they claim. They proved mathematically that for linear parts of the equation, the noise in the "weak" method vanishes as you get more data points. However, for non-linear parts (where things get messy, like $u \times u$ ), the noise suppression is only "partial," meaning it helps a lot but doesn't eliminate the bias entirely.

They also found that while the tool is great at finding the structure of the equation (which terms are there), it sometimes struggles to get the exact numbers right if the noise is very high and the system is complex (like the Riesz reaction-diffusion benchmarks). In those tricky cases, it might find the right terms but get the fractional order slightly wrong, or hit the boundaries of its search range.

But the main takeaway is clear: by using the "smoothie" trick of weak formulations and a smart search for the simplest, most accurate model, Weak-Pareto can uncover the hidden laws of fractional physics even when the data is messy. It turns a noisy, chaotic signal into a clear, interpretable story about how the world works.

Technical Summary: Robust Data-Driven Discovery of Fractional Differential Equations via Weak Formulations and Pareto-Based Subset Selection

1. Problem Statement

The discovery of governing equations from data is a central goal in scientific machine learning. While methods like SINDy and PDE-FIND have successfully identified integer-order partial differential equations (PDEs) by selecting sparse terms from a library, extending this to Fractional Partial Differential Equations (FPDEs) presents unique challenges. FPDEs model nonlocal dynamics found in anomalous diffusion, viscoelasticity, and transport in heterogeneous media.

Two primary difficulties hinder FPDE discovery:

Noise Amplification: Fractional differentiation acts as a high-pass filter in the frequency domain, scaling Fourier modes by $|\kappa|^\beta$ . Consequently, pointwise (strong-form) evaluation of fractional derivatives on noisy data amplifies high-frequency measurement noise, rendering strong-form regression fragile as the derivative order or noise level increases.
Continuous Order Optimization: Unlike integer-order discovery, the derivative orders in FPDEs are continuous real numbers. A fixed dictionary of candidate orders must either be coarse (introducing discretization bias) or dense (creating nearly collinear columns that destabilize support selection).

Existing fractional discovery methods typically evaluate derivatives pointwise or rely on neural network reconstructions, failing to address the noise sensitivity of fractional operators or the continuous nature of the orders effectively.

2. Methodology: Weak-Pareto

The authors propose Weak-Pareto, a framework that integrates an adjoint-consistent weak formulation with a Pareto-based subset selection strategy over continuous orders.

A. Adjoint-Consistent Weak Formulation
The core innovation is the construction of a weak library where fractional operators are transferred from the measured field to smooth test functions via integration by parts (adjoint identities).

Linear Terms: For linear right-hand-side terms, the fractional derivative acts entirely on the smooth test function $\phi$ $ϕ$ . The measured field $u$ $u$ remains undifferentiated. The feature is defined as $\langle u, X^*_\beta \phi \rangle$ $⟨ u, X_{β}^{*} ϕ ⟩$ , where $X^*_\beta$ $X_{β}^{*}$ is the adjoint of the candidate operator.
- Noise Robustness: Theoretical analysis (Proposition 1) demonstrates that for linear terms, the variance of weak features vanishes as $O(h_t h_x)$ under grid refinement, whereas the variance of pointwise strong-form features diverges as $O(h_x^{-2\beta})$ .
Nonlinear Terms: For terms like $u^p X_\beta u$ , the weak form is $\langle u, X^*_\beta (u^p \phi) \rangle$ . While differentiation still acts on the measured field, the integration provides an averaging effect that partially suppresses noise.
Adjoint Consistency: The framework explicitly handles different operator definitions (Caputo, Riemann–Liouville, Riesz, directional spectral) by using the specific adjoint and boundary terms for each, ensuring the identified operator matches the declared candidate.

B. Branch-Aware Continuous-Order Pareto Search
To avoid the pitfalls of fixed dictionaries, Weak-Pareto treats fractional orders as continuous variables optimized via differential evolution.

Encoding: Candidates are encoded as tuples $(m_\alpha, \alpha, p, \beta, \xi)$ , where $m_\alpha$ denotes the temporal branch (subunit $0<\alpha<1$ , exact integer $\alpha=1$ , or superunit $1<\alpha<2$ ).
Search Strategy: The algorithm performs a bi-level optimization:
1. Global Search: Differential evolution optimizes continuous orders ( $\alpha, \beta$ ) within specific branches.
2. Local Fit: For fixed orders and powers, coefficients $\xi$ are fitted analytically using ridge regression.
Model Selection: The method sweeps through support sizes $c$ and selects the optimal model at the "elbow" of the validation-error–complexity Pareto front. This avoids overfitting and the collinearity issues of dense dictionaries.
Refinement: After selection, the model undergoes an "exact-order refit," evaluating operators directly at the continuous optimal values rather than interpolating from a precomputed grid, removing interpolation error from the final coefficients.

3. Key Contributions

The paper claims three primary contributions:

Adjoint-Consistent Weak Library: A library of fractional features where linear terms do not differentiate the noisy data. The authors provide theoretical proofs showing that the variance of these features vanishes under grid refinement, contrasting with the divergence of strong-form features.
Continuous-Order Pareto Search: A search mechanism that optimizes continuous fractional orders and selects support sizes simultaneously. This approach avoids discretization bias and the severe collinearity associated with dense fixed-order dictionaries.
Controlled Analysis and Baselines: Comprehensive experiments isolating the benefits of the weak formulation (vs. pointwise) and continuous search (vs. fixed dictionary). The study includes a novel diagnostic for the previously unreported "superunit" temporal branch ( $1 < \alpha < 2$ ).

4. Experimental Results

The method was evaluated on four benchmarks: Fractional Advection–Diffusion (FADE), two Riesz Reaction–Diffusion equations, and Fractional Burgers.

Noise Robustness: In controlled comparisons against a strong-form counterpart, Weak-Pareto recovered the correct support in 5/5 runs across all tested multiplicative noise levels for FADE and Burgers. The strong-form framework failed to recover the correct Burgers support in any noisy run and failed to recover the correct support in 4/5 runs for FADE at 5% noise. While Weak-Pareto maintained correct support recovery at higher noise levels, coefficient estimation errors for FADE began to rise significantly at 20% noise as the smaller diffusion coefficient became difficult to estimate.
Accuracy: On FADE and Burgers, Weak-Pareto achieved low errors in order and coefficient estimation even with 10% noise. The Riesz reaction–diffusion benchmarks proved more challenging due to identifiability issues, though support recovery remained high.
Superunit Branch: A semi-analytic diagnostic confirmed that Weak-Pareto could correctly identify the superunit branch ( $\alpha = 1.65$ ) under clean and mild noise (0.5%), whereas the strong-form method failed to select the correct branch even at 0.5% noise.
2D Extension: A two-dimensional extension successfully recovered distinct coordinate-dependent orders and anisotropic models under noisy conditions.
Efficiency: On the FADE benchmark, Weak-Pareto demonstrated substantially lower runtime and more consistent operator recovery compared to a neural network baseline.

5. Significance and Claims

The paper positions Weak-Pareto as the first equation-discovery framework to simultaneously achieve:

Use of weak formulations for fractional operators.
Support for fractional (non-integer) equations.
Optimization of continuous orders.
Best-subset selection (rather than sparse regression on a fixed dictionary).
Adjoint-consistent handling of fractional boundary terms.

The authors claim that the weak library is the primary driver of noise robustness, as ablation studies showed that removing the weak formulation (using pointwise features) caused the method to fail under noise, even with the same selection strategy. Furthermore, the continuous-order search is shown to be superior to dense fixed dictionaries, which suffer from collinearity and discretization bias. The work provides a rigorous theoretical foundation for why weak formulations stabilize fractional discovery and offers a practical, interpretable tool for discovering nonlocal dynamics from noisy data.