💻 computer science

A Search‑Free Foundation Model for Symbolic Regression of Physical Laws

The paper introduces PISR, a search-free foundation model that leverages a physics-informed, contrastively-aligned formula-data manifold and flow matching to rapidly and accurately recover interpretable physical laws from data, outperforming existing methods in both speed and structural accuracy across diverse scientific and real-world benchmarks.

Original authors: Chenxi He, Jingqiu Chen

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chenxi He, Jingqiu Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of fingerprints or footprints, your clues are numbers. In the world of science, there is a special kind of puzzle called "symbolic regression." The goal isn't just to predict what happens next (like a weather app guessing rain), but to find the rule that explains why it happens. Think of it as trying to find the secret recipe for a cake just by tasting the crumbs. You want to discover the exact mathematical formula—like $E=mc^2$ or the laws of gravity—that nature is using.

For a long time, computers have been terrible at this. They are great at finding patterns in data, but they usually give you a "black box" answer: a complex, unreadable machine that works but doesn't tell you the story. Other methods try to guess the recipe by throwing thousands of random combinations at the wall to see what sticks, but this is slow, clumsy, and often breaks when the data is messy. The big question scientists have been asking is: Can we build a computer that doesn't just guess randomly, but actually understands the shape of scientific laws so it can find them quickly and correctly, even in noisy, real-world data?

This is where a new invention called PISR (Physics-Informed Symbolic Regression) comes in. Think of PISR not as a detective frantically searching through a library, but as a master chef who has memorized the entire history of cooking. The researchers behind PISR realized that real scientific laws aren't just random math; they live in a specific, organized neighborhood within the vast universe of all possible equations. They call this neighborhood the "formula manifold."

Instead of searching for the answer every time, PISR learns the "map" of this neighborhood. It uses a clever trick: it trains on a massive collection of known physics and math laws, learning to recognize the "vibe" or "shape" of a real law. When you give it a new set of messy data (like the chaotic movement of wind or the orbit of a distant star), PISR doesn't start from scratch. It instantly projects that data onto its internal map and asks, "What does this look like?" It then uses two superpowers to find the answer:

Flow Matching: It gently "flows" the data point toward the nearest known law on the map, like a river finding its way to the ocean.
Retrieval: It quickly grabs the closest neighbors from a library of 400,000 real formulas it has already studied.

The results are impressive. In tests, PISR found the correct mathematical structure for physical laws 51.3% of the time on a tough set of real-world physics problems, beating all other methods. It was also about 2.4 times faster than the best competitors. But the real magic happened when the researchers tested it on things it had never seen before. PISR successfully recovered famous laws from raw data in fields as different as finance, biology, and astronomy, including complex turbulence patterns and the relationship between a star's brightness and its temperature.

Crucially, the paper shows that PISR isn't just memorizing answers. The researchers proved that in the computer's "brain" (its mathematical space), real scientific laws are clearly separated from random, nonsense math. It's like having a room where all the real laws sit in a neat, organized circle, while random gibberish is scattered far away. Because PISR knows this circle exists, it can ignore the gibberish and focus only on the meaningful patterns.

However, this isn't a magic wand that solves everything. The paper is careful to note that PISR is designed for "closed-form" laws—equations you can write down on a piece of paper. It doesn't currently handle equations that involve changing rates over time (like differential equations) in the same way. Also, while it excels at finding laws that look like scientific formulas, it struggles with abstract math problems that don't follow the rules of physics, because it has learned to ignore those "off-manifold" structures.

In short, PISR represents a shift from "brute force searching" to "smart pattern recognition." It suggests that if we teach computers to understand the underlying structure of how the universe writes its rules, we can recover those rules from messy data much faster and more accurately than before. The researchers have shown that scientific laws occupy a special, structured corner of mathematical space, and by learning to navigate that corner, we can rediscover the secrets of the universe with a single, swift glance.

Technical Summary: A Search-Free Foundation Model for Symbolic Regression of Physical Laws

1. Problem Statement

Symbolic Regression (SR) aims to recover interpretable, closed-form mathematical laws from observational data. While central to scientific discovery, existing methods face a fundamental trade-off between accuracy and speed:

Search-based methods (e.g., Genetic Algorithms, GPU-parallel enumeration) are accurate on clean, low-dimensional data but suffer from high computational costs, brittleness on noisy real-world data, and the need to re-explore the combinatorial expression space for every new problem.
End-to-end deep learning methods (e.g., transformers) offer speed but typically treat SR as a sequence translation task. They learn to model the data distribution rather than the underlying space of formulas, lacking explicit priors on the structure of physical laws. Consequently, they struggle with the discrete, non-smooth, and super-exponentially growing space of expressions required for real physical discovery.

The paper posits that current data-driven algorithms fail to reason about the structured relations of real physics because they do not encode a model of the "space of formulas" they predict.

2. Methodology: PISR (Physics-Informed Symbolic Regression)

The authors propose PISR, a search-free foundation model that reframes symbolic regression as a generation problem within a pre-trained, contrastively-aligned formula–data manifold. The core hypothesis is that real physical laws occupy a structured, low-complexity sub-manifold within the vast space of all possible expressions, shaped by dimensional consistency and recurring compositional motifs.

Architecture and Training

Shared Latent Manifold: PISR employs two encoders aligned via a symmetric InfoNCE contrastive objective (CLIP-style):
1. Data Encoder ( $E_d$ ): A permutation-invariant encoder (pooled multi-head attention with 16 learned seed tokens) that maps point clouds $\{(x_i, y_i)\}$ to a latent vector $z_d$ .
2. Formula Encoder ( $E_f$ ): A graph neural encoder that maps the abstract syntax tree (AST) of an expression to a latent vector $z_f$ .
- Alignment: During training, a formula and the data it generates are pulled to nearby points in the latent space ( $\mathbb{R}^{640}$ ), making proximity a proxy for functional relationship.
Structural Supervision: The model is trained on a large physics/mathematics corpus (33.6M expressions) augmented with a physics knowledge graph. A structural loss term ( $L_{struct}$ ) enforces a physics-informed prior over expression structure, utilizing parent laws, preconditions, and derivation-DAG edges.
Model Scale: The full model comprises approximately 0.4 billion parameters, including the encoders, a flow-matching field, and a decoder.

Inference Pipeline (Search-Free)

PISR operates in a single amortized forward pass without per-instance symbolic search:

Latent Generation: A flow-matching field (DiT-style) transports the data latent $z_d$ toward candidate formula latents.
Retrieval: A library of 400,000 real-formula latents is indexed. The $k$ -nearest neighbors of $z_d$ are retrieved to provide a non-parametric complement to the generated candidates.
Conditioned Decoding: An autoregressive decoder renders both the flow-generated and retrieved latents into expression skeletons (typed tokens), conditioned on the data representation.
Selection: The free constants of candidate expressions are fitted via nonlinear least squares. Candidates are ranked solely by held-out predictive error (no complexity penalty is applied, as physical laws can be structurally complex). The top-performing expression is returned.

3. Key Contributions and Findings

The Formula-Manifold Hypothesis

The paper provides empirical evidence that real physical formulas are linearly separable from random expressions in the learned latent space.

Separability: A linear probe achieves an ROC-AUC of 94.4% in distinguishing real physical laws from structure-matched random expressions (controlling for operator frequency and length).
Geometry: Real laws form a rich, extended structure (participation ratio 21.5) distinct from random expressions (10.5), confirming that the "law region" is defined by compositional grammar rather than merely being a smaller subset.
Dimensional Analysis: This separability is grounded in the Buckingham- $\pi$ theorem, which restricts valid laws to dimensionless groups, effectively pruning the vast majority of syntactically valid but physically impossible expressions.

Benchmark Performance

PISR was evaluated on three benchmarks against four baselines: Genetic Programming (PySR), GPU-parallel enumeration (PSE), a domain-specialized transformer (PhyE2E), and latent generative SR (GenSR).

Real-Physics Benchmark (797 tasks): PISR achieved 51.32% structural accuracy, outperforming the next best model (PySR) by 7.66 percentage points.
Cross-Domain Generalization (270 tasks): PISR achieved 71.11% accuracy, exceeding PySR by 15.93 points. This suggests the model learned the structure of law-like formulas rather than just the physics distribution, enabling transfer to finance, economics, and biology.
Feynman Equations (119 tasks): PISR reached 39.50% accuracy, leading the GPU-enumeration baseline by 6.73 points.
Efficiency: PISR is approximately 2.4× faster than genetic programming and 2.5× faster than GPU enumeration, requiring only a single forward pass (~12.25s per task) compared to the iterative search of baselines.

Recovery of Complex Laws

PISR uniquely recovered named laws from raw data where all baselines failed, particularly in cases involving:

Transcendental functions: Correctly identifying exponential or logarithmic forms where baselines substituted algebraic surrogates (e.g., nested powers or rational approximations).
Variable exponents: Recovering forms with variable exponents (e.g., $x^n$ ) where transformers substituted fixed exponents.
Specific Domains: Successfully recovering the Kolmogorov two-thirds law from turbulence data, the Stefan–Boltzmann relation from stellar catalogs, and the Reynolds-stress closure (achieving $R^2 \approx 0.99$ where classical models failed).

4. Significance and Claims

The paper claims that PISR represents a paradigm shift from "searching" for laws to "generating" them within a learned, physics-informed manifold.

Inductive Bias: By specializing in the manifold of law-like formulas, the model achieves state-of-the-art accuracy on real scientific data while avoiding the "structure-free" region of abstract expression spaces where it performs poorly (a feature, not a bug, of its design).
Generalization: The model demonstrates that a single physics-trained foundation model can generalize to non-physics scientific domains (e.g., economics, biology) because these domains share the same "compositional grammar" of scientific laws.
Scalability: The search-free architecture makes it computationally feasible to attempt large-scale hypothesis generation and validation for open closure problems (e.g., turbulence modeling), moving the field from recovery of known laws to the proposal of new ones.

The authors emphasize that the model's success relies on the hypothesis that scientific laws are not arbitrary elements of the expression space but occupy a structured sub-manifold defined by dimensional consistency and physical motifs. This prior is decisive for handling the noisy, dimensionally-structured data characteristic of real-world science.