💻 computer science

Learned variable-support priors accelerate symbolic regression of physical laws

The paper introduces DenoisedSR, a solver-agnostic front-end that learns variable-support priors to significantly accelerate and improve the accuracy of symbolic regression for discovering physical laws by filtering irrelevant variables before they reach the backend solver.

Original authors: Chenxi He, Jingqiu Chen

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chenxi He, Jingqiu Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: What is the hidden rule that governs this set of data?

In the world of physics, this rule is usually a mathematical formula (like $F=ma$). Finding this formula automatically is called Symbolic Regression. The problem is that the "search space" for these formulas is like a library with billions of books, but only a few contain the right answer. Most of the books are nonsense, and the library is getting bigger every second because we keep adding more "distractor" variables (irrelevant clues) to the mix.

Here is how the paper explains their solution, DenoisedSR, using simple analogies:

1. The Problem: Searching a Haystack with a Needle

Imagine you are looking for a specific needle in a haystack. But this isn't just a normal haystack; someone has thrown in 20 other pieces of junk (straw, plastic, old socks) that look almost exactly like the needle.

Traditional Solvers: These are like a robot that blindly picks up every single piece of junk, tries to build a house with it, and only occasionally finds the needle. It's slow, and it often gets confused by the junk.
The Paper's Approach: Instead of building a better robot to search the whole haystack, they built a super-smart metal detector (the "front-end") that runs before the robot starts.

2. The Solution: The "Metal Detector" (DenoisedSR)

The authors created a system called DenoisedSR. Think of it as a highly trained assistant who looks at the pile of clues (the data) and says:

"Hey, I'm 96% sure that only these 3 or 4 items in this pile of 20 are actually important. Ignore the rest."

This assistant doesn't try to solve the mystery itself. It just narrows the search. It tells the main detective, "Don't waste time on those 16 other items; focus only on these."

3. How It Works (The "Two-Headed" Detective)

The system uses two different ways of thinking to make this prediction:

The Statistician: Looks at the numbers. "Does this variable go up when the answer goes up? Do they move together?"
The Graph Detective: Looks at the names and relationships. "In physics, we know that 'mass' and 'gravity' usually hang out together. If I see 'mass,' I should probably keep 'gravity' too."

By combining these two, the system creates a "Support Prior." It's a list of "likely suspects" that is so accurate it almost never misses the real culprit (the true variable).

4. The Results: Faster and Smarter

When they gave this "shortlist" to the main search engines (the "backends"), the results were dramatic:

Speed: The search became up to 6 times faster. It's like cutting the size of the haystack by 85% before you even start looking.
Accuracy: The success rate of finding the exact right formula jumped from 42% to 60%.
Reliability: Even when the data was noisy (like trying to hear a whisper in a storm), the "metal detector" still found the right clues.

5. Why It's Not Just "Picking Fewer Items"

A skeptic might say, "Maybe it just works because you're looking at fewer things."
To prove this wrong, the authors did a control test. They gave the search engine a random list of the same number of items, but picked without using the "metal detector."

Result: The random list made the search engine perform worse.
Conclusion: The magic isn't just having fewer items; it's having the right items. The "metal detector" provides genuine guidance, not just a smaller pile.

6. The "Black Box" vs. The "White Box"

Some modern AI tries to guess the whole answer at once (like a black box that spits out a formula). These often get the ingredients right but mix them up (e.g., saying "Mass times Velocity equals Gravity" instead of "Force").

DenoisedSR's Strategy: It admits, "I'm good at finding the ingredients, but I'm not good at cooking the final dish." So, it hands the shortlist of ingredients to a traditional, verifiable chef (the backend solver) to cook the final recipe. This ensures the final answer is mathematically correct and human-readable.

Summary

The paper introduces a tool that acts as a smart filter for scientific discovery. It doesn't replace the search engine; it just clears the clutter. By predicting which variables are relevant with high accuracy, it allows existing tools to find physical laws faster, more often, and with less confusion, even when the data is messy or full of red herrings.

Technical Summary: Learned Variable-Support Priors for Symbolic Regression

Problem Statement
Symbolic regression (SR) aims to discover interpretable physical laws by searching the space of mathematical expressions to find a compact formula that fits observed data. While SR offers advantages over black-box predictors—such as dimensional consistency and extrapolation capability—it faces a fundamental combinatorial obstacle. The number of candidate expression trees grows exponentially with the number of input variables and available operators. In physical applications, governing laws typically utilize only a small subset of measured variables and a limited set of operators, yet standard solvers must explore a vast search space containing many irrelevant branches. This inefficiency leads to poor scaling, especially when datasets contain numerous "distractor" columns (irrelevant variables), causing solvers to either fail to find the true law or converge on spurious, overfitted expressions.

Methodology: DenoisedSR
The authors propose DenoisedSR, a solver-agnostic, learned front-end designed to accelerate SR by predicting a "support prior"—a subset of input variables (and optionally operators) likely to participate in the governing law. Rather than attempting to decode the full equation directly (which often leads to structural errors), DenoisedSR narrows the search space before the backend solver begins.

The system architecture consists of:

Ensemble Predictor: A combination of a Graph-Attention Network (GAT) and a Random Forest (RF) model.
- The GAT operates on a graph where nodes represent input columns. Node features include statistical moments, correlation with the target, monotonicity, and dimensional descriptors. Edges encode a co-occurrence prior learned from a knowledge graph of physical laws, allowing the model to reason about which variables jointly support a plausible law.
- The RF scores columns based on per-column statistics.
- The final score is a weighted sum: $s(col) = RF(col) + \sigma(GAT(col))$ .
Thresholding: Variables are retained if their score exceeds a low threshold ( $\tau = 0.10$ ), prioritizing high recall to ensure the true law remains reachable.
Operator Prior (Optional): An analogous GAT model predicts a subset of relevant operators based on physical variable names (e.g., associating $\theta$ with trigonometric functions).
Integration: The predicted support set is passed to a conventional SR backend (e.g., PySR, gplearn, or GPU-based enumeration), which performs the final search over the reduced variable and operator set.

Key Contributions

Solver-Agnostic Acceleration: The method acts as a pre-processing layer that improves any tree-search or enumeration backend without modifying the solver's internal logic.
High-Recall Filtering: The model achieves a recall of $\ge 0.96$ on external physics benchmarks, effectively ensuring that true variables are rarely dropped, while significantly reducing the number of candidate columns.
Distinction from Dimensionality Reduction: The authors demonstrate that the performance gain stems from learned search guidance rather than mere dimensionality reduction, validated by controls using random support sets of the same size.

Results
The paper evaluates DenoisedSR across four external benchmarks (AI-Feynman, Strogatz, Nguyen, and SRSD-Feynman) and three structurally distinct backends (PySR, gplearn, and PSE).

Variable Recovery: On the AI-Feynman benchmark (118 equations), the ensemble achieves perfect recall (1.000), recovering all true variables. It outperforms five classical feature selectors (Pearson/Spearman correlation, Mutual Information, Random Forest Importance, Lasso), which miss true variables in 30–80% of cases even with oracle knowledge of the variable count. The model remains robust to label noise up to 30% relative Gaussian noise.
Backend Performance (PySR): Under a fixed 10-second budget, supplying the variable prior to PySR increases the exact-law recovery rate from $42 \pm 13\%$ to a seed-stable $60 \pm 0\%$ . The mean held-out $R^2$ improves from $0.906$ to $0.980$. The candidate column set is reduced by 85% (from ~23 to ~3.4 columns), and the search vocabulary is halved.
Scalability with Distractors: The performance gap between guided and unguided search widens monotonically as the number of distractor columns increases (from 5 to 30). While full PySR recovery drops significantly with more distractors, DenoisedSR maintains a stable recovery rate of ~61–62%.
Time-to-Solution: By pruning irrelevant branches, DenoisedSR accelerates time-to-solution by up to 6 $\times$ for non-trivial formulas, with a median speedup of ~1.5 $\times$ . Front-end inference adds only ~40 ms per task.
Generalization to Other Backends:
- gplearn (Classical GP): Exact recovery improves from 3.3% to 17.8%.
- PSE (GPU Parallel Enumeration): Exact recovery jumps from 0% to 40%, as the prior ensures the sampler draws informative subsets of variables.
Black-Box Data: On 24 PMLB/SRBench black-box tasks, DenoisedSR improves mean held-out $R^2$ significantly compared to full search, whereas a random-support control degrades performance, confirming the value of observation-conditioned guidance.
Limitations in Specific Regimes: On the SRSD-Feynman benchmark, which contains few physically plausible dummy variables, the prior is largely neutral. This confirms the method is most effective when the ratio of distractor-to-true variables is high.

Significance and Claims
The paper claims that learned support priors represent a "high-leverage, low-risk intervention" for symbolic regression. By decoupling the difficult task of variable selection from the equation discovery process, DenoisedSR allows existing solvers to operate more efficiently and reliably. The authors emphasize that while the neural front-end cannot yet reliably assign variables to specific positions in an expression (the "assignment gap"), it successfully identifies which quantities matter. This approach complements existing physics-aware priors and dimensional analysis, offering a practical path to faster and more robust discovery of physical laws without requiring end-to-end neural equation generation. The method is presented as immediately useful for enhancing current solvers, with a failure mode that is empirically safe (high recall ensures the true law is not excluded).