🤖 machine learning

Synthics: Synthetic Physics-like Datasets for Machine Learning

This paper presents a method for generating structurally faithful synthetic regression datasets using a Bayesian Probabilistic Context-Free Grammar and non-intrusive probing to characterize physical domains, demonstrating that models tuned on this data achieve hyperparameter selection performance comparable to tuning on real-world data.

Original authors: Jari Vepsäläinen

Published 2026-06-08✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jari Vepsäläinen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to drive a car, but you only have one single video of a car driving in perfect weather. If you try to train the robot on just that one video, it will likely crash the moment it sees rain or a pothole. In the world of machine learning, this is a common problem: we often don't have enough real-world data to teach our AI models how to handle the messy, complex world of physics and engineering.

This paper introduces a tool called Synthics (short for "Synthetic Physics") to solve this problem. Think of Synthics as a master chef who can invent new recipes that taste exactly like the classics, even though they've never tasted the original dish.

Here is how it works, broken down into simple steps:

1. The Problem: Not Enough Real Data

In engineering and physics, collecting real data is hard. It's expensive, slow, and sometimes dangerous. You can't just run a thousand experiments to see what happens when a bridge collapses. Machine learning models need lots of data to learn, but we often only have a tiny handful of real examples.

2. The Solution: Cooking Up Fake (But Realistic) Data

Instead of waiting for more real data, the author created a system to generate synthetic data. But here's the catch: if you just make up random numbers, the AI won't learn anything useful. The fake data has to look and feel like the real thing.

The author used a special library of 100 famous physics equations (from the Feynman Lectures on Physics) as a "cookbook."

3. The Secret Ingredient: The "Bayesian Grammar"

To create new equations that look like the old ones, the system uses something called a Bayesian Probabilistic Context-Free Grammar (B-PCFG).

The Analogy: Imagine a child learning to speak. If you just let them guess words randomly, they might say "Blue sky eat the moon." That's nonsense. But if you teach them the rules of grammar and how often certain words appear together, they start speaking like a native.
The Twist: The author didn't just teach the system the rules; they taught it the style of the Feynman equations. They used a mathematical trick (Bayesian smoothing) to ensure the system doesn't just copy the most common equations over and over. It learns to mix and match parts of equations to create brand new, never-before-seen formulas that still follow the same structural rules as the originals.

4. The Safety Check: The "Applicability Domain"

Just having a new equation isn't enough. You also need to feed it numbers that make sense.

The Problem: If you have an equation with a square root, you can't plug in a negative number, or the math breaks. If you have a formula for speed, you can't plug in a speed faster than light.
The Fix: Before generating data, the system does a "probing" test. It tries out random numbers to see which ones produce valid results. It creates a "safe zone" (like a fence around a playground) and only picks numbers that stay inside that fence. It also learns relationships, like "if variable A goes up, variable B must stay below a certain limit."

5. The Result: A New Dataset

The system combines the new, realistic equations with the safe, valid numbers to create a massive dataset of "fake" physics experiments.

6. Did It Work? (The Taste Test)

The author put the new data to the test in two ways:

The Math Test: They compared the structure of the new equations to the original Feynman equations. The new system (with the Bayesian "smoothing") passed all 8 structural tests, meaning the new equations looked just like the real ones. A simpler version without the smoothing only passed 2 tests, proving the special math trick was essential.
The Practical Test: They used the fake data to tune a machine learning model (a "Gradient Boosted Regressor"). They asked: "If we tune our AI using this fake data, will it pick the best settings for real-world problems?"
- The Result: The AI tuned on the Synthics data picked the 6th-best setting out of 20 options.
- The Comparison:
  - Tuning on real data: Also picked the 6th-best.
  - Tuning on random nonsense: Picked the 10th-best.
  - Tuning on pure noise: Picked the 19th-best (almost the worst).

The Bottom Line

This paper shows that you can teach a machine learning model to understand physics by feeding it synthetic data generated from a grammar that mimics real laws. It's not just random guessing; it's a structured, mathematically sound way to create training data when real data is scarce. The author calls this method Synthics, and it successfully bridges the gap between having too little data and needing to train powerful AI models.

Technical Summary: SYNTHICS – Synthetic Physics-Like Datasets for Machine Learning

Problem Statement

Machine learning models rely heavily on the quality and quantity of training data. However, many engineering and physical science applications suffer from data scarcity. Unlike large language models that leverage vast online text corpora, physical systems rarely have datasets of comparable scale. Acquiring representative real-world data is often prohibitively expensive, time-consuming, and constrained by safety or regulatory limits. While synthetic data generation offers a solution, its utility is contingent on whether the generated data faithfully reflects the structural and mathematical properties of real observations. Existing methods often fail to capture the specific algebraic structures of physical laws or fail to ensure that sampled inputs lie within physically meaningful applicability domains (e.g., avoiding singularities in denominators or square roots of negative numbers).

Methodology

The paper introduces SYNTHICS, a framework for generating synthetic regression datasets that structurally resemble physics equations derived from a given corpus. The workflow consists of two primary stages: equation generation and constrained input sampling.

1. Equation Generation via Bayesian PCFG

The core of the method is a Bayesian Probabilistic Context-Free Grammar (B-PCFG) learned from a corpus of physics equations (specifically, the 100 equations from the Feynman Lectures on Physics).

Grammar Learning: Equations are parsed into expression trees. A standard Probabilistic Context-Free Grammar (PCFG) estimates rule probabilities based on observed frequencies.
Bayesian Smoothing: To prevent the grammar from collapsing onto the most frequent rules (a risk with small corpora), a Dirichlet prior is applied. This adds pseudo-counts to rule probabilities, governed by a concentration parameter $\alpha$ .
Tree Depth Control: A "soft-forcing" mechanism, controlled by a temperature parameter $\tau$ , gradually increases the probability of terminating a branch as recursion depth increases, preventing excessively deep trees.
Optimization: The hyperparameters $\alpha$ and $\tau$ are jointly optimized using Optuna (Tree-structured Parzen Estimator) to maximize the statistical similarity between generated equations and the reference corpus.

2. Applicability Domain Characterization and Input Sampling

To ensure generated data points are physically valid, the method employs a non-intrusive probing strategy:

Domain Probing: For each generated equation, 1,000 random points are sampled from a nominal range. Points yielding finite, physically meaningful outputs are labeled "valid."
Constraint Recovery: From valid points, per-variable bounding boxes (5th–95th percentiles) and inter-variable dependency rules (e.g., $x_i < \theta \cdot x_j$ ) are automatically extracted to define the applicability domain.
Constrained Sampling: Inputs are sampled from random sub-ranges within these valid domains using a mixture of uniform and truncated normal distributions. A rejection filter ensures all samples satisfy the recovered dependency rules.

3. Validation

Structural Validation: The distribution of eight structural features (e.g., tree depth, operator counts, branching factor) of the generated equations is compared against the corpus using two-sample Kolmogorov–Smirnov (KS) tests.
Practical Validation: A downstream hyperparameter tuning task is performed. Gradient-boosted regressors (GBRs) are tuned on synthetic data (Synthics, random trees, or noise) and evaluated on real test equations to measure how well the synthetic data guides the selection of optimal hyperparameters.

Key Contributions

Constrained Input Sampling Framework: A method that combines uniform and truncated-normal sampling with expression-specific domain estimation to avoid invalid evaluations (e.g., division by zero, square roots of negatives) without requiring prior domain knowledge.
Grammar-Based Synthetic Data Generation: A framework that learns a B-PCFG from a physics corpus to generate novel expressions. Unlike random expression trees, this approach preserves structural characteristics such as operator frequencies, nesting depth, and variable interaction patterns while maintaining syntactic novelty.
Statistical and Practical Validation: The method is validated structurally using KS tests against the Feynman corpus and practically by demonstrating that models tuned on Synthics data can effectively guide hyperparameter selection for real-world tasks, outperforming baselines based on random trees and noise.

Results

Structural Fidelity: The optimized B-PCFG ( $\alpha^*=44, \tau^*=6$ ) successfully passed KS tests for all eight structural features when compared to the Feynman corpus. In contrast, a standard (unsmoothed) PCFG passed only two features. The Bayesian smoothing was identified as the critical factor in recovering the distribution of deeper, more complex trees and rare operators (e.g., trigonometric functions) present in the corpus.
Hyperparameter Tuning Performance: In the downstream task, tuning a GBR on Synthics data resulted in selecting, on average, the 6th-best configuration out of 20 when applied to real data. This performance matched the results of tuning directly on real data (which also selected the 6th-best on average) and substantially outperformed tuning on random expression trees (10th best) and pure noise (19th best).
Regret Analysis: The "regret" (performance loss compared to the real-optimal configuration) for Synthics was comparable to that of random trees. The authors attribute this to the "flat top" of the real-data performance landscape, where many configurations yield similar results, meaning even a sub-optimal rank (like 10th) can yield low regret.

Significance and Claims

The paper claims that SYNTHICS provides a practical route to generating training data for engineering domains where real measurements are scarce. By learning structural priors from a real physics corpus and enforcing physical constraints during sampling, the method produces datasets that are not only syntactically valid but structurally representative of physical laws.

The authors emphasize that the Bayesian prior is essential for structural fidelity given the limited size of typical physics corpora; without it, the grammar collapses to simple, shallow expressions. While the current work is limited to closed-form algebraic equations and validated on a single corpus, the results suggest that such structurally faithful synthetic data can serve as effective priors for meta-learning and model selection tasks, potentially enabling models to generalize to real-world tasks without direct training on real data. The authors remain modest, noting that the method does not yet handle differential equations and requires further validation on larger, more diverse corpora and advanced learning tasks.