🤖 AI

GESR: A Genetic Programming-Based Symbolic Regression Method with Gene Editing

This paper proposes GESR, a symbolic regression method that enhances traditional Genetic Programming by employing two BERT models to intelligently guide gene mutation and crossover, thereby significantly improving computational efficiency and overall performance.

Original authors: Yanjie Li, Liping Zhang, Min Wu, Weijun Li, Lina Yu, Jingyi Liu, Yusong Deng, Mingzhu Wan, Xin Ning

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yanjie Li, Liping Zhang, Min Wu, Weijun Li, Lina Yu, Jingyi Liu, Yusong Deng, Mingzhu Wan, Xin Ning

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding the "Secret Recipe"

Imagine you are a chef trying to figure out the exact recipe for a delicious soup, but you don't have the recipe card. You only have a bowl of the soup and a list of ingredients (data). Your goal is to write down the mathematical formula (the recipe) that explains exactly how those ingredients mix to create that taste.

In the world of science and AI, this is called Symbolic Regression. The challenge is that there are billions of possible recipes (formulas). Most are terrible (too salty, no flavor), and only a few are perfect.

The Old Way: The "Random Chef" (Traditional Genetic Programming)

For a long time, computers tried to find these recipes using a method called Genetic Programming (GP). Think of this like a kitchen where a "Random Chef" is trying to cook.

The Process: The chef starts with a bunch of random recipes.
The Mutation: To improve a recipe, the chef blindly swaps an ingredient. Maybe they swap "salt" for "sugar" or "onion" for "chocolate."
The Crossover: They take two recipes and swap a whole section of one with a section of the other.
The Problem: Because the chef is completely random, they make a lot of mistakes. They might swap salt for sand. Most of these changes make the soup taste worse. The chef has to try millions of random combinations before stumbling upon a good one. It's slow, wasteful, and frustrating.

The New Idea: The "God's Hands" (GESR)

The authors of this paper asked a question: What if we had a "God" who could look at a recipe, predict exactly which ingredient swap would make it taste better, and then make that specific change?

They call this "Gene Editing." Instead of random guessing, they built a system with two "Hands of God" (which are actually two advanced AI models called BERT).

Hand #1: The Mutation Guide (The "Ingredient Fixer")

How it works: Imagine the recipe is written as a sentence: "Add salt, then boil, then add pepper."
The AI's Job: The AI looks at the sentence and covers up the word "pepper" (this is called "masking"). It then looks at the data (the soup taste) and asks, "What word fits best here to make the soup taste right?"
The Result: Instead of randomly guessing "chocolate," the AI confidently says, "Add cumin." It guides the computer to make changes that are actually likely to work, skipping the millions of bad guesses.

Hand #2: The Crossover Guide (The "Recipe Swapper")

How it works: Sometimes, you need to swap a whole paragraph of a recipe (like the "cooking method" section) between two different recipes.
The AI's Job: In the old random way, the chef would just cut the recipes in the middle of a sentence, ruining the grammar. The AI looks at two recipes and the soup data, then points to the exact spot where swapping a section would be safe and beneficial.
The Result: It ensures that when recipes are mixed, the new "offspring" recipes are still grammatically correct and likely to taste good.

The Results: Faster, Smarter, and Simpler

The paper tested this new "Gene Editing" method (called GESR) against the old "Random Chef" methods on many different math problems (like predicting how planets move or how chemicals react).

Here is what they found:

Speed: GESR found the correct formulas much faster. It didn't waste time trying bad recipes because the "Hands of God" guided it away from them.
Accuracy: It found formulas that predicted the data just as well as, or better than, the old methods.
Simplicity: The formulas GESR found were often shorter and cleaner. The old random methods tended to create "bloated" recipes with unnecessary ingredients (like adding salt, then sugar, then more salt). GESR found the elegant, simple solution.

The "God" Metaphor

The paper uses the metaphor of a "God" not in a religious sense, but as a metaphor for perfect foresight. In nature, evolution is random; mutations happen by chance, and most are bad. The authors realized that if we could use AI to act like a "God" that knows which mutations are good before they happen, we could speed up the discovery of scientific laws dramatically.

Summary

Think of the old method as a monkey typing on a keyboard hoping to write a Shakespeare play by accident. It will take forever.
The new method (GESR) is like having a smart editor sitting next to the monkey, saying, "Don't type 'banana' here; type 'the'." It guides the process, making the search for the perfect mathematical formula much faster, more efficient, and more likely to succeed.

Note: The paper focuses strictly on improving how computers find mathematical formulas from data. It does not claim to use this for medical diagnosis, clinical treatments, or specific future applications beyond the realm of scientific discovery and data analysis.

Technical Summary: GESR – A Genetic Programming–Based Symbolic Regression Method with Gene Editing

1. Problem Statement

Symbolic regression aims to automatically discover mathematical expressions from observational data that achieve both high predictive accuracy and strong interpretability. While Genetic Programming (GP) is a classical and widely adopted approach for this task, it relies heavily on random genetic operators (mutation and crossover) to evolve expression structures. The authors argue that this randomness, while mimicking natural evolution, inevitably generates a large number of ineffective or detrimental variations. This lack of controllability reduces search efficiency, slows convergence, and increases computational costs. Existing methods that attempt to guide GP (e.g., using LSTMs for initialization or backward calculation for subtree prediction) often suffer from high computational overhead or fail to effectively guide the mutation and crossover processes during the evolutionary search.

2. Methodology

The paper proposes GESR (Gene Editing Symbolic Regression), a framework that integrates multimodal BERT models into the GP evolutionary loop to perform "gene editing." This approach replaces purely random genetic operators with data-conditioned, guided operations. The framework consists of two primary components, metaphorically described as the "Hands of God":

2.1 Guided Mutation via Masked Language Modeling

Representation: Symbolic expressions are represented as binary trees and linearized into preorder traversal sequences.
Mechanism: A multimodal BERT model is trained to guide mutation. During the mutation phase, specific operator or operand tokens in an expression sequence are randomly masked.
Input: The masked sequence is fed into the BERT model alongside the regression dataset $(X, Y)$ . The data is encoded using a SetTransformer to ensure permutation invariance and fused with symbolic token embeddings.
Output: The model predicts the most suitable replacement symbol for the masked position based on the conditional probability distribution derived from the data distribution. This transforms mutation from a random perturbation into a targeted edit aimed at improving regression fitness (e.g., $R^2$ ).

2.2 Guided Crossover via Subtree Prediction

Mechanism: A second multimodal BERT model guides the crossover operation.
Input: Two candidate expression sequences are concatenated using a special [SEP] token, along with the dataset $(X, Y)$ .
Output: The model predicts the root nodes of the subtrees in both parent expressions that are most suitable for exchange.
Strategy: Instead of randomly selecting crossover points, the model identifies structural fragments that, when exchanged, are likely to improve the fitness of the offspring under the given data constraints.

2.3 Training Data Construction

The authors designed specific pipelines to construct high-quality training data for the BERT models:

For Mutation: The pipeline generates pairs of expressions where one is a "better" version of the other. It involves generating a target expression, sampling data, creating a random initial expression, applying differentiable mutation functions, optimizing parameters via BFGS, and discretizing the result. If the mutated expression improves the fit ( $R^2$ ), the original expression is masked, and the improved symbols serve as the target.
For Crossover: Training data is collected online during the GP evolution process. Pairs of parent expressions are crossed (using standard GP or guided mutation first), and the offspring are evaluated. If the offspring outperform the parents, the parent pair and the specific crossover nodes used are recorded as a training sample. This ensures the model learns crossover points that lead to fitness improvements.

2.4 Evolutionary Loop

The GESR framework iterates through generations:

Mutation: The population undergoes guided mutation using the mutation-BERT.
Crossover: Guided crossover is performed using the crossover-BERT.
Optimization: Constants in the resulting expressions are refined using BFGS.
Selection: Individuals are evaluated based on $R^2$ , and the top $N$ are selected for the next generation.

3. Key Contributions

The paper outlines four main contributions:

A Gene Editing Framework: Proposes a symbolic regression framework that enables targeted guidance of genetic mutations and crossovers, substantially improving evolutionary efficiency.
Dual "Hands of God" Models: Trains two multimodal BERT models to perform gene editing, which can serve as a plug-and-play module for other GP-based symbolic regression algorithms.
Data Construction Method: Designs a mutation function and a method for constructing high-quality gene editing training data based on fitness improvement.
Conceptual Integration: Introduces the concept of "gene editing" into the field of evolutionary computation, leveraging deep language models to guide search.

4. Experimental Results

The authors evaluated GESR on 13 datasets, including standard benchmarks (Nguyen, Keijzer, Korns, etc.) and the SRBench suite (Feynman, Strogatz, Black-box).

Fitting Performance: GESR achieved superior or comparable $R^2$ scores compared to state-of-the-art baselines (PySR, NGGP, SNIP, GPlearn). On average, GESR achieved an $R^2$ of 0.9903, outperforming PySR (0.9876) and NGGP (0.9784).
Complexity (Interpretability): GESR produced more compact expressions. The average node count for GESR was 22.0, significantly lower than PySR (26.9), NGGP (28.1), and GPlearn (35.8).
Efficiency: GESR demonstrated significantly faster inference times. The average time to reach $R^2 > 0.999$ was 117.2 seconds for GESR, compared to 176.1 seconds for the unguided GP baseline and 124.4 seconds for a variant with only mutation guidance.
Ablation Studies: Removing either the mutation or crossover guidance module resulted in degraded performance in terms of accuracy, complexity, and search time, confirming that both components contribute to the method's success.
Chaotic Systems: In tests on 16 chaotic dynamical systems (e.g., Lorenz, Duffing), GESR showed higher vector-field fitting accuracy and better short-horizon rollout consistency compared to baselines.
Transferability: When integrated into the NGGP framework, the BERT-guided gene editing strategy improved NGGP's performance, demonstrating that the approach is not limited to the specific GESR architecture.

5. Significance and Claims

The paper claims that integrating deep language models for intelligent gene editing provides a novel and efficient research direction for symbolic regression. By replacing unguided random exploration with data-driven, directed optimization, GESR:

Reduces Ineffective Search: Significantly lowers the number of detrimental mutations and crossovers.
Improves Convergence: Achieves faster convergence speeds and higher search stability.
Enhances Interpretability: Generates more compact and human-interpretable mathematical formulas.

The authors acknowledge limitations, noting that the method depends on the quality of the pretrained BERT models and faces scalability challenges when the dimensionality of test data exceeds the range supported during pretraining. They suggest future work should focus on developing lighter, more transferable models and scalable frameworks for high-dimensional inputs.

In summary, the paper posits that the "metaphorical God" of gene editing—realized through multimodal BERT models—can substantially enhance the efficiency of evolutionary symbolic regression by learning to predict beneficial structural changes based on data distribution, thereby transforming the search process from blind exploration to guided optimization.