💻 computer science

GPU-Accelerated Genetic Programming for Symbolic Regression with Beagle Framework

This paper introduces Beagle, a GPU-accelerated framework for symbolic regression that significantly outperforms leading CPU-based systems like StackGP and PySR in benchmarking on the Feynman dataset by maximizing throughput for population and fitness case processing.

Original authors: Nathan Haut, Ilya Basin, Marzieh Kianinejad, Ruchika Gupta, Elijah Smith, Zachary Perrico, Wolfgang Banzhaf

Published 2026-03-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nathan Haut, Ilya Basin, Marzieh Kianinejad, Ruchika Gupta, Elijah Smith, Zachary Perrico, Wolfgang Banzhaf

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive jigsaw puzzle, but instead of picture pieces, you have mathematical formulas. Your goal is to find the one perfect formula that explains a set of data points. This is called Symbolic Regression.

Usually, finding this formula is like searching for a needle in a haystack while blindfolded. You have to guess a formula, check if it works, and if it doesn't, you throw it away and try a new one. This process is slow and computationally expensive.

This paper introduces a new tool called Beagle that changes the game by using GPUs (Graphics Processing Units) to speed up the search. Here is a simple breakdown of what they did and why it matters.

1. The Problem: The Slow Search

Traditional methods (like StackGP and PySR) are like a single detective walking through the haystack, checking one needle at a time. They are smart, but they are slow. If you have millions of possible needles to check, it could take days or weeks.

2. The Solution: Beagle and the GPU Super-Team

The authors built a new framework called Beagle. Instead of one detective, Beagle hires an army of millions of tiny robots (thanks to the GPU) to search the haystack all at once.

The GPU Advantage: A standard computer processor (CPU) is like a very smart professor who can do complex math but can only do one thing at a time. A GPU is like a stadium full of 10,000 students who are slightly less smart individually but can all do simple math simultaneously.
The Strategy: Beagle sends millions of potential formulas to the GPU. The GPU checks them all in parallel. While the CPU (the professor) is busy organizing the results and deciding which formulas to keep, the GPU is already checking the next batch.

3. Two Ways to Judge the Formulas

To know if a formula is "good," you need a scorecard. Beagle tested two different scorecards:

The "Point-to-Point" Scorecard: This checks if the formula's answer matches the data exactly, like checking if a student's answer is "5" when the answer key says "5."
The "Correlation" Scorecard: This is a bit more sophisticated. It doesn't just look at the exact numbers; it looks at the shape of the relationship. It asks, "Does the formula go up when the data goes up, even if the numbers aren't perfect?"
- The Surprise: Even though the "Correlation" scorecard takes a tiny bit longer for the GPU to calculate, it actually guides the search much better. It's like having a better map; you might spend a second looking at the map, but you don't waste hours walking in the wrong direction.

4. Handling the "Impossible" (NaNs)

Sometimes, math problems involve numbers that don't exist (like the square root of a negative number, which creates an "imaginary" number). In many systems, this crashes the program or forces the researchers to throw away that data.

Beagle is special because it treats these "impossible" numbers as clues. If the real-world data has a gap where a number shouldn't exist, Beagle rewards formulas that also have a gap there. It's like a detective realizing that the fact a witness didn't see something is just as important as what they did see.

5. The Results: Who Won?

The researchers tested Beagle against the two best existing tools (StackGP and PySR) using a famous set of 100 physics puzzles (the Feynman dataset). They gave everyone a strict time limit: 10 minutes and 30 minutes.

The Winner: Beagle crushed the competition.
- In 10 minutes, Beagle (using the Correlation scorecard) solved 82 out of 100 puzzles.
- The next best tool (StackGP) only solved 65.
- PySR solved 64.
The Takeaway: By using the GPU's massive parallel power and a smarter way to score formulas, Beagle found solutions that the others simply couldn't reach in the same amount of time.

Why Should You Care?

Think of this as the difference between a horse-drawn carriage and a jet engine.

Before, if you wanted to solve a complex scientific problem using AI, you had to wait hours or days.
With Beagle, you can get high-quality answers in minutes.

This makes advanced AI accessible to more people. It means scientists, engineers, and even students can use these powerful tools to discover new laws of physics or understand complex data without needing a supercomputer or a PhD in computer science. It turns a "wait overnight" task into a "coffee break" task.

1. Problem Statement

Genetic Programming (GP) is a powerful evolutionary algorithm for Symbolic Regression (SR), which aims to discover mathematical expressions that describe data. However, GP is computationally expensive due to the repeated cycles of evaluation, variation, and selection required over large populations. Traditional CPU-based implementations often struggle with:

Scalability: Difficulty in maintaining large population sizes (millions of individuals) due to memory fragmentation and garbage collection overhead.
Time Constraints: The inability to provide high-quality results within short timeframes (e.g., under 30 minutes), limiting practical adoption compared to other machine learning techniques.
Hardware Underutilization: Failure to fully leverage the massive parallel processing capabilities of modern Graphics Processing Units (GPUs).

The paper addresses the need for a framework that can efficiently map GP tasks to GPU hardware to accelerate the search process, handle massive populations, and support complex fitness functions (including those dealing with invalid data points like NaNs).

2. Methodology

The Beagle Framework

The authors introduce Beagle, a new open-source GP framework designed specifically for heterogeneous computing environments (CPU + GPU). Key architectural features include:

Language & Stack: Built using C# and the ILGPU library. C# offers faster execution than Python, while ILGPU allows low-level CUDA programming without high-level abstractions, enabling custom optimization.
Linear Genetic Programming (LGP): Beagle uses a custom Genome Computer Language (GCL) based on Reverse Polish Notation (RPN). Unlike tree-based GP, GCL is not heap-based, making it highly efficient for GPU memory management and mutation operations.
Hybrid Execution Model:
- GPU: Handles the most computationally intensive tasks: evolutionary runs and fitness function evaluations. The GPU kernel maps one CUDA block per organism and one thread per fitness case, minimizing thread divergence.
- CPU: Handles selection, birth/death loops, and mutations.
- Data Exchange: To minimize CPU-GPU transfer overhead, Beagle processes individuals in large batches (e.g., 512 or 1024 fitness cases per individual) and ensures an individual is sent to the GPU only once per generation.
Memory Management: The framework eliminates garbage collection (GC) and memory fragmentation by using a "dead pool" strategy where memory from "dead" individuals is recycled rather than deallocated.
Scalable Selection: For populations in the millions, traditional sorting is a bottleneck. Beagle employs a Monte-Carlo-inspired selection approach, sorting a random sample of 100 individuals to determine fitness percentiles for the whole population, achieving ~30,000x speedup over traditional ranking.
Fitness Functions:
1. Point-to-Point (Default): Similar to RMSE, comparing model output to target values. It handles invalid numbers (NaNs) by rewarding models that produce invalid values in the same locations as the training data.
2. Correlation Fitness: Based on the correlation coefficient ( $r$ ), maximizing $r^4$ . This function is computationally heavier but theoretically more robust for finding correct structural relationships.

Benchmarking Setup

Dataset: The Feynman Symbolic Regression Benchmark, consisting of 100 real-world physics equations.
Comparators:
- StackGP: A Python-based stack-based GP system (CPU-only).
- PySR: A widely used Julia-based SR system (CPU-based, multi-threaded).
Hardware:
- Beagle: 4x Intel Xeon Platinum 8260 CPUs + 2x NVIDIA V100 GPUs.
- StackGP: 1 CPU (single-threaded).
- PySR: 4 CPUs (multi-threaded).
Constraints: Runs were limited to 10 minutes and 30 minutes to simulate practical, time-constrained scenarios.
Success Criteria: A problem is "solved" if a model achieves <0.1% error on a test set. "Typical" performance is defined as 50% of independent runs solving the problem; "Best" is at least one run solving it.

3. Key Contributions

Introduction of Beagle: A novel, open-source framework that successfully bridges GP and GPU computing, enabling population sizes in the millions.
Performance Benchmarking: Comprehensive comparison against state-of-the-art CPU systems (StackGP, PySR) on the Feynman dataset.
Correlation Fitness on GPU: Implementation and assessment of a correlation-based fitness function on GPU hardware, demonstrating that the search efficiency gains outweigh the computational cost.
Handling of Invalid Data: Demonstration of a fitness function that intelligently handles NaNs and imaginary numbers, allowing the system to explore broader domains without discarding potentially useful models.

4. Results

Performance Comparison (10-Minute Runs)

Beagle (Correlation): Solved 82 problems typically (median) and 89 at best.
Beagle (Point-to-Point): Solved 66 typically and 77 at best.
StackGP: Solved 65 typically and 82 at best.
PySR: Solved 64 typically and 78 at best.
Insight: Beagle with the correlation fitness function significantly outperformed all CPU-based systems, solving 17 more problems than PySR in the typical case.

Performance Comparison (30-Minute Runs)

Beagle (Correlation): Solved 84 typically and 91 at best.
Beagle (Point-to-Point): Solved 74 typically and 80 at best.
PySR: Solved 72 typically and 86 at best.
StackGP: Solved 70 typically and 83 at best.
Insight: Even with extended time, Beagle (Correlation) maintained the lead. The results indicate that none of the systems reached a performance plateau within 10 minutes, but Beagle's GPU acceleration allowed it to explore the search space more effectively.

Special Case: NaN Handling

The authors tested a quadratic equation problem where the domain was expanded to include imaginary numbers (converted to NaNs).

Beagle: Found a perfect solution in 10/10 runs when allowed to handle NaNs.
StackGP/PySR: Found 0/10 perfect solutions (neither supports NaN-aware fitness functions).
Conclusion: Incorporating knowledge of invalid data locations into the fitness function drastically improves search success for complex domains.

5. Significance

Paradigm Shift in GP: The paper demonstrates that GPU acceleration is not just a marginal improvement but a transformative step for Genetic Programming, enabling population sizes (millions) that were previously infeasible on CPUs.
Practical Viability: By solving a high percentage of the Feynman benchmark in under 30 minutes, Beagle makes Symbolic Regression a viable option for time-sensitive real-world applications where interpretability is required.
Algorithmic Innovation: The successful integration of correlation-based fitness and NaN-handling on GPU hardware suggests new directions for evolutionary search strategies that were previously too computationally expensive to implement.
Accessibility: As an open-source framework, Beagle lowers the barrier for researchers to utilize high-performance computing resources for evolutionary algorithms, potentially democratizing access to massive-scale GP.

In summary, the Beagle framework represents a significant leap forward in Symbolic Regression, proving that strategic GPU utilization combined with efficient memory management and advanced fitness functions can outperform leading CPU-based systems by a wide margin.