🤖 machine learning

LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search

The paper introduces LLM-ACES, a closed-loop framework that leverages large language models to guide adaptive data acquisition and symbolic hypothesis construction, significantly outperforming existing methods in accurately and efficiently recovering governing ordinary differential equations from limited, noisy data.

Original authors: Nikhil Abhyankar, Sha Li, Sanchit Kabra, Naren Ramakrishnan, Yulia Gel, Chandan K. Reddy

Published 2026-06-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikhil Abhyankar, Sha Li, Sanchit Kabra, Naren Ramakrishnan, Yulia Gel, Chandan K. Reddy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the secret rules of a mysterious machine. You can watch the machine's gears turn and see how they move, but you don't know the mathematical formula that makes them work. This is the challenge scientists face when trying to discover the "governing equations" of natural systems, like how a virus spreads or how a planet orbits.

The paper introduces a new method called LLM-ACES (LLM-guided Active Closed-loop Equation Search). Think of it as a super-smart detective team that doesn't just watch the machine; it actively asks questions to figure out the rules faster and more accurately than anyone else.

Here is how it works, broken down into simple steps:

1. The Problem: The "Blind Spot"

Usually, scientists try to guess the rules by looking at a fixed set of data (like a video recording of the machine). The problem is that if you only watch the machine for a short time or in one specific spot, many different rulebooks could explain what you see.

Analogy: Imagine watching a car drive in a straight line. You might guess the driver is just pressing the gas pedal. But if you only saw that one straight line, you couldn't tell if the car also had a steering wheel or brakes. You need to see the car turn or stop to know the full rules.
The Issue: Old methods often get stuck guessing the wrong rules because they only look at the data they already have. They might find a formula that fits the video perfectly but fails the moment the conditions change.

2. The Solution: The "Smart Detective" (LLM-ACES)

LLM-ACES changes the game by using a Large Language Model (LLM)—a type of AI that knows a lot about math and science—as a guide. Instead of just guessing the final answer, the AI helps design the search for the answer.

The process happens in a loop, like a game of "Hot and Cold":

Step A: The AI Sets the Rules of the Game
The AI looks at the problem and suggests a "menu" of possible mathematical ingredients (like addition, multiplication, sine waves, or exponentials) that are likely to be part of the solution. It doesn't write the final equation yet; it just narrows down the search area to make it smarter.
- Metaphor: Instead of searching for a needle in a haystack, the AI tells you, "The needle is probably in this specific pile of hay, and it's likely made of steel."
Step B: The Team Makes Guesses
The system generates several different candidate equations based on that menu. Some might be simple, others complex.
- Metaphor: The team writes down 5 different theories about how the car works.
Step C: The "Disagreement" Test (The Secret Sauce)
This is the most important part. The system looks at all 5 theories and asks: "Where do these theories disagree the most?"
- If Theory A says the car will go fast and Theory B says it will stop, the system knows that this specific moment is the best place to test the car.
- Analogy: If your friends are arguing about whether a bridge can hold a truck or a motorcycle, the smartest thing to do is drive a heavy truck across it. That test will immediately tell you who is right.
- The system then asks the "Oracle" (a simulator or real experiment) to run the machine under those specific conditions to get new data.
Step D: The Feedback Loop
The new data is fed back to the team. The theories that were wrong get thrown out or fixed. The theories that were right get stronger. The AI then updates its "menu" of ingredients for the next round.
- Result: The team gets smarter with every single test, quickly narrowing down to the one true set of rules.

3. Why It's Better

The paper tested this method on 122 different mathematical systems (like weather models or chemical reactions) and compared it to the best existing methods.

Accuracy: LLM-ACES found the correct mathematical formulas much more often than the others. While other methods might get the numbers right for the specific data they saw, they often got the structure of the formula wrong. LLM-ACES got the structure right.
Efficiency: It needed 10 times less data to find the correct answer. It didn't need to watch the machine for hours; it knew exactly which short clip to watch to solve the mystery.
Robustness: Even when the data was noisy (like a shaky video), LLM-ACES could still find the true rules, while other methods got confused and invented fake rules to fit the noise.

Summary

In short, LLM-ACES is a system that uses AI to not just guess the laws of nature, but to design the experiments needed to prove them. It treats scientific discovery as a conversation: it makes a guess, asks a specific question to test that guess, learns from the answer, and repeats. This allows it to find the true "secret rules" of complex systems faster, with less data, and with greater accuracy than previous methods.

Technical Summary: LLM-ACES

Problem Statement

Recovering governing Ordinary Differential Equations (ODEs) from data is a fundamental challenge in modeling dynamical systems. Existing approaches typically frame this as a static inference problem over fixed datasets, assuming observed trajectories are sufficiently informative. However, dynamical systems often evolve over large state spaces where limited data can render structurally distinct equations observationally indistinguishable. This leads to "identifiability gaps," where a candidate equation fits the observed data well but predicts incorrect behavior under new initial conditions or longer time horizons. Current methods fail to address the full closed-loop nature of discovery: they either search for equations on static data (passive) or acquire data without using structured symbolic hypotheses to guide experimentation.

Methodology: LLM-ACES

The authors propose LLM-ACES (LLM-guided Active Closed-loop Equation Search), a framework that jointly optimizes symbolic hypothesis construction and adaptive data acquisition. The core insight is that the symbolic hypothesis space and the acquired dataset should co-evolve through iterative feedback.

1. LLM-Guided Hypothesis Generation

Instead of asking Large Language Models (LLMs) to directly output final equations, LLM-ACES uses an LLM ( $\pi_\theta$ ) to generate operator priors.

Process: At each iteration, the LLM receives task metadata, available operator vocabularies, and an experience buffer containing feedback from previous iterations.
Output: The LLM generates a set of $K$ operator priors (constraints on unary and binary operators, e.g., $\sin, \cos, +, \times$ ). These priors define constrained symbolic subspaces within the larger hypothesis space.
Strategy: One prior exploits high-performing operator patterns from the experience buffer (exploitation), while others explore structurally distinct compositions (exploration).
Fitting: A symbolic regression backend (e.g., PySR) fits candidate equations within these constrained subspaces to minimize prediction error and complexity.

2. Hypothesis-Driven Data Acquisition

The framework employs a predictive-divergence-driven acquisition strategy to resolve identifiability gaps.

Divergence Metric: The system calculates the average pairwise predictive disagreement (Normalized Mean Squared Error) between rollouts of the current candidate population over a prediction horizon.
Selection: New initial conditions are selected by maximizing this divergence. The goal is to query regions of the state space where competing dynamical hypotheses disagree most significantly.
Feedback Loop: The newly acquired trajectories are added to the dataset. All candidates are re-evaluated, and the experience buffer is updated with high-performing (reinforcing effective operators) and low-performing (discouraging spurious structures) candidates. This refines the hypothesis space for the next iteration.

3. Implementation

The framework is instantiated with LLM backbones (GPT-4o-mini and Qwen-3-32B) and a SciPy-based ODE solver for the oracle. It operates in a closed loop where the LLM guides the search space, the regression backend fits equations, and the disagreement among candidates drives the acquisition of new data.

Key Contributions

Closed-Loop Equation Discovery: A unified framework coupling LLM-guided symbolic hypothesis construction with adaptive trajectory acquisition, enabling the hypothesis space and dataset to co-evolve.
LLM-Induced Symbolic Search Spaces: Utilizing LLMs to construct domain-informed operator priors that constrain symbolic regression, decoupling hypothesis-space design from equation fitting.
Divergence-Driven Data Acquisition: A strategy that selects initial conditions by maximizing predictive divergence among candidates, directly targeting regions where competing hypotheses are hardest to distinguish.
Empirical Improvements: Demonstrated superior performance in reconstruction, generalization, and out-of-distribution settings compared to passive and active baselines.

Experimental Results

The method was evaluated on 122 ODE systems spanning ODEBench (63 systems) and ODEBase (59 systems).

Performance Metrics: The paper reports Normalized Mean Squared Error (NMSE), symbolic accuracy, and expression complexity.
Accuracy: LLM-ACES achieved the lowest median NMSE across all settings (reconstruction, generalization, and out-of-distribution), outperforming state-of-the-art baselines (including SINDy, PySR, ODEFormer, LLM-ODE, and active methods like Bayesian Optimization) by several orders of magnitude.
- ODEBench: Median NMSE of $1.33 \times 10^{-17}$ (Reconstruction) with GPT-4o-mini.
- ODEBase: Median NMSE of $3.70 \times 10^{-15}$ (Reconstruction) with Qwen-3-32B.
Symbolic Accuracy: LLM-ACES achieved the highest symbolic accuracy, recovering the correct functional form in 46.2% of ODEBench cases (GPT-4o-mini) and 52.4% of ODEBase cases (Qwen-3-32B).
Sample Efficiency: LLM-ACES outperformed baselines even when competing methods were provided with significantly larger data budgets (up to 10x more samples). It achieved better performance with only one-tenth the data required by other methods.
Robustness: The feedback-driven acquisition made the method robust to noise and prevented the recovery of spurious terms that fit data locally but fail to capture true governing relationships.

Significance and Claims

The paper claims that LLM-ACES addresses a fundamental limitation in current scientific discovery: the separation of equation search from data collection. By treating discovery as an active inference process where hypotheses guide data acquisition, the framework resolves identifiability gaps that static methods cannot overcome.

Scientific Impact: The results suggest that equation discovery should be viewed not as a static regression problem, but as an iterative process where hypotheses and observations continually inform one another.
Role of LLMs: The paper emphasizes that LLMs are most effective not as direct equation generators, but as tools for inducing structured search spaces and reasoning about operator compositions based on data-driven feedback.
Limitations: The authors note that the current framework focuses on autonomous ODE systems and relies on access to a simulator or experimental oracle. It may require modification for PDEs, stochastic dynamics, or settings where data acquisition is expensive or safety-constrained. Additionally, performance is sensitive to the choice of LLM backbone and prompt engineering.

The work concludes that integrating foundation models with structured priors and adaptive experimentation enables more interpretable, data-efficient, and reliable scientific discovery.