🤖 AI

Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?

This paper introduces SymbolBench, a comprehensive benchmark for evaluating the ability of Large Language Models to perform symbolic reasoning over real-world time series data, and proposes a unified framework that integrates LLMs with genetic programming to enhance automated scientific discovery.

Original authors: Zewen Liu, Juntong Ni, Xianfeng Tang, Max S. Y. Lau, Qi He, Wenpeng Yin, Wei Jin

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zewen Liu, Juntong Ni, Xianfeng Tang, Max S. Y. Lau, Qi He, Wenpeng Yin, Wei Jin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of looking for fingerprints or footprints, you are looking at a series of numbers that change over time—like a heart rate monitor, a stock market ticker, or the movement of planets.

The "mystery" is the hidden rule (the math equation) that created those numbers. If you can find the rule, you can predict the future.

This paper, SymbolBench, is essentially a "training camp" and a "test" designed to see if Artificial Intelligence (like ChatGPT) is smart enough to be a scientific detective.

Here is the breakdown of how they did it:

1. The Three Types of Mysteries (The Tasks)

The researchers didn't just give the AI simple math problems. They gave it three distinct "crime scenes":

The Smooth Slider (Symbolic Regression): Imagine a car accelerating. The numbers go up smoothly. The AI has to find the exact mathematical formula (like $y = x^2$ ) that describes that smooth movement.
The Light Switch (Boolean Networks): Imagine a complex circuit where things are either ON or OFF. The AI has to figure out the logic: "If Switch A is ON and Switch B is OFF, then Light C turns ON." This is like figuring out the "logic" of a biological cell.
The Domino Effect (Causal Discovery): Imagine a row of falling dominoes. The AI has to figure out which domino hit which, and how much of a delay there was between them. It’s about finding the "cause and effect" chain.

2. The "Detective's Notebook" (The Framework)

The researchers realized that just asking an AI "What is the formula?" isn't enough. They created a Closed-Loop System. Think of it like a detective who doesn't just guess once, but follows a cycle:

The Guess (Proposal): The AI makes a hypothesis (e.g., "I think the rule is $x + y$ !").
The Test (Verification): The system checks the guess against the real data to see if it actually works.
The Review (The Judge): A second "AI Judge" looks at the guess and says, "That math works, but it doesn't make sense for a human heart. Try something more biological."
The Refinement (Context): The AI takes all that feedback, looks at its notes, and tries a better guess. It repeats this until it gets it right.

3. What did they find? (The Results)

The researchers discovered some very interesting things about how "smart" these AIs actually are:

The Good News: When it comes to smooth math and finding cause-and-effect, the AI is actually quite good—sometimes even better than traditional, old-school math programs!
The Bad News: When it comes to the "Light Switch" logic (Boolean networks), the AI struggles. It’s like asking a poet to fix a computer circuit; they might be brilliant, but they aren't trained for that specific kind of logic.
The "Context" Secret: The AI performs much better when you give it "clues." If you tell the AI, "You are looking at a biological cell," it stops suggesting math formulas that only make sense in physics. Context is the AI's superpower.

The Big Picture

In the past, AI was mostly used to recognize patterns (like "this is a picture of a cat"). This paper is pushing AI toward understanding laws (like "this is the law of gravity").

If we can perfect this, we won't just have AI that can describe what is happening; we will have AI that can act as a digital scientist, helping humans discover new medicines, understand climate change, or unlock the secrets of the universe by finding the hidden "rules" written in the data.

Technical Summary: SymbolBench

SymbolBench: Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?

1. Problem Statement

The fundamental goal of scientific discovery is to uncover hidden symbolic laws (mathematical equations, logical rules, or causal structures) from observed data. While traditional methods like Genetic Programming (GP) and sparse optimization (e.g., SINDy) focus on numerical data fitting, they often struggle with interpretability and domain-specific context. Conversely, while Large Language Models (LLMs) excel at structured reasoning, their ability to perform symbolic reasoning over time series—extracting formal, interpretable laws from temporal data—remains underexplored. Existing efforts are often limited to simple algebraic equations and fail to leverage the LLM's potential for theory-aligned, context-aware reasoning.

2. Methodology

The authors propose a two-pronged approach: a new benchmark and a unified reasoning framework.

A. SymbolBench (The Benchmark):
To move beyond simple algebraic fitting, SymbolBench evaluates LLMs across three distinct symbolic tasks using real-world scientific data:

Multivariate Symbolic Regression (CDEs): Recovering coupled Ordinary Differential Equations (ODEs) from continuous data (e.g., biological cell dynamics).
Boolean Network Inference (BNs): Identifying logical update rules (AND, OR, NOT) from discrete binary state transitions (e.g., gene regulatory networks).
Causal Discovery (SCMs): Uncovering Structured Causal Models (directed graphs with time lags) to identify variable dependencies.

The benchmark is designed with real-world relevance (using datasets from physics, biology, and healthcare), varying complexity (1 to 4 dimensions), and stress tests (Out-of-Distribution and noisy data).

B. Unified Symbolic Reasoning Framework:
The authors introduce a closed-loop, iterative refinement system that integrates LLMs with traditional tools:

Proposal Generation: LLMs act as "Predictors" using various prompting strategies: Naive, Base (using history), Context (adding domain metadata), and CoT (Chain-of-Thought reasoning). Alternatively, a Hybrid Method uses GP to perform evolutionary operations (crossover/mutation) to expand the candidate pool.
Verification: A dual-verification strategy is used:
- Quantitative: Measuring numerical fitness (e.g., $R^2$ for CDEs, F1 score for BNs, CI-score for SCMs).
- Qualitative: An "LLM-as-Judge" evaluates candidates based on a rubric (Context Alignment, Scientific Plausibility, Conciseness, and Logical Coherence).
Context Management: A manager selects the top- $k$ highest-scoring candidates from a history pool to guide the next round of generation.

3. Key Contributions

SymbolBench Dataset: A comprehensive, multi-task benchmark that spans continuous, discrete, and causal symbolic forms.
Unified Framework: A hybrid system that combines the "knowledge-rich" reasoning of LLMs with the "search-efficient" nature of Genetic Programming.
Empirical Insights: A systematic evaluation of how context, reasoning depth, and test-time compute affect symbolic discovery.

4. Key Results

Task-Specific Performance: LLMs outperform traditional baselines in CDEs and SCMs, showing a superior ability to incorporate domain knowledge. However, they lag behind specialized GP methods (like LogicGep) in Boolean Network Inference, where numerical fitting is more critical than contextual reasoning.
The Power of Context: Providing domain descriptions and variable metadata (Context-aware prompting) consistently improves both numerical accuracy and the ability to find generalizable (OOD) structures.
Complexity vs. Generalization: The study challenges "Occam’s Razor." In complex scientific domains, the simplest equation is not always the most generalizable; sometimes, more structurally complex models are required to capture true underlying laws.
Test-Time Compute: Increasing reasoning depth (Long CoT) and iterative refinement (verification loops) lead to moderate but consistent improvements in solution quality.
Hybrid Synergy: Combining LLMs with GP (using LLMs to seed populations or judge fitness) significantly outperforms using either method in isolation.

5. Significance

This work shifts the paradigm of LLM application in time series from mere "pattern recognition" or "description" to "automated scientific discovery." By demonstrating that LLMs can act as reasoning agents that align mathematical models with scientific theory, the paper provides a roadmap for building AI systems capable of proposing testable hypotheses in complex fields like biology, physics, and medicine.