🤖 machine learning

What Should a Large Language Model See? Physical Invariants as a Data Representation for PDE Discovery

This paper introduces "data interpretation," a training-free method that converts raw spatiotemporal fields into theorist-relevant physical invariants as direct inputs for large language models, thereby nearly tripling the accuracy of automated partial differential equation discovery compared to using raw data.

Original authors: Fan Yang, Matt Thomson

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fan Yang, Matt Thomson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of molecular science, researchers are constantly trying to understand how tiny, invisible interactions between molecules create the large-scale behaviors we can see. Imagine watching a drop of ink spread through water or a pattern of spots appear on a leopard's skin; these are macroscopic events driven by complex chemical rules. For decades, scientists have relied on human experts to write down the mathematical laws that govern these processes. This is a slow, labor-intensive task that often takes months or even years of specialized work. Meanwhile, modern experiments have become incredibly fast, capable of generating thousands of different chemical systems in a single day. The gap between the speed of data collection and the speed of human theory-building has become a major bottleneck. To bridge this, scientists have turned to artificial intelligence, hoping to automate the discovery of these governing laws. The challenge, however, is that the data these systems produce is not in a format a computer can easily "read" like a sentence. It is a vast, shifting field of values that changes over space and time, and feeding this raw information directly into an AI is often too expensive and confusing for the model to make sense of.

A new study from researchers at the California Institute of Technology proposes a clever solution to this problem. Instead of forcing the artificial intelligence to stare at the raw, overwhelming data, the team introduced a step called "data interpretation." Think of this as a translator that converts the complex, shifting field of data into a short, clear summary of physical facts—exactly the kind of notes a human expert would write down before trying to solve the puzzle. In their experiments, the researchers simulated various physical fields, such as waves or chemical reactions, and then tested whether an AI could figure out the underlying equations that created them. They compared two approaches: one where the AI was shown the raw data, and another where the AI was shown the interpreted summary. The results were striking. When the AI received the interpreted summary, it was nearly three times more accurate at recovering the correct equations than when it was shown the raw data. This improvement happened without any extra training for the AI and at a very low cost, proving that giving the machine a "theorist's perspective" is far more effective than giving it a raw data dump.

The core of this discovery lies in how the researchers transformed the data. Rather than presenting the AI with thousands of numbers representing the state of the field at every point, they first analyzed the data to extract specific physical quantities. They asked the data four simple questions: Is the system changing in a simple, smooth way, or is it oscillating like a wave? How fast are the changes happening? Is the system behaving in a straight line, or are different parts of it interacting in complex, non-linear ways? And is there a directional flow, like a current carrying something along? These questions produced a compact set of measurements that took up very little space and could be read by the AI in a fraction of a second. This process mimics how a human scientist approaches a new phenomenon: they do not memorize every single data point but instead look for the salient structures, such as shock waves or repeating patterns, that hint at the underlying rules. By feeding these insights directly to the AI, the researchers allowed the model to reason about the physics rather than just crunching numbers.

The team tested this method using a library of eight common mathematical terms that describe transport, growth, and reaction in chemical systems. They created forty-four different simulated scenarios, ranging from simple linear systems to complex ones with non-linear interactions. In the most straightforward cases, where the data was clean and the rules were simple, the AI achieved perfect accuracy when given the interpreted summary. Even in the more difficult cases involving complex, non-linear interactions, the AI significantly outperformed its attempts to guess based on raw data. The study also included a control group where the researchers shuffled the summaries, giving the AI a description of one system while asking it to solve another. In these cases, the AI's performance collapsed, dropping to the level of random guessing. This confirmed that the AI was not just memorizing common patterns but was genuinely using the provided physical measurements to construct its answers. The researchers noted that the AI struggled most when the data interpretation could identify that a complex interaction existed but could not specify exactly what form it took, highlighting that the quality of the summary is directly linked to the quality of the final answer.

This approach offers a practical path forward for automating scientific discovery. The process does not require the AI to be retrained on new data, nor does it need massive computational power. The interpretation step is fast, taking only a small fraction of a second per field, and the resulting summary is much smaller than the raw data it replaces. By separating the task of understanding the data from the task of proposing the equation, the researchers have created a pipeline where an AI can co-evolve with experimentation. As laboratories generate new data from thousands of molecular systems, this method could automatically produce candidate theories for each one, validated against the very data that created them. While the current study was limited to simulated, noise-free data, the authors suggest that with further development to handle real-world experimental noise, this technique could transform how scientists build theories, turning a process that once took years into one that happens in minutes. The work demonstrates that for artificial intelligence to truly assist in science, it may not need to see everything; it just needs to see the right things.

Technical Summary: Physical Invariants as a Data Representation for PDE Discovery

Problem Statement
The discovery of theoretical models for complex chemical systems faces a critical bottleneck: while modern experimentation can generate vast datasets of molecular interactions, conventional theory building relies on specialized human expertise and cannot scale to match this volume. Existing automated approaches for recovering Partial Differential Equation (PDE)-based field theories generally fall into two categories: bottom-up coarse-graining or top-down symbolic regression. Current top-down methods, particularly those leveraging Large Language Models (LLMs) for symbolic regression, suffer from a fundamental data representation issue. Because spatiotemporal field data cannot be directly placed into an LLM prompt, these models typically rely solely on a numerical evaluator's score (e.g., a loss function) to guide the search. This indirect feedback loop forces the LLM to infer physical structures without "reading" the data in the manner a human theorist would—by identifying salient patterns and physical invariants.

Methodology
The authors propose a pipeline that inserts a data interpretation stage between the raw field data and the LLM. This pipeline operates in a loop consisting of four stages: Data Interpretation $\rightarrow$ LLM Equation Proposal $\rightarrow$ Parsing/Canonicalization $\rightarrow$ Evaluation.

Data Interpretation: Instead of feeding raw field slices to the model, the system computes a compact set of physical diagnostics (approximately 300 tokens) derived from spectral analysis. These diagnostics answer four specific questions a theorist would ask:
- Temporal Order: By analyzing the time evolution of Fourier mode amplitudes ( $|\hat{u}_k(t)|$ ), the system distinguishes between first-order (monotonic decay/growth) and second-order (oscillatory) time derivatives.
- Linear Coefficients: Regression on the decay rates and oscillation frequencies of modes recovers linear coefficients (e.g., diffusion $\nu$ , reaction $\sigma$ , wave speed $c$ ).
- Nonlinearity: The system detects nonlinearity by measuring spectral energy transfer to high wavenumbers and the degradation of the linear regression fit ( $R^2$ ) across modes.
- Advection: Spatial correlation under translation is measured to detect directional transport (advection) versus isotropic evolution (diffusion/growth), estimating advection velocity.
LLM Proposing: The computed diagnostics are appended to a prompt instructing the LLM (specifically QwQ-32B) to propose a PDE structure using a defined set of symbols (field $u$ and its derivatives) without a pre-defined library of candidate terms. The LLM reasons in prose and outputs the equation with explicit coefficients.
Parsing and Canonicalization: The proposed equation is rewritten into a single canonical form to ensure that notational variations do not affect scoring.
Evaluation: A numerical evaluator fits the coefficients of the proposed structure using regularized least squares and scores the equation based on a weak-form residual and a sparsity penalty.

Key Contributions

Data Interpretation Stage: The primary contribution is the introduction of a pre-processing stage that translates raw spatiotemporal fields into physical quantities a theorist would consult. This allows the LLM to "read" the data as a human expert would, rather than relying solely on a scalar score.
Library-Free Discovery: By providing physical context via interpretation, the method eliminates the need for a predefined library of PDE term candidates, allowing the LLM to propose arbitrary symbolic structures.
Zero-Shot Efficiency: The pipeline requires no training of the LLM or the interpretation module. The diagnostics are computationally cheap (a fraction of a second per field) and token-efficient compared to raw data.

Results
The authors evaluated the method on a benchmark of 44 simulated scalar fields generated from a library of eight standard transport and kinetic terms. The results were compared across three input conditions:

Data Interpretation: The proposed method.
Permuted Interpretation: Controls where the LLM receives diagnostics from a different sample to test if it relies on the input.
Raw Field Slices: Ten 1D slices of the raw field data.

Accuracy: The data interpretation approach achieved an F1 score of 0.720, significantly outperforming raw field slices (0.225) and a "best fixed term set" baseline (0.391).
Exact Recovery: Interpretation led to exact equation recovery in 14 out of 44 samples, compared to only 2 for raw data.
Dependency on Input: When interpretations were permuted (mismatched with the data), accuracy collapsed to 0.244, confirming that the LLM actively utilizes the provided physical measurements rather than relying on prior knowledge or common terms.
Strata Performance: The method achieved perfect recovery (F1=1.0) for linear-only systems. Performance dropped for nonlinear systems, where the interpretation successfully identified the presence of nonlinearity but could not always determine the specific functional form of the nonlinear term.

Significance and Claims
The paper claims that data interpretation offers a practical route to automated field theory construction that can co-evolve with experimentation. By allowing an LLM to read field data as a theorist does, the method bridges the gap between high-throughput experimental data and human-level theory building.

The authors emphasize that this approach is modest in scope but significant in principle:

It demonstrates that a small set of physical measurements is a more effective input for LLMs than raw data or numerical scores alone.
It automates the "top-down" discovery of PDEs without requiring human expertise to define the search space (library) or to manually analyze the data.
The pipeline is designed to scale with experimentation, potentially allowing new datasets from thousands of molecular systems to automatically yield candidate theories validated against the data that produced them.

The authors acknowledge limitations, noting that the current characterization tools are insufficient to fully determine the form of nonlinear terms and that the diagnostics are currently fragile to noise, requiring robust differentiation for application to real experimental data. However, they assert that the observed gains in accuracy are substantial and statistically significant, validating the core concept of physical invariants as a superior data representation for PDE discovery.

Technical Summary: Physical Invariants as a Data Representation for PDE Discovery

More like this