LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data

Technical Summary: AgentODE for ODE Discovery from Aggregate Data

Problem Statement
Mechanistic modeling via ordinary differential equations (ODEs) offers interpretable insights into complex dynamics, which is critical in clinical settings. However, discovering ODE structures and inferring parameters in rare diseases faces significant hurdles: individual-level data is often scarce, noisy, heterogeneous, and restricted by privacy constraints. While population-level summary statistics provide a privacy-preserving alternative, existing methods fail to jointly discover ODE structures and refine parameter distributions using only these aggregate statistics. Traditional symbolic regression and sparse identification require dense individual-level data, while population-level approaches like nonlinear mixed-effects models require pre-specified structures. Furthermore, recent Large Language Model (LLM)-based discovery methods still rely on individual-level data for evaluation, leaving a gap in handling data-scarce, privacy-constrained scenarios where parameters must be modeled as distributions to capture heterogeneity.

Methodology: AgentODE
The authors propose AgentODE, an end-to-end framework that jointly discovers ODE structures and refines parameter distributions solely from population-level summary statistics. The framework operates through two tightly coupled feedback loops mediated by an LLM:

1. Outer Loop (Structure Discovery):

   • Based on the LLM-SR paradigm, an LLM proposes candidate ODE structures conditioned on the problem specification.
   • Structures are evaluated not by direct fitting to individual trajectories, but by how well they perform when paired with inferred parameter distributions.
   • An Experience Buffer stores high-performing structures and parameter distributions to guide future proposals, utilizing an island-based model to maintain diversity and avoid local optima.

2. Inner Loop (Parameter Inference Agent):

   • For a given candidate structure, a tool-augmented agent iteratively refines parameter distributions (assumed to be independent LogNormal distributions to capture inter-individual heterogeneity).
   • Visual Summaries: The agent utilizes structured visual summaries of population-level data (e.g., mean trajectories with 95% confidence intervals, stratified by initial conditions, and Spearman correlation heatmaps of first differences) as empirical inputs.
   • Diagnosis–Update Loop:
     • Diagnosis: The agent simulates synthetic data from the current parameter distribution, computes summary statistics, and compares them against empirical statistics. It uses statistical tools to quantify discrepancies (e.g., in means, variances, autocorrelations, and cross-variable dependencies) and generates a diagnostic report identifying failure modes.
     • Update: The agent reasons over the diagnostic report and past successful iterations to update the parameter distributions. It identifies which parameters govern specific dynamical failures and adjusts their distributions accordingly.
   • Scoring:
     • Log Synthetic Likelihood (logSL): Used to evaluate parameter fit within a specific structure by comparing synthetic and empirical summary statistics.
     • Mean Normalized Summary Discrepancy (MNSD): Used to compare different ODE structures, normalizing discrepancies by the empirical interquartile range to ensure comparability across structures with different parameterizations.

Key Contributions

1. First Joint Framework: AgentODE is the first method to jointly discover ODE structures and refine parameter distributions using only population-level summary statistics, addressing the gap between data scarcity/privacy and mechanistic modeling.
2. Robustness in Sparse Settings: The authors demonstrate that reasoning from summary statistics serves as a robust signal for structure discovery in sparse and noisy clinical environments. By condensing population patterns and mitigating individual-level noise, summary statistics guide the LLM toward the true underlying mechanism, whereas methods with access to individual-level data often overfit to noise, recovering implausible or degenerate structures despite lower prediction error.
3. Synthetic Population Generation: The jointly inferred structure and parameter distributions can generate synthetic patient populations that capture observed heterogeneity, facilitating further analysis without compromising privacy.

Experimental Results
The framework was evaluated on three synthetic benchmarks (Apoptosis, Polymer DA Cross-linking, PKPD-Immune) and two clinical datasets: Acute Kidney Injury (AKI) from MIMIC-IV and Recessive Dystrophic Epidermolysis Bullosa (RDEB).

• Performance: AgentODE achieved performance comparable to baselines with full trajectory access (e.g., LLM-SR†, Neural ODE) on benchmarks and clinical datasets, often recovering more compact and mechanistically meaningful structures.
• RDEB Case Study: In the RDEB dataset (46 patients, 231 observations), AgentODE recovered a structure consistent with an independently developed expert model. In contrast, baselines with individual-level data (LLM-SR†) achieved lower Root Mean Square Error (RMSE) but discovered mechanistically implausible structures (e.g., treating a key dynamic variable as static or introducing dense, negative-only causal loops).
• Ablation Studies: Removing the iterative diagnosis–update loop or relying solely on LLM-based parameter initialization resulted in significantly worse performance and frequent generation of invalid (unstable) trajectories, highlighting the necessity of the iterative refinement and visual summary reasoning.

Significance and Claims
The paper claims that AgentODE opens new possibilities for mechanistic modeling of rare diseases where data scarcity and privacy constraints traditionally limit analysis. It posits that in sparse and noisy clinical settings, predictive performance (e.g., RMSE) is an insufficient criterion for evaluating mechanistic model quality; a model with lower error may simply be overfitting to noise. By operating on population-level summary statistics, AgentODE provides a principled, privacy-preserving approach that prioritizes mechanistic consistency over raw predictive accuracy, enabling the recovery of underlying dynamical behaviors that are robust across the cohort. The authors emphasize that the recovered structures should be interpreted as functionally consistent with observed population behavior rather than ground truth, requiring validation by domain experts before clinical application.