Technical Summary: SIGA – Self-Evolving Coding-Agent Adapters for Scientific Simulation

1. Problem Statement

Scientific simulators (e.g., GEOS, OpenFOAM, LAMMPS) expose specialized, complex input languages (often Domain-Specific Languages or DSLs) that translate high-level scientific goals into executable configurations. While modern coding agents possess general capabilities for file navigation, code editing, and command execution, they lack the specific "executable contract" of a given simulator. This contract includes:

• Vocabulary: Specific solver names, material models, and schema attributes.
• Structural Constraints: Syntax rules, schema validation (e.g., XSD), and cross-file consistency requirements.
• Termination Conditions: Criteria for when a configuration is complete and valid.

Without this grounding, agents frequently produce invalid, incomplete, or structurally unsound configurations, requiring domain scientists to spend hours or days debugging. The paper frames simulator setup not as a need for a new agent architecture, but as an interface-grounding problem: determining the minimal, simulator-specific adaptations required to make an off-the-shelf coding agent reliable.

2. Methodology: SIGA

The authors introduce SIGA (Simulator-Interface Grounding Adapter), a lightweight wrapper around a frozen, off-the-shelf coding agent (specifically the Claude Code harness). SIGA does not reconstruct the agent's planning loop; instead, it injects grounding at three specific interfaces: Context, Tools, and Termination.

SIGA is composed of four modular components:

1. Retrieval (R): Adds semantic search capabilities over simulator documentation, schemas, and example code. This addresses the "unknown vocabulary" problem where agents fail to search for the correct terms using standard keyword tools.
2. Procedural Memory (M): Injects a compact, always-on "cheatsheet" into the system prompt. This contains high-frequency simulator conventions, element names, and anti-patterns, ensuring critical domain knowledge is visible without requiring the agent to decide to retrieve it.
3. Validator-Driven Self-Refinement (X): Exposes a schema validator (e.g., xmllint for GEOS) as an agent-callable tool. This allows the agent to check and repair its own work mid-trajectory before finalizing.
4. Validation-Enforced Termination (S): A mandatory stop-hook that prevents the agent from terminating until the generated deck passes structural validation. If validation fails, the agent receives structured repair feedback and is forced to continue.

Self-Evolution: The paper also explores a self-evolving variant where an offline agent analyzes prior trajectories to rewrite the adapter's contents (the system primer and memory cheatsheet), optimizing the grounding layer without modifying the underlying agent model.

3. Key Contributions

• Adapter-over-Reconstruction: The paper demonstrates that adapting existing, engineered coding harnesses is more effective and portable than building simulator-specific agent loops from scratch.
• Modular Grounding Design: It defines a compact design space (R, S, X, M) where the dominant mechanism shifts based on the simulator's specific bottleneck (e.g., structural completeness vs. domain correctness).
• Self-Evolution of Grounding: It shows that the adapter's textual contents can be automatically improved via self-evolution, matching or outperforming hand-designed configurations.

4. Experimental Results

Primary Evaluation: GEOS Simulator

Evaluated on the GEOS multiphysics simulator for subsurface science:

• Reliability vs. Quality: On in-distribution tasks, the baseline agent already achieved high quality. However, on harder, held-out tasks, SIGA significantly improved reliability. It reduced the across-run standard deviation by approximately 16× and rescued tasks that would otherwise result in catastrophic failures (empty or unparseable decks).
• Efficiency: SIGA-equipped agents produced complete, high-quality GEOS decks in ~5 minutes, matching the quality of a domain expert who required ~3 hours (a 36× speedup).
• Error Correction: The adapters primarily fixed "missing block" errors (structural omissions). Residual errors were largely "bad attribute values" (semantic/physical correctness), which are harder to solve via schema validation alone.
• Self-Evolution: The self-evolved adapter (SE) achieved the highest mean performance on held-out tasks, outperforming the best hand-designed configuration.

Transfer Studies

• OpenFOAM (CFD): The dominant mechanism shifted to Termination (S). The stop-hook ensured all required files were generated, allowing SIGA to produce complete cases where native OpenFOAM agents (Foam-Agent, MetaOpenFOAM) failed to generate required files entirely.
• LAMMPS (Molecular Dynamics): The dominant mechanism shifted to Memory (M) and Retrieval (R). Since LAMMPS scripts are structurally simple but require precise physical parameters, the grounding layer improved performance by injecting domain knowledge (unit conventions, command ordering) rather than enforcing structural completeness.

5. Significance and Claims

The paper argues that the immediate bottleneck for AI in science is not autonomous discovery, but the reliable operation of existing scientific software.

• Pragmatic Target: SIGA demonstrates that lightweight, self-improvable grounding layers can turn general coding agents into practical operators of complex scientific tools, reducing setup time from hours to minutes.
• Interface-Dependent Mechanisms: The study claims that the "correct" grounding strategy depends on the simulator's interface. If the bottleneck is structural completeness (missing files/blocks), termination hooks (S) are critical. If the bottleneck is domain correctness (wrong values), procedural memory (M) and retrieval (R) are paramount.
• Autonomy Limitations: In a study on agent autonomy, the authors found that agents rarely consulted human supervisors when "on-disk" example libraries were available, suggesting that agents substitute retrieval for human consultation unless the information is truly missing.

The authors conclude that adapting existing harnesses with minimal, targeted grounding is a viable and efficient path to deploying AI in scientific workflows, distinct from the longer-term goal of fully autonomous scientific discovery.