Grounded autonomous research: a fault-tolerant LLM pipeline from corpus to manuscript in frontier computational physics

Technical Summary: Grounded Autonomous Research in Frontier Computational Physics

Problem Statement
Autonomous research agents have achieved end-to-end automation in "calibration-friendly" domains like machine learning, where execution feedback (e.g., training loss curves) or established reference values provide intrinsic verification. However, frontier physical science presents a categorical challenge: physical reasoning dictates methodology choices, toolchains are often underdocumented, and execution provides no ground truth. In these environments, unscaffolded agents default to internal priors, producing plausible but unverifiable results and hallucinating methodology that cannot be corrected by execution alone. The core problem is the lack of a structural mechanism to force agents to confront their outputs against external literature anchors, rather than merely citing them.

Methodology: The Grounded Pipeline
The authors present an end-to-end pipeline that transforms a corpus of 11,083 condensed-matter physics arXiv papers into a publication-grade manuscript. The system operates across six phases using 47 "fresh-context" LLM sessions that share only on-disk state, preventing the accumulation of conversational history errors.

1. Architecture & Scaffolding:

   • Fresh-Context Isolation: Each session begins with a clean context window. Communication occurs solely via disk artifacts, ensuring that errors or hallucinations in one session do not propagate uncritically to the next.
   • Distributed Grounding: The pipeline utilizes a curated knowledge base (INDEX.md, PSEUDOPOTENTIALS.md) and "house rules" (PILOT_HOUSE_RULES.md) containing operational constraints and verified workflows. Crucially, these are loaded only from the Pilot stage (S2) onward, keeping the idea conception phase (Breadth/Depth) free from methodological bias.
   • Adversarial Review: Dedicated sessions are prompted to "find rather than confirm," acting as a fault-tolerance mechanism to catch overclaims.

2. The Six-Phase Workflow:

   • Breadth (3 sessions): Maps the corpus to identify candidate research themes.
   • Depth (5 sessions): Expands candidates into specific research questions with explicit reproduction targets. A diversity ledger ensures non-overlapping directions.
   • Pilot (24 sessions): The core "grounding" engine. It involves:
     • Program Selection (S1): Selects a direction based on software feasibility (must be executable by named open-source tools).
     • Tooling Check (S2): Verifies the local stack and loads the curated knowledge base.
     • Reproduction (S3): Reproduces 5 published papers as numerical anchors.
     • Iterative Gate-Review: A cycle of computational gates followed by adversarial reviews. If a result deviates from literature, the agent must confront the discrepancy. The process is bounded by a hyperparameter (max_pilot_cycle) that forces a transition to production or termination if convergence is not achieved.
   • Pre-production & Production: Consolidates the pilot recipe into a locked plan and executes the novel computations.
   • Writing: Drafts the manuscript in fresh-context sessions, integrating findings with the literature.

3. Operational Definition of "Grounding":
   The paper defines "grounding" not as literature availability, but as structurally enforced numerical confrontation. At calibration checkpoints, the pipeline forces a numerical comparison between agent-computed values and published reference values. If the values do not match within tolerance (T4 tier: ≤20% deviation), the agent must revise its methodology or issue a "NOT PASS" verdict.

Key Contributions

1. End-to-End Pipeline: A system that autonomously conceives a research direction, calibrates methodology via reproduction, conducts novel first-principles computations, and writes a manuscript in the domain of altermagnetic piezomagnetism.
2. Operational Grounding Mechanism: The identification of the Pilot stage as the architectural element that operationalizes grounding. Through two paired failure modes (a pre-architecture baseline lacking topic-selection grounding and a no-pilot ablation lacking execution grounding), the authors demonstrate that without enforced numerical confrontation, agents accept un-anchorable topics or proceed with unverified assumptions.
3. Fault-Tolerance Primitives: The system employs fresh-context isolation, distributed grounding, and adversarial review to catch errors. In the pilot stage, 15 "catch episodes" were documented where these mechanisms corrected overconfident or erroneous outputs.
4. Quantified Intervention Pattern: Human intervention was limited to operational knowledge curation (e.g., adding a sanity check rule or Wannier validation protocol after a failed reproduction) and paywall/API recovery. Zero scientific direction or parameter choices were made by humans.

Results
The pipeline produced a submission-grade manuscript on altermagnetic piezomagnetism containing three substantive physics findings:

1. Symmetry-permitted anomalous Hall response in MnTe at a deep-valence-band anchor, with a plateau amplitude of $\alpha = 397$ S/cm.
2. Orbital piezomagnetic response in MnTe, yielding $\Lambda_{orb}^{zxx} = -1.96$ $\mu_B$/cell per unit strain.
3. Cross-symmetry-class response in d-wave Lieb CsV$_2$Te$_2$O under B$_{1g}$ shear, yielding $\Lambda_{topo} = -24.9$ $\mu_B$/cell per unit shear.

The system successfully navigated 2,162 literature-consultation events across 14 channels. It identified and documented systematic uncertainties, such as a basis-truncation systematic in the MnTe orbital magnetization anchor, refusing to certify convergence where the data was insufficient (issuing "NOT PASS" verdicts).

Significance and Claims
The paper claims that this work establishes a foundation for autonomous research in high-stakes scientific domains where execution does not provide ground truth.

• Shift from "Plausible" to "Anchored": The system moves beyond generating plausible-sounding text to producing results that are structurally forced to confront external reality.
• The Role of the Pilot: The pilot stage is identified as the "grand anchor mechanism." Without it, agents either select topics that cannot be calibrated (pre-architecture baseline failure) or proceed with unverified methods despite having access to reference values (no-pilot ablation failure).
• Honest Uncertainty: The pipeline's value lies in its ability to identify its own limits. By documenting systematic uncertainties and refusing to certify convergence when anchors are inconclusive, it produces "anchored failures" that are tractable for future iterations, rather than "concealed mysteries" of ungrounded hallucination.
• Frontier Applicability: While current frontier anchors are scarce and provisional (e.g., single published values for new phenomena), the architecture treats consistency with literature as the primary metric of success, acknowledging that this represents "measured exposure" rather than absolute truth.

The work does not claim to replace human scientists but to provide a scaffolded, fault-tolerant framework that handles the operational burden of literature confrontation, allowing human researchers to focus on the curation of knowledge and the interpretation of anchored results.