💻 computer science

Theory Under Construction: Orchestrating Language Models for Research Software Where the Specification Evolves

This paper introduces Comet-H, an iterative prompt automaton that orchestrates the coupled development of code, theory, and documentation to prevent hallucination and desynchronization in research software, demonstrating its effectiveness through a Python static-analysis tool that significantly outperforms baselines on a 90-case benchmark.

Original authors: Halley Young, Nikolaj Björner

Published 2026-05-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Halley Young, Nikolaj Björner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a new kind of car, but you don't have a finished blueprint. Instead, you have a team of incredibly talented, fast-thinking engineers (the AI) who can draw parts, build engines, and write the owner's manual all at once.

The problem is that these engineers are prone to two specific mistakes:

The "Fake It Till You Make It" Trap: They might write in the manual, "This car goes 200 mph," before they've actually built the engine to prove it. Then, the next engineer reads that claim, assumes it's true, and builds a chassis designed for 200 mph. If the engine can't actually do it, the whole project is built on a lie.
The "Lost in Translation" Trap: The engineer who designed the engine might change their mind about how it works, but the person writing the manual doesn't know. Now the manual describes an old engine, the blueprint shows a new one, and the actual car on the assembly line is something else entirely. They are all drifting apart.

This paper, "Orchestrating Language Models for Research Software Where the Specification Evolves," introduces a new system called Comet-H to fix these problems. It treats research not as a straight line, but as a dance where the music, the steps, and the dancers must constantly adjust to each other.

The Core Idea: A "Conductor" for the AI

Instead of just asking the AI to "write code," the authors built a Conductor (a controller) that manages the whole orchestra. This Conductor doesn't just tell the AI what to do; it constantly checks the "workspace" (the code, the math theory, the benchmarks, and the paper) to see what is missing or out of sync.

Here is how Comet-H works, using simple analogies:

1. The "Fading To-Do List" (Obligation Decay)

Imagine you are writing a book. You have a sticky note that says, "I need to fact-check this chapter."

Old way: If you forget to check it, the note stays there forever, cluttering your desk, or you ignore it and move on.
Comet-H way: The note has a half-life. Every time you take a step forward in the project, the note gets slightly fainter. If you don't address it soon, it fades away. But if it's very recent, it glows bright red.
Why this matters: This forces the AI to deal with unfinished business (like "prove this claim") while it's still fresh. If the AI tries to ignore the debt, the "glow" gets brighter, and the Conductor forces the AI to stop and fix it before moving on.

2. The "Reality Check" (Reactive Grounding)

Whenever the AI changes the "public face" of the project (like the README file or the research paper), Comet-H hits the Pause Button.

The Rule: You cannot change the story without checking the facts.
The Process: If the AI writes, "Our tool is 10x faster," the system immediately stops and says, "Okay, show me the race results." It forces the AI to run the code and generate a "grounding ledger" (a machine-readable receipt) proving the claim.
The Result: This stops the "Fake It Till You Make It" trap. A lie can only survive for one step before it gets caught and corrected.

3. The "Safe Step" (Adjacency Constraints)

Sometimes, an AI gets excited and wants to jump from "building a bicycle" to "building a spaceship."

The Rule: Comet-H only allows adjacent moves. The AI can take one small step forward (e.g., "add a gear to the bicycle"), but it cannot jump to a completely different universe.
Why this matters: This keeps the project grounded. If the AI wants to change the core theory, it has to do it in a way that still connects to what was built yesterday. This prevents the team from drifting so far apart that they forget what they were originally trying to build.

The Results: The "a3" Case Study

The authors tested this system by building a portfolio of 46 different research software projects. The star of the show is a tool called a3, a program designed to find bugs in Python code.

The Challenge: Usually, bug-finding tools are like noisy neighbors; they scream "BUG!" at everything, even when it's fine. This creates a lot of false alarms.
The Comet-H Approach: The system didn't just build the tool; it evolved the theory behind it. It started with a simple idea, realized it was too hard to compute, and the Conductor allowed the team to pivot to a new mathematical approach (using "safety certificates") that actually worked.
The Outcome: The final tool was incredibly accurate. It caught real bugs (high precision) without screaming about things that weren't broken. It scored 0.768 on a test scale, while the next best tool only scored 0.364.

What We Learned About AI Behavior

By watching the AI work on these 46 projects, the authors noticed some interesting patterns:

The "Cleanup Crew" is Real: In the beginning, the AI is busy building new features. But as the project nears the end, the AI spends most of its time auditing and fixing. It's like a construction crew that spends the last week of a project just checking that the paint is dry and the doors open, rather than building new walls.
Honesty Emerges: When forced to prove their claims, the AI became surprisingly honest. Instead of hiding failures, it started explicitly stating, "We can't solve this specific type of problem yet." The system didn't program this honesty; it emerged because the "Reality Check" made lying too difficult.
Self-Organization: Over time, the AI started organizing its own code into cleaner, more logical structures, even though no one explicitly told it to do so.

The Big Picture

The paper argues that building research software is different from fixing a typo in a document. It's a co-evolution process. The theory, the code, the tests, and the story must grow together.

If you just ask an AI to "write a paper and code," it will likely drift, hallucinate, and lose sync. But if you give it a Conductor that constantly checks the score, forces reality checks, and ensures the steps stay connected, you can build complex, trustworthy research tools that actually work.

In short: Comet-H is a system that stops AI from daydreaming and forces it to keep its promises, ensuring that the story it tells matches the code it writes.

1. Problem Statement: The Co-Evolution Challenge

The paper addresses a critical gap in automated research software development: the co-evolution problem. Unlike standard software engineering tasks where the specification is fixed, research software involves a specification that is under construction. The mathematical thesis, executable code, benchmark surfaces, and public claims must mature together.

Current Large Language Model (LM) agents fail in this regime due to two specific failure modes:

Hallucination Accumulation: Unsupported assertions or fabricated data introduced in one step (e.g., in a paper draft) are treated as ground truth in subsequent steps, propagating errors through the development loop.
Desynchronization: The code, theory, or the LM's internal world model drifts apart. For example, code may be optimized for an outdated specification, or the LM's internal representation of the repository becomes stale as files accumulate, leading to global incoherence even if pairwise consistency exists.

Existing frameworks (e.g., SWE-agent, AI Scientist) assume fixed tasks or fixed evaluation frames, making them unsuitable for research where the goal itself evolves.

2. Methodology: Comet-H

The authors propose Comet-H, an iterative prompt automaton designed to orchestrate the coupled evolution of theory, code, evidence, and claims.

Core Architecture

Comet-H operates as a Co-Evolution State Machine that manages a workspace $W$ consisting of six components:

$T$ (Theory): The mutable mathematical thesis.
$R$ (Repository): The code implementation.
$P$ (Public Projection): The paper and README.
$E$ (Evidence): Benchmarks and grounding ledgers.
$U$ (Utility Hypothesis): The "why" of the software.
$Q$ (Open Obligations): A queue of unverified claims or pending tasks.

Key Mechanisms

Reactive Grounding Triggers:
- Any change to public-facing artifacts ( $P$ : paper/README) immediately triggers a Grounding Creation followed by a Skeptical Audit.
- This forces the system to verify claims against executable code and benchmarks before they can propagate, bounding hallucination propagation to a single step.
Adjacency Constraints:
- The controller prevents the system from drifting arbitrarily. Any expansion or pivot must be "adjacent" to the current state (reachable in a single conceptual step) and must strengthen existing benchmarks or theory. This prevents the model from jumping to unrelated problems that lack evaluation surfaces.
Prompt Selection as a Contextual Bandit:
- The controller selects the next prompt from a fixed alphabet of 17 prompt families (e.g., Ideation, Seed Generation, Paper Strengthening, Skeptical Audit).
- Scoring: A transparent, hand-set linear scorer computes a score based on a feature vector of workspace deficits (e.g., "low test coverage," "unverified claims").
- Obligation Decay: Unfinished follow-up work is stored in an obligation vector $o_t$ that decays exponentially (half-life of 8 steps). This ensures the controller focuses on recent, high-priority debts without being overwhelmed by stale history, mimicking non-stationary bandit strategies.
Development Loop Phases:
- Seed: Selects a differentiated research question and formal core.
- Reactive Iteration: Generates code/papers, then iteratively hardens them based on the weakest surface (e.g., if benchmarks are weak, it runs Benchmark Tightening).
- Tail: A fixed sequence of audit and polish steps to ensure the final output is grounded, regardless of remaining budget.

3. Key Contributions

Definition of Co-Evolution Failure Modes: The paper formally identifies hallucination accumulation and desynchronization as the primary causes of failure in automated research, distinct from standard coding errors.
Co-Evolution State Machine: A formal framework (Definition 4.1) that treats theory as a mutable state variable and enforces cross-artifact synchronization via a finite control mode and decaying obligation memory.
Transparent Orchestration: A novel controller design pairing a linear, hand-set scorer with a fading record of unfinished work. This avoids learned policies (black boxes) in favor of auditable, interpretable decision-making.
Empirical Portfolio: The creation of 46 research-software repositories across 12+ domains, demonstrating the scalability of the approach.

4. Results and Case Study (a3)

The authors provide a deep dive into a3, a Python static-analysis tool developed entirely within the Comet-H loop.

Performance: On a 90-case benchmark, a3 achieved an F1 score of 0.768, significantly outperforming the next-best baseline of 0.364.
Monotone Gains: Ablation studies showed that each added layer of theory/practice coupling (e.g., interprocedural analysis, dynamic symbolic execution) improved performance without degradation.
Theory Evolution: The system successfully revised its underlying theory twice during development (shifting from a "distance-to-boundary" framing to "safety certificates" and integrating SMT solvers) without breaking the codebase, demonstrating the controller's ability to handle specification pivots.
Behavioral Observations:
- Audit Dominance: In the later phases of development, the majority of commits were for grounding, auditing, and repairing claims rather than adding features.
- Visible Failure: The system learned to openly report limitations (e.g., "UNKNOWN" verdicts for recursive cases) rather than hiding them, a behavior driven by the Skeptical Audit prompt.
- Self-Organization: Over long horizons, the LM naturally reorganized code into more compositional structures (e.g., converting to submodules) without explicit prompting.

5. Significance

This paper shifts the paradigm of AI research agents from task completion (solving a fixed problem) to state management (maintaining the integrity of a co-evolving system).

Governance over Generation: It argues that the hardest part of research automation is not generating code, but ensuring that the mathematical thesis, the code, and the claims remain synchronized.
New Evaluation Metrics: It suggests that future evaluations of research agents should measure stability, recoverability from drift, and latency of integrity repair, rather than just final benchmark scores.
Practical Impact: By proving that LMs can manage the full lifecycle of research software—including theory revision and rigorous auditing—Comet-H provides a blueprint for building trustworthy, autonomous scientific discovery systems.

In summary, Comet-H demonstrates that by treating research as a co-evolutionary process with explicit mechanisms for grounding, auditing, and bounded drift, LMs can successfully navigate the complex, non-linear path of creating novel research software.