AI Models Excel at Orchestration but Falter at Biological Judgment: Findings from an Agentic Gene Annotation Study

Technical Summary: AI Models Excel at Orchestration but Falter at Biological Judgment

Problem Statement
Large Language Model (LLM) agents are increasingly deployed in biological workflows to direct analyses and interpret results. However, these systems typically conflate two distinct functions: workflow control (deciding which analysis to perform next to reduce uncertainty) and biological adjudication (interpreting accumulated evidence to reach a final biological conclusion). Current end-to-end evaluations make it difficult to determine if a model's success stems from efficient orchestration or reliable biological reasoning. Furthermore, it is unclear whether the general-purpose reasoning capabilities of frontier LLMs are necessary for workflow control, or if constrained decision models suffice, and whether LLMs are reliable enough to serve as the final biological decision layer.

Methodology
The study introduced Genome Skeptic, an evidence-constrained framework for bacterial gene annotation designed to decouple sequence measurement, workflow control, and final adjudication.

• Architecture: Sequence-level measurements were generated exclusively by deterministic bioinformatics tools (BLAST+, MMseqs2, DIAMOND, HMMER). Controllers could only request follow-up analyses from a predefined registry, preventing the generation of new measurements or modification of existing data.
• Cohort: A prospectively locked, blinded cohort of 20 bacterial genomes was selected, balanced for the presence and absence of tet(A) and tet(B) genes. Truth was established via two independent evidence routes (reference panel comparison and HMM profile/phylogenetic placement).
• Experimental Arms:
  1. Controller Comparison: Four strategies were evaluated using a fixed, rule-based adjudicator: Fixed control, Exhaustive control, Jev (a non-LLM, RLCD-based typed decision model), and GPT-5.6 Sol (an LLM acting as planner and critic).
  2. Role-Swap: To isolate the effect of adjudication, the final evidence records generated under GPT-5.6 Sol control were passed unchanged to either the rule-based adjudicator or GPT-5.6 Sol acting as the final decision-maker. No new analyses were run; only the interpretation of the identical evidence state changed.
• Metrics: Accuracy, sensitivity, specificity, number of follow-up analyses, latency, and cost. UNRESOLVED calls were scored as incorrect per the pre-specified plan.

Key Results

1. Orchestration Efficiency: As a workflow controller, GPT-5.6 Sol achieved the same accuracy (18/20 correct) as fixed and exhaustive strategies but required 32% and 47% fewer follow-up analyses, respectively. The non-LLM controller, Jev, matched Sol's efficiency and accuracy while operating with significantly lower latency and cost.
2. Adjudication Failure: When the same GPT-5.6 Sol model was given final decision authority over the identical evidence state, accuracy dropped significantly from 18/20 to 11/20 (McNemar P = 0.016).
   • Specificity fell from 1.00 to 0.30.
   • The model failed to correct any of the two errors made by the rule-based adjudicator.
   • The primary failure mode was not generating false positives but converting correct negative calls (ABSENT) to UNRESOLVED. All nine endpoint disagreements involved shifting negative calls to unresolved, including seven cases the rule-based system had correctly identified as negative.
3. Precursor Study: In a 60-case precursor study, changing controllers altered the final evidence records in 30–45 cases but changed the final endpoint in zero cases, confirming that controller differences affected evidence acquisition but not the final decision when the adjudication layer was fixed.

Key Contributions

• Decoupling of Agent Roles: The study provides empirical evidence that workflow orchestration and biological adjudication are separable computational problems with different performance requirements. A model effective at selecting informative actions is not necessarily reliable at converting evidence into a final biological call.
• Evaluation of Non-LLM Controllers: The inclusion of Jev demonstrates that constrained, non-LLM decision models can match the efficiency and accuracy of frontier LLMs for routine workflow routing, suggesting that general-purpose language reasoning may be unnecessary for tool selection in well-defined biological contexts.
• Failure Mode Analysis: The research identifies a specific failure mode where LLMs, when acting as adjudicators, tend to abstain (output UNRESOLVED) on cases that deterministic rules can resolve, rather than correcting errors or identifying missed positives.

Significance and Claims
The paper argues that scientific agent design should treat orchestration and adjudication as distinct layers. It claims that:

• LLMs add value in orchestration by efficiently selecting which analyses to perform to reduce uncertainty, but do not add value (and may reduce reliability) in final biological adjudication when compared to explicit, auditable decision rules.
• Tiered Architecture: A robust scientific agent architecture should assign deterministic systems to measurement and final judgment, lightweight or constrained controllers to routine routing, and reserve LLMs for higher-level orchestration or open-ended analysis selection.
• Evaluation Standards: Claims about end-to-end scientific automation should be preceded by evaluating individual agent capabilities (planning, retrieval, adjudication) separately, as end-to-end metrics can mask failures in critical decision layers.

The author maintains modesty regarding the scope, noting the study focused on a single gene-family endpoint and one LLM configuration. They do not claim LLMs are universally unsuitable for biological interpretation but emphasize that competence in orchestration does not imply competence in adjudication, and these roles must be evaluated independently.