Unmodeled states and uncertain action outcomes in agentic scanning tunneling microscopy

Technical Summary: Unmodeled States and Uncertain Action Outcomes in Agentic Scanning Tunneling Microscopy

Problem Statement
Autonomous scientific agents are increasingly deployed to control physical instruments, yet they often operate under the assumption that experimental states and action outcomes are sufficiently specified. In reality, physical experiments involve "unmodeled states" and "uncertain action outcomes." Interventions can alter hidden experimental states in unpredictable ways that are only apparent through subsequent measurements. This creates a critical challenge for agents: how to interpret the consequences of their actions when the relevant physical information (hidden states) is not directly observable, and when the transition dynamics of the system are incompletely understood. The authors define this as "action-outcome uncertainty," distinct from perception uncertainty, where an agent's intervention may degrade a system rather than improve it, without a deterministic way to predict the outcome beforehand.

Methodology
The authors investigate this problem using Scanning Tunneling Microscopy (STM) tip conditioning, a task traditionally requiring extensive human expertise due to inaccessible tip apex conditions and stochastic outcomes. They introduce Quailbot, an agent harness that integrates Large Language Model (LLM) agents directly into the instrument feedback loop.

• System Architecture: Quailbot links physical interventions with experimental readbacks. It provides the LLM with four instrument-control CLI tools: Get (read parameters), Set (write values), Ramp (step parameters), and Act (invoke named actions).
• State Definition: The system distinguishes between public states (directly exposed settings and readbacks like bias voltage and current) and hidden states (experimentally decisive but unobservable physical information, such as the atomic configuration of the tip apex).
• Apprenticeship: The LLM agents underwent a brief apprenticeship with a human expert. During this phase, the agent learned procedural knowledge (e.g., tip shaking, bias pulsing) and domain-specific judgment (distinguishing surface debris from tip artifacts). The agent utilized memory and skills modules to consolidate this training, allowing it to refine strategies based on action histories and new measurements.
• Experimental Setup: Experiments were conducted on a low-temperature STM using tungsten tips on FeTe samples at 4 K. The agents were tested using frontier models (gpt-5.6-sol, claude-opus-5) and a deliberately less intelligent probe (gpt-5.4-mini).

Key Contributions and Results
The study demonstrates that LLM agents can autonomously perform end-to-end tip conditioning but also highlights the specific failure modes arising from unmodeled states.

1. Autonomous End-to-End Delivery: In "launch-only" runs (where the agent receives a task prompt and operates without further human intervention), frontier LLM agents successfully conditioned tips from sabotaged initial states to measurement-ready endpoints.

   • One run reduced the tip state from a "doubled apex" (BraggZ score of 20.7) to a single, stable, atom-resolving tip (BraggZ score of 295.4) in 70.8 minutes.
   • The agent demonstrated adaptive reasoning, such as withdrawing the tip upon detecting a protrusion or increasing piezo walk steps when encountering triangular junk features.
   • A second run with a different model (claude-opus-5) and a different tip successfully conditioned the tip despite a piezo walker jam, adapting its relocation strategy to smaller, verified steps. This run also produced a spin-polarized tip, evidenced by qAFM modulation in the Fourier transform, despite spin polarization not being an explicit objective.

2. Limits of Autonomy in Unmodeled Scenarios: The study systematically exposed the limits of agent autonomy when relevant states were hidden or unmodeled:

   • Bent Tip: When a human operator intentionally bent a tip without informing the agent, the agent failed to identify the root cause. It recognized imaging abnormalities and current spikes but attributed them to "second tunneling apices" or surface contamination. It exhausted its conditioning options without resolving the lattice, requiring human takeover.
   • Undisclosed Walker Jam: When a walker jam occurred without prior disclosure, the agent recognized abnormal behavior but could not diagnose the jam. It incorrectly assumed the tip had moved off the crystal onto epoxy. Only after the human operator disclosed the jam and manually adjusted the drive voltage (a capability disabled in the agent's workspace) did the agent recover.
   • Sample Rotation: In a case where the agent could not find a clean surface region, it suggested tip replacement. The human operator resolved the issue by mechanically rotating the sample holder by 120 degrees—a physical intervention outside the agent's software interface. The agent then succeeded, suggesting the tip was already near the required condition.

3. Model Dependence: A less intelligent model (gpt-5.4-mini) failed to adapt to a disclosed walker jam, issuing increasingly large travel commands without adjusting its strategy, resulting in no successful delivery.

Significance and Claims
The paper claims that while LLM agents can successfully navigate complex, uncertain experimental workflows, their autonomy is fundamentally bounded by the observability and action limits of the instrument interface.

• Failure Modes: The authors identify a class of failure where agents recognize anomalies but cannot identify the underlying cause if the necessary information is unmodeled or unavailable through the interface. This contrasts with conventional benchmarks that often collapse such issues into general "reasoning failures."
• Requirements for Future Autonomy: The results argue that future autonomous physical experimentation requires agents capable of inferring hidden and unmodeled states, tracking the outcomes of their actions, and operating within the constraints of real instruments.
• Instrument Design: The authors conclude that "agent-ready" scientific instruments must provide access to essential control functions and diagnostic information, maintain clear records linking actions to outcomes, and support automated verification. However, they note that software standardization alone cannot reveal atomic structures or predict the stochastic effects of conditioning pulses; additional sensors or actuators may be required to bridge the gap between digital interfaces and physical reality.

The study does not claim that LLMs have solved the problem of unmodeled states, but rather that they provide a distinct route for handling uncertainty through iterative planning and reflection, while simultaneously exposing the specific boundaries where human intervention remains necessary.