Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors

Technical Summary: Evaluating Frontier AI Agents as Autonomous Clinical Security Auditors

Problem Statement
Clinical AI models are increasingly deployed in high-stakes environments (e.g., mortality prediction, radiology screening) without formal adversarial testing. While these models may perform well on held-out test data, they remain susceptible to small, deliberate input perturbations that can lead to patient harm. Current security auditing is a structural bottleneck: it requires statistical expertise, custom software implementation (e.g., gradient-based attacks, shadow models, calibration metrics), and significant time. Most clinical deployment teams lack these resources, leading to ad hoc, incomplete, or vendor-delegated evaluations. The paper posits that frontier AI agents, capable of multi-step reasoning and tool use, could autonomously bridge this gap by implementing structured security evaluation specifications without human-in-the-loop guidance.

Methodology
The authors constructed an open evaluation task based on the METR Task Standard v0.3.0 to test whether frontier AI agents can autonomously execute a Security Posture Score (SPS) evaluation.

• Task Environment: The agent operates within a Docker container with access to a pre-trained clinical model, a patient dataset, and a written audit specification. The agent has no scaffolding code; it must implement all logic from pseudocode provided in the instructions.
• The SPS Framework: The agent must implement four distinct adversarial components and aggregate them into a weighted score:
  1. FGSM Robustness ($s_{fgsm}$): Implementing the Fast Gradient Sign Method using a surrogate logistic regression to approximate gradients for non-differentiable target models.
  2. Membership Inference Resistance ($s_{mi}$): Training four shadow models on overlapping data subsets to construct an attack classifier and testing it against the target model.
  3. Expected Calibration Error ($s_{ece}$): Binning predicted probabilities and computing the weighted mean absolute deviation between accuracy and confidence.
  4. Boundary Attack Resistance ($s_{ba}$): Iteratively walking from test points toward the opposite class to measure decision boundary proximity.
• Constraints: The agent must write a structured JSON report to submission.txt via a bash tool interface within a 40-turn limit.
• Experimental Design:
  • Datasets: Two clinical datasets were used: Wisconsin Diagnostic Breast Cancer (WDBC) and MIMIC-IV (ICU mortality).
  • Models: Three model architectures with increasing defense strength (Logistic Regression baseline, Random Forest with Platt scaling, XGBoost with adversarial training).
  • Agents: Three frontier models were evaluated: Claude Sonnet 4.6, GPT-4.1, and GPT-4o.
  • Protocol: Each model performed three independent runs per variant (6 variants total), resulting in 54 total evaluations.

Key Contributions

1. Open Evaluation Task: The paper releases a METR-standardized task for clinical AI security auditing, including the SPS specification, automated scoring infrastructure, and public assets for the WDBC variants.
2. Empirical Comparison: A 54-run pilot evaluation comparing the task completion rates, token efficiency, and numerical precision of three frontier models.
3. Failure Mode Taxonomy: A characterization of three distinct failure modes observed in GPT-4o: early session termination before file writing, arithmetic errors in weighted aggregation, and empty submission files.

Results

• Task Completion: Claude Sonnet 4.6 and GPT-4.1 achieved 100% task completion (18/18 runs each) with perfect evaluator scores. They produced numerically identical results across runs (with minor variance in GPT-4.1 due to stochastic shadow-model training seeds).
• GPT-4o Performance: GPT-4o achieved a 61% completion rate (11/18 runs). The 7 failed runs resulted in no valid submission or a partial score due to aggregation errors.
• Efficiency and Cost:
  • Token Usage: GPT-4o consumed roughly 5 times more input tokens per run than Claude Sonnet 4.6 (169K vs. 34K).
  • API Costs: Across 18 runs, GPT-4.1 cost ~$8, Claude Sonnet 4.6 cost ~$12, and GPT-4o cost ~$27.
  • Turns: Claude averaged 4.7 turns; GPT-4.1 averaged 10.4 turns; GPT-4o averaged 16.9 turns, with some runs hitting the 40-turn limit.
• Failure Analysis: The GPT-4o failures were attributed to "agentic execution discipline" issues rather than a lack of reasoning capability. Specific failures included:
  • Terminating sessions before writing the final JSON file (5 cases).
  • Manual arithmetic errors in the final weighted aggregation step instead of programmatic calculation (1 case).
  • Creating an empty submission file (1 case).

Significance and Claims
The paper claims that frontier AI agents are capable of autonomously implementing structured, multi-step clinical security evaluations with high fidelity, provided the task is well-defined. The results suggest that model selection is a primary architectural decision for practitioners building agentic security pipelines; the gap between the perfect performance of Claude Sonnet 4.6/GPT-4.1 and the 39% failure rate of GPT-4o is significant enough to impact deployment reliability.

The authors emphasize that this is a pilot evaluation ($n=18$ per model) intended to identify patterns rather than provide precise point estimates. They explicitly note that the SPS framework weights and thresholds are taken from a companion preprint (Eniolade, 2026) and have not yet undergone independent peer review. The study concludes that while the technology is viable, the "execution discipline" of the agent—specifically the ability to track progress and avoid context inflation or premature termination—is a critical differentiator in multi-step technical tasks. The paper does not claim that these agents can replace human statisticians entirely but rather that they can automate the implementation of rigorous evaluation protocols that are currently inaccessible to many clinical teams.