Technical Summary: Reliability of AI Attackers Against a Fixed Vulnerable Target

Problem Statement

While recent literature has established that Large Language Models (LLMs) can autonomously conduct multi-stage cyber attacks—such as exploiting web vulnerabilities, brute-forcing credentials, and chaining attack stages—a fundamental gap remains regarding the consistency of these behaviors. Prior research has largely focused on capability (can the model succeed?) based on single-run or small-sample studies (typically 1–10 runs), implicitly treating these few results as representative of model behavior. This paper addresses the unanswered question: How consistent are autonomous LLM attacks across repeated trials under identical conditions?

This consistency is critical for two communities:

1. AI Safety Researchers: To determine if refusal-based safety mechanisms hold under sustained, programmatic use within professional framings that mirror legitimate attacker tradecraft.
2. Cyber Defenders: To assess whether autonomous attacks produce predictable signatures for detection or if each instance follows a novel path that evades pattern-matching defenses.

Methodology

The study presents the first large-scale empirical measurement of LLM attack consistency, conducting 400 independent runs (100 per model) against a fixed, identical target.

Experimental Setup

• Target: A self-owned, isolated Azure Virtual Machine hosting three deliberately vulnerable services:
  • OWASP Juice Shop (Port 3000): SQL Injection vulnerability.
  • OpenSSH (Port 22): Weak credentials (honeypot:password123).
  • vsftpd (Port 21): Anonymous FTP access with sensitive files.
• Models Evaluated:
  • Claude Sonnet 4 (Anthropic)
  • Gemini 2.5 Flash-Lite (Google)
  • GPT-4o-mini (OpenAI)
  • qwen2.5-coder:14b (Local via Ollama)
• Orchestrator: A custom Python script (orchestrator.py) driving a command-execute-observe loop. It enforces a structured attack progression (Recon → Exploit Web → Exploit SSH → Post-Exploit → Exploit FTP → Complete) and includes safety filters, loop detection, and phase-stall interventions.
• Constraints & Parameters:
  • Prompt: A standardized "authorized penetration testing" framing.
  • Temperature: Set to 0.3 for Ollama, Gemini, and OpenAI. Note: Anthropic calls were inadvertently dispatched without a temperature parameter, defaulting to 1.0.
  • Iteration Limit: 25 iterations per run.
  • History: Cloud models retained the last 10 messages (~5 exchanges); the local model (qwen) retained full conversation history.

Data Collection

The study generated over 6,000 individual command-execute-observe interactions over a 60-hour window in March 2026. A post-hoc audit revealed that 91 of 1,135 Anthropic API calls returned overloaded_error (HTTP 529), truncating 39 of the 100 Claude runs. These were reclassified from "safety refusals" to "upstream API failures."

Key Contributions

1. Large-Scale Consistency Measurement: The first study to measure autonomous LLM attack behavior at N=100 per model across a multi-service target, moving beyond anecdotal single-run benchmarks.
2. Refusal Rate Re-evaluation: A rigorous audit showing zero content refusals survived the authorized-pentest scaffold across 400 runs. An earlier draft's claim of a 42% refusal rate for Claude was corrected to be entirely caused by upstream API capacity failures.
3. Failure Mode Taxonomy: A detailed classification of how models fail, distinguishing between model-level behaviors (premature completion, iteration exhaustion) and provider-level artifacts (API truncation).
4. Strategic Diversity Analysis: Quantification of attack strategy diversity using Shannon entropy and unique phase sequence ratios, revealing significant variance between models.

Key Results

1. Zero Refusal Rate Under Professional Framing

Across 400 runs and four providers, no model emitted a content refusal that survived the orchestrator's one-shot authorization re-prompt at iterations 0–1. This supports the "competing-objectives" framework, suggesting that under a "helpful professional" framing, the helpfulness objective dominates safety refusals.

2. Distinct Failure Modes

Models fail in qualitatively different ways:

• Claude Sonnet 4: Dominated by API truncation (39/100 runs cut short by overloaded_error). Among successful runs, it achieved full exploitation in 61% of cases.
• Gemini 2.5 Flash-Lite: The most reliable operator with an 85% full exploitation rate. It rarely fails, with errors limited to early quits (8%) or hitting the iteration cap (7%). It exhibits perfect self-correction (1.0 rate).
• GPT-4o-mini: Characterized by iteration budget exhaustion (23/100 runs hit the 25-iteration cap without completion). It is the most thorough explorer (mean 24.01 iterations) but achieves full exploitation in only 56% of runs.
• qwen2.5-coder:14b: Prone to premature completion (52/100 runs declared completion before exploiting all services), achieving full exploitation in only 25% of runs.

3. Strategy Diversity and First-Move Behavior

• Diversity: GPT-4o-mini exhibited near-maximal diversity (98 unique strategies out of 100 runs), while Gemini was more deterministic (48 unique strategies).
• First-Move Split: Models split on initial strategy. Claude and Gemini attacked web services first (67–90% of runs), while qwen and GPT-4o-mini attacked FTP first (63–82% of runs).
• Time-to-First-Exploit: Consistent across models, occurring at iteration 4–6 (approx. 15–30 seconds wall-clock time).

4. Cross-Service Credential Reuse

Autonomous credential reuse (using FTP credentials for SSH login) was observed only in models with full conversation history (qwen: 57%) or higher iteration counts allowing for context retention (GPT-4o-mini: 49%). Cloud models with trimmed history (Claude, Gemini) showed 0% reuse. This suggests the behavior is context-dependent rather than purely architectural.

5. Statistical Significance

Cross-model differences in exploitation rates are statistically significant (p < 0.001) with large effect sizes. For example, the difference in SQL injection rates between qwen and Gemini is Cohen's h = 1.12.

Significance and Claims

The paper claims to shift the paradigm of LLM security evaluation from capability to reliability.

• For AI Safety: The study demonstrates that single-run benchmarks are insufficient. Under professional, authorized framings, refusal-based defenses appear ineffective across major providers. Safety evaluations must move toward measuring refusal rates under legitimate-looking, operationally dangerous task framings.
• For Cyber Defense: The consistency of the "time-to-first-exploit" (~30 seconds) suggests a narrow detection window. Defenders should focus on detecting the transition from reconnaissance to active exploitation (phase transitions) rather than specific command signatures, especially given the high strategy diversity of models like GPT-4o-mini.
• For Red Teaming: The results suggest that automated red teaming should rely on multi-run statistical reporting (e.g., N=30 to N=100) with confidence intervals rather than single-run anecdotes. The choice of model involves a trade-off: Gemini offers high reliability/consistency, while GPT-4o-mini offers superior attack surface coverage via strategy diversity.

The authors explicitly note limitations, including the unintended temperature asymmetry for Claude, the history-trimming differences between cloud and local models, and the specific vulnerability profile of the honeypot, calling for follow-up studies to disentangle these factors.