Technical Summary: Can Open-Source LLM Agents Replace Static Application Security Testing Tools?

Problem Statement

The paper addresses the emerging question of whether Agentic AI, specifically general-purpose Large Language Model (LLM) agents, can effectively replace or rival traditional Static Application Security Testing (SAST) tools for cybersecurity vulnerability scanning. While GenAI has revolutionized code generation, the authors investigate its efficacy in validating code for security fitness. The core problem is determining if a modern, open-source, general-purpose LLM agent can reliably identify common vulnerabilities (e.g., SQL injection, insecure secrets, path traversal) in Python codebases under realistic conditions, or if it suffers from inherent limitations such as hallucinations, high false-positive rates, and lack of explainability compared to established rule-based tools.

Methodology

The study employed an empirical comparison between a custom-built LLM agent and a vetted, open-source SAST tool.

• Baseline Tool: Bandit, a Python-focused SAST tool known for low false-positive rates and moderate accuracy, served as the "source of truth" and baseline for comparison.
• Experimental Agent: Snitch, a Python-based agent application designed to recursively scan directories. Snitch leverages Ollama to host and run local LLMs.
• Models Evaluated: Three distinct open-source, general-purpose models were tested:
  1. Gemma3 (Google): Marketed as a coding specialist.
  2. Llama3.1 (Meta): A general-purpose LLM.
  3. Qwen2.5 (Alibaba): Intended for reasoning, coding, and debugging.
• Test Targets: Three open-source Python repositories with varying complexity and maturity:
  1. Yum: A retired, enterprise-grade package manager (high maturity).
  2. Beaverhabits: A personal habit tracker (casual commercial).
  3. Fail2ban: A security-oriented service daemon.
• Experimental Design:
  • Bandit scanned all three repositories first to establish a baseline of Medium and High severity findings.
  • Snitch then scanned the same repositories using each of the three models (9 total agent scans).
  • Prompting: A fixed, detailed prompt was used for all agent runs, instructing the model to act as an expert cybersecurity engineer identifying specific SAST issues and outputting results in a structured JSON format.
• Evaluation Metrics:
  • Recall: The ratio of Bandit's Medium/High findings also detected by the agent (partial matches weighted at 0.5).
  • False Positive (FP) Rate: The ratio of agent-flagged issues that were not flagged by Bandit and were determined to be contextual misunderstandings or hallucinations upon human review.
  • Composite Score: Calculated as Recall - FP Rate.
  • Execution Time: Recorded to compare efficiency against Bandit.

Key Results

The empirical data indicates that the evaluated open-source LLM agents significantly underperformed compared to the traditional SAST tool.

• Low Recall: No model achieved a recall score exceeding 0.25. The agents missed the vast majority of vulnerabilities identified by Bandit.
• High False Positives: All models exhibited extremely high false-positive rates, ranging from 40% to 90%.
  • Common hallucinations included flagging standard library constants (e.g., calendar.MONDAY) as hardcoded secrets, misinterpreting .env file usage as production leaks, and identifying benign code patterns (like comments or license headers) as vulnerabilities.
  • The Gemma3 model had the highest FP rate, while Qwen2.5 had the lowest, though still substantial.
• Hallucinations and Accuracy:
  • The agents frequently hallucinated the location of vulnerabilities (file names and line numbers), often referencing non-existent files or code blocks.
  • While the agents occasionally identified unique issues missed by Bandit (e.g., specific insecure requests.get usage or path traversals), these were unverifiable as "true positives" without external ground truth and were outweighed by the noise.
• Performance and Efficiency:
  • Bandit completed scans in under one minute per repository.
  • Snitch (LLM agents) required 1 to 4 hours per repository, depending on the codebase size.
  • The total runtime for all agent scans across the three models was approximately 10–12 hours, compared to minutes for Bandit.

Key Contributions

1. Empirical Assessment: The paper provides a rigorous, data-driven evaluation of agentic AI for SAST, moving beyond theoretical speculation to measurable performance metrics (precision, recall, FP rate).
2. Baseline Comparison: It establishes a direct performance baseline using Bandit, demonstrating that general-purpose open-source LLMs currently lack the reliability required for specialized security scanning tasks.
3. Identification of Limitations: The study highlights specific failure modes of LLM agents in this domain, particularly the "hallucination" of vulnerability locations and the misinterpretation of standard development patterns (e.g., environment variables) as security flaws.
4. Composite Scoring: The introduction of a composite score (Recall - FP Rate) to holistically evaluate the trade-off between missed vulnerabilities and noise generation.

Significance and Claims

The paper concludes that modern open-source GenAI LLM-based agents are not currently suitable for the specialized task of SAST scanning under realistic conditions.

• Reliability Gap: The high false-positive rates and low recall render these agents unreliable as standalone security tools. The "hallucinations" regarding vulnerability locations make the output nearly worthless for automated remediation, as developers cannot trust the specific file or line number provided.
• Efficiency Barrier: The computational cost and time required for LLM agents (hours vs. minutes) present a prohibitive barrier for integration into standard CI/CD pipelines compared to rule-based tools.
• Role of AI: The authors suggest that while agentic AI shows promise for code generation and assistance, it currently lacks the precision and explainability required for validation. The paper argues that traditional rule-based tools like Bandit remain superior for code analysis due to their consistency, speed, and traceability.
• Future Outlook: The authors posit that while closed-source or specialized models with advanced prompt engineering might improve results, the fundamental limitations of generative systems (hallucination, lack of external knowledge bases like CVE definitions) currently prevent them from replacing established SAST tools. They suggest a potential hybrid approach where agents act as supplements to traditional tools, focusing only on novel issues, rather than replacements.