SoK: Honeypots & LLMs, More Than the Sum of Their Parts?

1. Problem Statement

Cybersecurity honeypots have long faced a fundamental design paradox: Low-Interaction Honeypots are safe and easy to deploy but lack the realism (fidelity) to deceive sophisticated attackers, while High-Interaction Honeypots offer high fidelity but carry significant operational risk and management overhead.

The advent of Large Language Models (LLMs) promised to resolve this by enabling low-interaction decoys to generate dynamic, adaptive, and believable interactions. However, the field has rapidly evolved from ideation to prototype without a cohesive understanding of:

• The architectural patterns required to make LLM honeypots viable.
• The core challenges (e.g., latency, context limits, detection vectors).
• Standardized evaluation paradigms to measure efficacy.
• The specific threat landscape these systems are best suited to counter.

The paper argues that current evaluations show only incremental progress in real-world deployments, often hindered by a "data desert" of low-sophistication attack traffic that fails to stress-test the capabilities of LLMs against intelligent adversaries.

2. Methodology

The authors conducted a Systematization of Knowledge (SoK) to synthesize the emerging literature on LLM-powered honeypots. Their methodology involved:

• Literature Review: A comprehensive survey of papers from late 2022 to October 2025, focusing on the intersection of LLMs and honeypots.
• Taxonomy Construction: Developing a structured framework to categorize honeypot detection vectors and map LLM capabilities against them.
• Architectural Synthesis: Analyzing disparate implementations to identify a "canonical" architecture.
• Evaluation Framework Development: Creating a unified set of metrics (the "Evaluation Tetrad") to compare different systems.
• Threat Modeling: Proposing a new classification of adversaries (Trichotomy) to align honeypot design with the appropriate target threat.
• Gap Analysis: Identifying limitations in current research, particularly regarding data scarcity and the lack of open-source tooling.

3. Key Contributions

A. Taxonomy of Honeypot Detection Vectors

The paper identifies four persistent vectors attackers use to fingerprint honeypots, analyzing where LLMs can help and where they fundamentally fail:

1. Contents & Network Posture: Inconsistencies in banners, services, or file systems. LLM Aid: Can generate diverse, believable content. LLM Fail: Cannot obscure underlying network stack signatures (e.g., TCP/IP behavior) or open ports.
2. Outputs & Behavior: Unrealistic response content or metadata (e.g., timing anomalies, static output for dynamic commands like top). LLM Aid: Can provide context-aware responses. LLM Fail: Cannot reproduce real-time temporal fidelity or dynamic system states (e.g., vim or htop behavior) without complex custom architectures.
3. Functional Limitations: Inability to perform actual system actions (e.g., outbound network traffic, file downloads). LLM Aid: Can simulate success messages. LLM Fail: Cannot generate actual traffic or execute binaries; attackers can detect the lack of real egress.
4. Feature Synthesis: Machine learning classifiers combining multiple vectors. LLM Impact: Increases feature-space complexity, potentially breaking static signatures, but combined vectors still increase detection confidence.

B. Canonical LLM-Honeypot Architecture

The authors synthesize a Canonical Architecture (Figure 1) that has emerged from recent research, consisting of:

• Filter/Router: A critical pre-LLM component to block scanners, prevent prompt injection, and cache deterministic responses to reduce cost/latency.
• Prompt Creator & State Manager: Modules that curate session history (pruning context) and construct rich, context-aware prompts.
• LLM Engine: The core generation component (potentially using multiple models or LoRA adapters for different protocols).
• Deterministic Responder: Handles simple commands without invoking the LLM to save resources.
• Gap Identified: No comprehensive, open-source implementation of this full architecture currently exists.

C. The Evaluation Tetrad & Attacker Trichotomy

To address the lack of standardized metrics, the paper proposes:

• Evaluation Tetrad:
  1. Believability & Deception: Measured via human studies or semantic similarity.
  2. Fidelity: Measured via latency, Levenshtein distance, and command success rates.
  3. Attacker Cost & Intelligence: Measured by session length and diversity of elicited MITRE ATT&CK techniques.
  4. Defender Cost: Token consumption, API latency, and hardware overhead.
• Attacker Trichotomy:
  • Tier 1 (Scripted Bots): Low sophistication. LLMs are likely overkill; static decoys suffice.
  • Tier 2 (Autonomous LLM-Agents): The primary target. These agents reason and adapt. LLM honeypots are uniquely suited to waste their resources and gather intelligence.
  • Tier 3 (Skilled Humans): LLMs alone are insufficient; high-interaction real systems are still required.

D. Evolution of Log Analysis

The paper charts the "Semantic Leap" in honeypot log analysis:

• Phase 1: Data reduction (clustering).
• Phase 2: Session classification (labeling high/low impact).
• Phase 3: Automated mapping to MITRE ATT&CK (using LLMs to translate raw commands to TTPs).
• Phase 4: Operationalization via Agents: Using RAG (Retrieval-Augmented Generation) systems to feed honeypot-derived intelligence into real-time production detection systems.

4. Results & Findings

• Incremental Progress: Real-world deployments show marginal improvements in engagement (e.g., session length increasing from ~3 to ~6 commands) because 99% of traffic consists of low-sophistication scanning scripts that do not stress the LLM.
• The Data Desert: There is a critical lack of high-quality, labeled data involving sophisticated automated attackers. Most studies rely on human user tests (which are too smart) or script replay (which are too dumb), making it difficult to iterate on LLM efficacy.
• Architecture is Key: Simple "prompt-and-hope" approaches fail. Successful systems require hybrid architectures with filtering, context pruning, and deterministic caching to manage costs and security.
• LLM Limitations: LLMs struggle with commands requiring stateful interaction (e.g., vim), continuous output (e.g., top), and outbound network actions. These remain fundamental detection vectors.
• Target Shift: The field must pivot from trying to trick humans or simple bots to countering Tier 2 Autonomous LLM-Agents. Against these agents, LLM honeypots offer the best potential for "Denial-of-Wallet" (DOW) attacks and long-term occupation.

5. Significance & Future Roadmap

This paper serves as the first comprehensive foundation for the field, moving it from fragmented prototypes to a structured research domain. Its significance lies in:

1. Defining the Target: Shifting the focus to AI-vs-AI conflict, where LLM honeypots are the primary defense against autonomous attackers.
2. Standardizing Evaluation: Providing the "Evaluation Tetrad" to allow rigorous comparison of future systems.
3. Identifying the Path Forward:
   • Dynamic Adversarial Ecosystem: Creating a sandbox where LLM attackers and honeypots fight to generate high-quality training data.
   • Autonomous Feedback Loops: Developing systems that automatically reconfigure themselves based on real-time threat intelligence (e.g., using Reinforcement Learning to optimize deception strategies).
   • Open-Source Frameworks: Urging the community to build modular, open-source implementations of the canonical architecture.
   • Privacy-Preserving Models: Promoting the use of fine-tuned, locally hosted Small Language Models (SLMs) with LoRA adapters to balance fidelity, cost, and privacy.

In conclusion, the authors argue that the true potential of LLMs in honeypots is not just creating "better decoys," but building autonomous, self-improving deception systems capable of engaging the next generation of intelligent, automated cyber threats.