DEFENGRAPH: Knowledge Graph-Enhanced LLMs for Blue Team Cyber Defense

Technical Summary: DEFENGRAPH

Problem Statement
Large Language Models (LLMs) show promise for cybersecurity decision-making but suffer from critical limitations in high-stakes, time-evolving environments. Specifically, they are prone to hallucinations, struggle with temporal reasoning, and lack grounding in specific system contexts. Traditional Retrieval-Augmented Generation (RAG) approaches, which rely on surface-level or embedding-based similarity, often retrieve irrelevant documents (e.g., retrieving a Windows incident ticket for a Linux alert) that confuse LLM reasoning. Furthermore, cybersecurity data is characterized by a low signal-to-noise ratio, containing vast amounts of benign or misleading logs (e.g., mistyped passwords) that standard Knowledge Graph (KG) retrieval methods fail to filter effectively. The core challenge is to create a system that can ground LLM outputs in both long-term domain knowledge and real-time, evolving event contexts to provide faithful, temporally aware defense strategies.

Methodology: The DEFENGRAPH Framework
The authors propose DEFENGRAPH, a dual-layer Knowledge Graph-enhanced RAG framework designed to support Blue Team defenders. The system integrates structured knowledge representations with LLM-based generation through the following pipeline:

1. Dual-Layer Knowledge Graph Construction:

   • Static Knowledge Graph (SKG): Encodes long-term infrastructure knowledge, system architecture, typical attacker behaviors, and historical defensive playbooks derived from cyber range blueprints and documentation.
   • Dynamic Knowledge Graph (DKG): Captures time-evolving security events, attacker traces, and defender ticket actions recorded during live exercises. Nodes include alerts, tickets, IPs, and files, with edges encoding temporal and semantic relationships.

2. Graph-Based Retrieval Process (GRP):
   Upon receiving a SIEM (Wazuh) alert, the system constructs a query graph and retrieves relevant subgraphs from the SKG and DKG using:

   • Semantic-based Triple Matching (STM): Uses pre-trained language models to encode and match triples (entity, relation, entity) between the query and the knowledge base.
   • Path Propagation Score (PMC): Aggregates features from neighboring triples to compute path matching scores, identifying the most likely relevant paths.
   • Subgraph Composition: Merges top-matching paths into subgraphs, sampling 2-hop neighbors to capture surrounding context.

3. LLM-Based Contextual Filtering and Re-ranking:
   To address the noise inherent in retrieved subgraphs, DEFENGRAPH employs two LLM-driven modules:

   • Contextual Filter (CF): Evaluates the retrieved paths against the specific alert context to score and select the most relevant entities and paths, filtering out irrelevant "grey" nodes.
   • Reasoning Re-ranker: Re-orders the filtered paths based on their relevance to the alert and historical knowledge, incorporating Temporal Weight Assignment. This assigns higher weights to entities closer in time to the query, ensuring the system prioritizes recent, evolving threats.

4. Defense Activity Generation:
   The final, ranked reasoning paths are embedded into the LLM prompt to generate actionable, context-aware mitigation strategies that align with human defensive practices.

Key Contributions

• DEFENGRAPH Framework: A novel temporal KG-RAG framework that combines static domain knowledge with dynamic, real-time event data to generate context-aware defense strategies.
• ACDC Cyber Range Dataset: The construction and release of a comprehensive cybersecurity dataset derived from live Red vs. Blue team exercises in a simulated maritime port environment. It includes over 8.2M firewall logs, 6.1M Wazuh alerts, 326 incident response tickets, and Red Team traces, representing a realistic, noisy, and heterogeneous data source.
• Performance Gains: Demonstration that DEFENGRAPH significantly enhances the faithfulness and interpretability of LLM outputs across multiple models (GPT-4o, LLaMA-3, DeepSeek-R1, QWen-3), producing defense recommendations that align more closely with human-authored ground truth.

Experimental Results
The framework was evaluated on the ACDC dataset using four leading LLMs and compared against baselines including LLM-only, Hybrid RAG, and GraphRAG. Key findings include:

• Reasoning Recall (RAR): On GPT-4o, DEFENGRAPH improved reasoning recall from 61.45% (SKG-only) to 73.49% (Full). Similar gains were observed across other models (e.g., LLaMA-3: 46.99% → 61.45%).
• Ticket-Action Recall (TAR): The system recovered a significantly higher proportion of ground-truth defense actions. On GPT-4o, TAR increased from 52.17% to 72.46%.
• Precision and Fault Rates: While precision (TAP) saw moderate gains (24.49% → 29.24% on GPT-4o), the system maintained a low fault action rate (below 4%), with errors primarily consisting of over-reactions or false positives that are easily identifiable by human defenders.
• Cold-Start Robustness: In scenarios with no prior ticket history (reset DKG), the system maintained strong reasoning capabilities, though action recall dropped, highlighting the importance of dynamic context for precise action alignment.
• Ablation Studies: Results confirmed that both the Static and Dynamic KGs are essential; removing the DKG reduced TAR by ~20%, while removing the SKG caused a ~25% drop, demonstrating the necessity of both historical structure and real-time context.

Significance
The paper positions DEFENGRAPH as a state-of-the-art solution for grounded, temporally aware cybersecurity decision support. Unlike prior KG-based systems that focus on intrusion detection, attack tracing, or threat intelligence structuring, DEFENGRAPH directly targets the generation of actionable, ticket-aligned mitigation steps. By integrating a dual-layer graph with LLM-based filtering and re-ranking, the framework addresses the brittleness of traditional RAG in noisy, structured domains, enabling LLMs to provide faithful reasoning that reflects both the static system topology and the dynamic evolution of an attack. The authors assert that this approach bridges the gap between KG-enabled understanding and operationally useful response guidance for Blue Teams.