OntoLogX: Ontology-Guided Knowledge Graph Extraction from Cybersecurity Logs with Large Language Models

1. Problem Statement

Cybersecurity system logs are a critical source of Cyber Threat Intelligence (CTI), containing evidence of attacker behaviors, vulnerabilities, and malicious traces. However, extracting actionable intelligence from these logs is hindered by several challenges:

• Unstructured and Heterogeneous Data: Logs vary significantly in format, syntax, and semantics across different devices and sessions.
• Fragmentation: Attack scenarios are often split across multiple log entries, making it difficult to reconstruct the full narrative.
• Limitations of Traditional Methods: Rule-based parsers and heuristics lack the adaptability to generalize across evolving threat behaviors and diverse log formats.
• LLM Limitations: While Large Language Models (LLMs) excel at natural language processing, they often lack factual consistency, domain-specific grounding, and the ability to produce structurally valid outputs without heavy user intervention or pre-processing.

The core problem is the need for an autonomous system that can transform noisy, raw log data into coherent, semantically consistent, and interoperable Knowledge Graphs (KGs) without manual intervention, while ensuring the output adheres to strict domain ontologies.

2. Methodology: The OntoLogX Framework

OntoLogX is an autonomous AI agent designed to process log events sequentially and incrementally. It leverages LLMs, Retrieval-Augmented Generation (RAG), and iterative validation to construct ontology-grounded KGs. The workflow consists of four main stages:

A. Ontology Design

The system uses a lightweight, custom log ontology tailored for cybersecurity.

• Structure: It defines core classes such as Event, Source (device/application), Parameter (e.g., timestamps, network addresses, users), and UserCredential.
• Alignment: It maps to standard provenance ontologies (e.g., prov-o) and W3C time ontologies to ensure interoperability.
• Validation: A companion SHACL (Shapes Constraint Language) schema enforces constraints on property cardinalities, data types, and required fields to ensure syntactic and semantic validity.

B. Hybrid Retrieval (RAG)

Before generating a KG for a new log event, OntoLogX retrieves semantically similar past examples to serve as few-shot prompts.

• Hybrid Search: It combines vector search (for semantic similarity) and full-text search (for exact keyword matching) against a graph database of previously generated KGs.
• Re-ranking: The system uses Maximal Marginal Relevance (MMR) to balance relevance with diversity, ensuring the retrieved examples cover a broad range of patterns rather than clustering around a single interpretation.

C. Ontology-Guided Generation

The LLM generates a candidate KG based on the raw log, optional context (e.g., device type), and the retrieved examples.

• Structured Output: The model is constrained to output in a predefined schema (e.g., JSON or Turtle format) defined by the ontology, preventing free-form text generation.
• Model Agnosticism: The prompt design is generic, allowing compatibility with various LLM backends.

D. Iterative Validation and Correction

To mitigate LLM hallucinations and structural errors, OntoLogX employs a self-correction loop:

1. Syntactic Check: Verifies the output format.
2. Ontology Compliance: Runs the output against SHACL rules.
3. Semantic Validation: Checks for logical inconsistencies (e.g., missing Event nodes, undefined relationships).
4. Correction Prompt: If violations are found, the system generates a targeted prompt to the LLM to fix specific errors. This loop repeats (up to 3 times) until a valid KG is produced or the system defaults to an empty graph.

E. Downstream Application: Tactics Prediction

Once KGs are generated and grouped into sessions (based on temporal or logical proximity), a final LLM call analyzes the aggregated graph to predict MITRE ATT&CK tactics. This bridges low-level log evidence with high-level adversarial objectives.

3. Key Contributions

1. Lightweight Log Ontology & SHACL Schema: A domain-specific schema designed to balance expressiveness with the simplicity required for automated extraction from raw logs.
2. OntoLogX Agent: A retrieval-augmented, LLM-based agent that autonomously transforms raw logs into ontology-compliant KGs without human intervention.
3. Iterative Correction Mechanism: A novel feedback loop using SHACL validation to iteratively refine LLM outputs, significantly improving structural accuracy.
4. Empirical Evaluation: A comprehensive study evaluating the system on both public benchmark datasets and a real-world honeypot dataset, including an ablation study of system components and a comparison across eight different LLMs.

4. Experimental Results

The authors evaluated OntoLogX using eight LLMs (including Llama 3.3, Claude 3.5, Mistral, and Qwen3 Coder) on two datasets: the AIT log dataset (heterogeneous, non-attack focused) and a Cowrie honeypot dataset (real-world adversarial activity).

• KG Construction Quality:

  • Performance: The full pipeline (Retrieval + Structured Output + Correction) achieved the highest Precision, Recall, and F1 scores.
  • Model Comparison: Claude Sonnet 4 and Qwen3 Coder (32B) performed best. Notably, code-specialized models (Qwen) and reasoning models showed varying success; reasoning models (e.g., gpt-oss) struggled with structured output tools, often failing to call the function correctly.
  • Ablation: Retrieval and correction mechanisms were crucial. Retrieval alone improved consistency, while correction significantly reduced SHACL violations.
  • Trade-off: There is a trade-off between G-Eval scores (semantic richness) and F1 scores (structural precision). The authors argue that a balanced approach (G-Eval ~0.7–0.8) is optimal to avoid noisy but semantically "rich" graphs.

• MITRE ATT&CK Tactics Prediction:

  • OntoLogX outperformed a baseline (LLM predicting tactics directly from raw logs).
  • Strengths: It showed superior Recall for complex, context-dependent tactics like Discovery, Credential Access, and Defense Evasion, which require multi-step reasoning over structured entities.
  • Limitations: Performance was lower for tactics requiring long-term correlation (e.g., Persistence), as these often rely on environmental context not fully captured in short log sessions.

5. Significance and Future Work

Significance:

• Automation: OntoLogX demonstrates that autonomous agents can effectively bridge the gap between unstructured logs and structured CTI, reducing the need for manual parsing and rule engineering.
• Interoperability: By grounding outputs in a standard ontology, the system enables semantic querying and integration with existing CTI frameworks.
• Actionable Intelligence: The ability to map low-level events to MITRE ATT&CK tactics provides security analysts with immediate, high-level threat context.

Limitations & Future Directions:

• Computational Cost: The reliance on LLMs introduces latency and cost, which may be challenging for high-throughput, real-time environments.
• Hallucinations: While RAG and validation reduce risks, hallucinations cannot be entirely eliminated.
• Scalability: Future work aims to optimize resource orchestration, explore incremental learning, and extend the ontology to cover diverse enterprise and industrial log sources.

In conclusion, OntoLogX represents a significant step forward in automated Cyber Threat Intelligence, proving that combining ontology-guided constraints with the generative power of LLMs can yield robust, interpretable, and actionable security insights.