🤖 machine learning

TACTIC-KG: Toward Small Agent Teams for Cyber Threat Intelligence Knowledge Graph Construction

TACTIC-KG is a cost-effective agentic framework that constructs Cybersecurity Knowledge Graphs from unstructured threat reports by decomposing the task into specialized roles for lightweight LLMs, thereby outperforming monolithic large models in accuracy, stability, and efficiency.

Original authors: Mouhamed Amine Bouchiha, Gregory Blanc

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mouhamed Amine Bouchiha, Gregory Blanc

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a massive, detailed map of a city, but instead of having blueprints, you only have thousands of messy, handwritten diary entries from different people. Some entries are clear, some are scribbled, some are contradictory, and some are just made up. This is what cybersecurity experts face when they try to turn Cyber Threat Intelligence (CTI) reports into Knowledge Graphs. These reports describe hackers, their tools, and their attacks, but they are written in unstructured, messy text.

The paper introduces a new system called TACTIC-KG to solve this problem. Here is how it works, using simple analogies:

The Old Way: The "Super-Genius" Solo Artist

Previously, researchers tried to use one giant, super-powerful AI (a "monolithic" model) to read the messy diary, understand the story, find the names of hackers and tools, figure out how they are connected, and draw the map all at once.

Think of this like hiring one Super-Genius Artist to do the entire job alone.

The Problem: Even geniuses get tired. When the story is too long or too complex, the Super-Genius starts to hallucinate. They might invent a connection between two people that never existed, or they might forget a key detail because they were trying to do too many things at once.
The Cost: This Super-Genius is also incredibly expensive to hire and slow to work.

The New Way: The "Specialized Team" (TACTIC-KG)

The authors of this paper say, "Why rely on one overworked genius? Let's hire a small team of specialized workers."

TACTIC-KG breaks the job down into four distinct roles, each handled by a smaller, cheaper, and more focused AI agent. They work in a line, passing the work down the chain:

The Extractor (The Scavenger):
- Job: This agent just looks at the text and pulls out raw pieces of information (like "Pegasus," "iOS," "NSO Group"). It doesn't worry about what they mean yet; it just grabs the facts.
- Analogy: Like a scavenger bird picking up shiny objects from the ground. It grabs everything that looks interesting without judging if it's a diamond or a piece of glass.
The Typer (The Labeler):
- Job: This agent takes the shiny objects the scavenger found and puts them in the right boxes. Is "Pegasus" a Malware? Is "NSO Group" an Organization?
- Analogy: Like a librarian who takes the books the scavenger brought and puts them on the correct shelves (Fiction, History, Science).
The Verifier (The Fact-Checker):
- Job: This is the most important new step. This agent looks at the labeled items and asks, "Did the original diary actually say this?" If the scavenger and labeler made a guess that isn't supported by the text, the Verifier throws it out.
- Analogy: Like a strict editor who checks a news story. If the reporter claims "The suspect was in Paris," but the source text only says "The suspect was in Europe," the editor cuts the specific claim. This stops the system from making things up (hallucinations).
The Curator (The Map-Maker):
- Job: This agent takes the verified facts and connects them into a final, clean map. It fixes small errors, merges duplicate names (like "TrickBot" and "Trick Bot"), and ensures the map makes logical sense without adding fake bridges.
- Analogy: Like a cartographer drawing the final map, ensuring the roads connect logically and removing any bridges that lead nowhere.

Why This Team is Better

The paper claims that this team approach is superior to the "Super-Genius" for three main reasons:

Less Hallucination: Because the Verifier acts as a strict gatekeeper, the team doesn't accidentally invent fake connections. The "Super-Genius" often tries to be too helpful and fills in the blanks with guesses; this team only fills in blanks if the text explicitly supports it.
Better Recall (Finding More): The "Scavenger" is allowed to grab everything it sees, even if it's unsure. Then the "Fact-Checker" sorts the good from the bad. This means they miss fewer real threats than the single model, which might get overwhelmed and skip details.
Cheaper and Faster: Instead of using one massive, expensive computer brain, the team uses several smaller, lightweight brains (3 billion to 8 billion parameters). It's like using a team of efficient interns rather than one highly paid, slow executive.

The Results

When the authors tested this system on real-world cybersecurity reports, they found that:

The team of small agents was more accurate at finding and labeling threats than the giant single models.
The maps (Knowledge Graphs) they built were more consistent and had fewer fake connections.
They achieved these results using much smaller, cheaper computer models.

In short, TACTIC-KG proves that for complex tasks like mapping cyber threats, a well-organized team of specialists is better than one overworked superstar.

Technical Summary: TACTIC-KG

Problem Definition

The construction of Cybersecurity Knowledge Graphs (CSKGs) from Cyber Threat Intelligence (CTI) reports faces significant challenges due to the unstructured, heterogeneous, and noisy nature of the source text. While Large Language Models (LLMs) have been applied to automate this extraction, monolithic approaches relying on single, large models often suffer from high deployment costs, limited controllability, and unstable performance. Specifically, these systems struggle with:

Hallucination and Faithfulness: Generating plausible but unsupported triples (e.g., inferring attacker roles or temporal links not explicitly stated).
Over-Completion: Artificially forcing graph connectivity to create complete attack chains, even when evidence is partial or disconnected.
Context-Size Trade-offs: Large context windows can dilute attention in lightweight models, while small windows lead to omission errors.

Existing state-of-the-art systems (e.g., CTINEXUS) often rely on In-Context Learning (ICL) with massive models (e.g., 671B parameters) or complex retrieval-augmented generation (RAG) pipelines, which can be brittle, expensive, and sensitive to retrieval quality.

Methodology: TACTIC-KG Framework

The paper introduces TACTIC-KG, an agentic framework that reformulates CSKG construction as a controlled, multi-stage reasoning process. Instead of a single monolithic inference task, the framework decomposes the workflow into specialized, lightweight LLM agents (3B–8B parameters) using Low-Rank Adaptation (LoRA). The core design principle is "faithfulness before connectivity."

Workflow Architecture

The pipeline processes raw CTI reports through the following stages:

Semantic Chunking: Long reports are segmented at sentence or discourse boundaries with controlled overlap to preserve local semantic coherence and prevent context dilution.
Extractor Agent: Generates candidate triples $(h, r, t)$ grounded strictly in the text chunks. It performs no typing or global reasoning, preventing premature abstraction.
Typer Agent: Assigns ontology-compliant semantic types (e.g., MALOnt, STIX 2.1) to entities based on local context and relation semantics. It operates within a closed label space to prevent ontology drift.
Verifier Agent: Acts as a filtering stage, performing binary classification to determine if a typed triplet is SUPPORTED or NOT_SUPPORTED by explicit or implicit textual evidence. It enforces a closed-world assumption, rejecting plausible but unstated relations.
Curator Agent: Performs final sanitization, including deduplication, entity normalization (alias resolution), and minimal structural repair. It introduces new edges only when logically necessary and supported by evidence, acting as a structural firewall against hallucinations.

Technical Implementation

Model Specialization: The framework utilizes lightweight models (e.g., Ministral-3B, Ministral-8B, Qwen3-8B) fine-tuned with LoRA. This allows for modular, agent-specific adapters that reduce GPU memory footprints and enable independent upgrades.
Training Strategy:
- Extractors are trained on chunked reports with strict JSON output formats.
- Typers are trained under a closed label space embedded in prompts.
- Verifiers are trained as triplet-level binary classifiers on annotated corpora to distinguish supported from unsupported relations.
- Curators are trained to predict minimal sets of bridging triplets to connect graph components without hallucinating unsupported edges.

Key Contributions

Agentic Decomposition: The paper proposes shifting from monolithic generation to a multi-agent system where extraction, typing, verification, and curation are handled by specialized agents. This reduces task entropy and localizes hallucination risks.
Faithfulness Enforcement: By explicitly separating generation from verification and enforcing a closed-world assumption at the triplet level, TACTIC-KG significantly reduces hallucinated relations and over-completion.
Cost-Efficient Performance: The framework demonstrates that lightweight models (3B–8B), when specialized via LoRA and orchestrated in a multi-agent pipeline, can outperform or match much larger monolithic models (e.g., DeepSeek-V3.1 671B) in extraction quality and graph consistency.
Ontology Grounding: The system integrates ontology constraints (MALOnt) directly into the typing and verification stages, ensuring the resulting graphs are structurally consistent and actionable.

Experimental Results

The framework was evaluated on human-annotated CTI reports (CTI-HAL and CTINEXUS benchmarks) against state-of-the-art baselines, including CTINEXUS with various large models (DeepSeek-V3.1, Kimi-K2, Devstral-2).

Extraction Quality: TACTIC-KG achieved higher F1-scores and significantly improved recall compared to monolithic baselines. For instance, on TEST2, the Ministral-3-8B variant achieved an F1 of 78 and Graph Similarity of 63, outperforming DeepSeek-V3.1 (F1: 66, Graph: 52).
Recall vs. Precision: The agent specialization allowed for explicit control over the precision-recall trade-off. Extractors achieved high recall (up to 90%), while verifiers and curators maintained stable precision, filtering out hallucinations without sacrificing coverage.
Typing and Grounding: TACTIC-KG demonstrated superior typing accuracy (Partial Typing Accuracy up to 72% vs. 69% for DeepSeek) and better semantic grounding, attributed to reduced cognitive load on individual agents and ontology-aware verification.
Data Efficiency: The framework required only moderate training data (approximately 200 chunks) to reach strong performance, with data diversity proving more critical than sheer volume.
Hybrid Design: Experiments showed that using a shared, stronger reasoning model for verification and curation (while keeping extraction specialized) further enhanced stability and consistency.

Significance and Claims

The paper claims that TACTIC-KG offers a robust, cost-effective alternative to monolithic LLM approaches for CSKG construction. By treating the problem as a controlled multi-agent reasoning process, the framework:

Minimizes Hallucination: Explicit verification and curation stages prevent the propagation of unsupported facts.
Improves Stability: Modular agents reduce the variance in performance often seen with large, single-pass models.
Enables Deployment: The use of lightweight models makes the system feasible for organizations with limited computational resources, including potential enterprise applications where data privacy requires local deployment.

The authors conclude that while full typing accuracy remains a challenge across all systems, the decomposition of tasks into specialized agents provides a superior balance of accuracy, stability, and cost-efficiency for constructing actionable Cyber Threat Intelligence Knowledge Graphs.