💬 NLP

TTPrint: Evidence-Grounded TTP Extraction via Diverge-then-Converge Verification

TTPrint is a novel framework that improves the extraction of MITRE ATT&CK techniques from cyber threat intelligence reports by employing a "diverge-then-converge" strategy to first broadly identify candidate behaviors anchored to specific evidence and then rigorously verify them against authoritative definitions, thereby significantly outperforming existing methods in both recall and precision.

Original authors: Yutong Cheng, Changze Li, Raihan Sultan Pasha Basuki, Qian Cui, Wei Ding, Peng Gao

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yutong Cheng, Changze Li, Raihan Sultan Pasha Basuki, Qian Cui, Wei Ding, Peng Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a massive, chaotic crime scene report. The report is hundreds of pages long, written in a mix of technical jargon and vague descriptions. Your job is to find specific "Modus Operandi" (how the criminals did it) and match them to a giant, official rulebook called MITRE ATT&CK.

The problem is tricky:

Don't miss anything: If you miss a technique, the criminals get away with a new trick.
Don't make things up: If you guess a technique that isn't actually there, you waste everyone's time chasing ghosts.

Existing methods (like old rulebooks or single-pass AI) struggle to do both. They either miss too many clues or guess too many wrong ones.

Enter TTPRINT, a new system that solves this by acting like a human detective team using a "Diverge-then-Converge" strategy. Think of it as a two-step process: Spread Wide first, then Narrow Down.

The TTPRINT Detective Workflow

Phase 1: The "Diverge" (Spreading Wide)

Instead of trying to guess the answer immediately, TTPRINT breaks the massive crime report into tiny, bite-sized pieces called "atomic behaviors."

The Analogy: Imagine taking a 50-page novel and cutting it into individual sentences.
What it does: For each tiny sentence, the AI asks, "What could this be?" It generates a long list of possible matches from the rulebook. It deliberately casts a wide net, even if it means suggesting some unlikely options. The goal here is Recall: make sure no real clue is left behind.

Phase 2: The "Anchor" (Finding the Evidence)

Before the AI can confirm a guess, it must prove where in the text the clue came from.

The Analogy: If you say, "The suspect used a crowbar," you must point to the exact sentence in the report that says, "The suspect used a crowbar." You can't just say, "I think they used a crowbar because they broke a window."
What it does: TTPRINT uses a strict, non-AI algorithm to find the specific "window" of text that supports the behavior. This anchors every guess to a specific piece of evidence, preventing the AI from hallucinating (making things up) based on the whole document's general vibe.

Phase 3: The "Converge" (Narrowing Down)

Now comes the rigorous verification. The AI takes the list of "possible" matches and the specific "evidence window."

The Analogy: A senior detective reviews the junior's list. They look at the specific sentence (the evidence) and the official rulebook definition side-by-side. They ask: "Does this sentence actually match the rulebook definition, or is it just a loose guess?"
What it does: The AI assigns a confidence score. If the match is weak, it's thrown out. If it's strong, it's kept. This step ensures Precision: only the facts that are truly supported by the text remain.

Why This Works Better

The paper claims that previous methods tried to do all these steps in one go (like a detective guessing the whole story in one breath). This leads to mistakes:

Too cautious: They miss subtle clues to avoid guessing wrong.
Too confident: They guess wrong because the wording in the report doesn't perfectly match the rulebook.

TTPRINT separates the "guessing" from the "checking." By splitting the job, it can be bold when guessing (catching everything) and strict when checking (keeping only the truth).

The Results

The researchers tested TTPRINT on two datasets:

TRAM-Clean: A cleaned-up version of an existing dataset (they fixed errors in the original labels).
TTPRINT-Bench: A brand new dataset of 150 full reports they created from scratch.

The Scorecard:

On the new dataset, TTPRINT got an 87.39% success rate (Macro-F1).
The best previous method (a standard AI using "Chain of Thought") only got 58.02%.
TTPRINT improved the score by nearly 30%, meaning it found significantly more real techniques while making far fewer mistakes.

The "Knob" for Control

One cool feature is a "threshold knob" (called $\tau$ ).

Turn it up: You get fewer results, but you are 100% sure they are right (High Precision). Good for automated systems that can't afford to make a mistake.
Turn it down: You get more results, catching almost everything, even if some might be shaky (High Recall). Good for security teams who want to catch every possible threat and investigate later.

Summary

TTPRINT is like a detective who first writes down every possible theory based on every sentence in a report, then rigorously checks each theory against the specific evidence before writing the final case file. This approach allows it to catch more real threats and make fewer false alarms than any previous method.

Note: The paper focuses strictly on extracting these techniques from English-language reports. It does not claim to predict future attacks, stop hackers in real-time, or function as a medical or legal diagnostic tool.

Technical Summary: TTPRINT

Problem Statement
Automatically extracting MITRE ATT&CK techniques from unstructured Cyber Threat Intelligence (CTI) reports is an open-set, multi-label problem requiring the simultaneous optimization of high recall (capturing all genuinely described techniques) and high precision (avoiding hallucinations of unsupported techniques). Existing approaches face significant limitations:

Rule-based and Supervised Methods: Systems like TTPDrill or fine-tuned transformers (e.g., LADDER, MITREtrieval) struggle with generalizability across diverse report styles and evolving attack descriptions. They often rely on explicit keywords or fixed ontologies, failing to capture implicit or paraphrased behaviors.
Single-Pass LLM Approaches: While Large Language Models (LLMs) show promise, applying them in a single inference step couples the objectives of hypothesis generation and verification. This coupling leads to two compounding failure modes: (1) models sacrifice coverage for confidence, omitting less salient behaviors, and (2) surface-level lexical mismatches between operational report language and canonical MITRE definitions cause incorrect technique mapping.

Methodology: TTPRINT
The authors propose TTPRINT, a four-stage pipeline inspired by human analyst workflows, utilizing a "diverge-then-converge" design to decouple recall-oriented generation from precision-oriented verification.

Divergent Phase (Maximize Coverage):
- Behavior Extraction: The raw CTI report is decomposed into a set of atomic, self-contained attack behaviors. Crucially, this stage preserves specific technical artifacts (tool names, command strings, registry paths) to prevent context collapse, ensuring downstream stages have sufficient detail for accurate identification.
- TTP Proposal: For each atomic behavior, the LLM proposes a broad set of candidate MITRE ATT&CK techniques (up to $k$ distinct parent-level identifiers). This stage prioritizes recall, deliberately deferring judgment to ensure no potential technique is prematurely discarded.
Convergent Phase (Filter by Evidence):
- Span Localization: A deterministic algorithm anchors each proposed behavior to a specific, contiguous sentence window (evidence span) within the original report. This restricts all subsequent reasoning to verifiable textual support rather than the full document context, reducing noise and hallucination risks.
- TTP Verification: The LLM evaluates each candidate technique against three inputs: the behavior text, the localized evidence span, and the official MITRE ATT&CK definition. The model outputs a confidence score ( $c_{ver}$ ) indicating whether the evidence explicitly supports the technique.
- Thresholding: Candidates exceeding a tunable confidence threshold ( $\tau$ ) are retained. This parameter allows practitioners to explicitly control the precision-recall trade-off without retraining or architectural changes.

Key Contributions

Architectural Principle: The paper identifies the coupling of precision and recall in single-pass extraction as a structural limitation and proposes the diverge-then-converge principle, explicitly separating hypothesis generation from evidence-grounded verification.
TTPRINT Pipeline: A novel implementation featuring deterministic span localization that ensures every prediction is traceable to a specific source passage, enabling full auditability.
Evaluation Resources:
- TRAM-Clean: A systematically cleaned version of the widely used TRAM benchmark, correcting false positives and negatives in the original annotations to provide a reliable baseline.
- TTPRINT-Bench: A new document-level benchmark comprising 150 full CTI reports (2022–2025) with double-annotated, adjudicated ground truth, designed to evaluate extraction from unstructured, full-length documents rather than isolated sentences.

Results
Evaluated on TRAM-Clean and TTPRINT-Bench, TTPRINT demonstrates significant improvements over state-of-the-art baselines:

Performance: TTPRINT achieves a Macro-F1 of 76.48% on TRAM-Clean and 87.39% on TTPRINT-Bench. This represents a 63.5% and 29.4% improvement over the leading baseline (CoT GPT-4o), respectively.
Precision and Recall: Unlike single-pass methods that struggle to balance both metrics, TTPRINT improves them simultaneously. On TTPRINT-Bench, recall increased from 50.70% to 95.50%, and precision from 69.90% to 82.15%.
Ablation Studies: Removing any stage (Behavior Extraction, Span Localization, or Verification) leads to significant performance degradation. Notably, span localization improves both precision and recall by reducing context dilution and hallucination risks.
Model Agnosticism: The pipeline's effectiveness holds across six different LLM backbones (including GPT-4o, Claude Sonnet, Gemini, Qwen, and DeepSeek). Smaller models equipped with the TTPRINT pipeline often outperform larger models using single-pass prompting, suggesting the architectural design is the primary driver of performance.
Threshold Sensitivity: The verification threshold $\tau$ provides a smooth, monotonic control over the precision-recall frontier, allowing users to select operating points for high-recall (SOC triage), balanced, or high-precision (automated response) scenarios.

Significance and Claims
The paper claims that TTPRINT addresses a critical gap in the CTI analyst workflow by providing a method for high-recall, precise TTP extraction that is directly traceable to source evidence. By decoupling generation and verification, the system mitigates the inherent trade-offs of single-pass LLM prompting. The authors emphasize that the approach is model-agnostic, allowing organizations with data sovereignty concerns to deploy locally hosted open-source LLMs without sacrificing performance. The release of TRAM-Clean and TTPRINT-Bench aims to elevate the field from sentence-level evaluation to realistic, document-level extraction, facilitating future research in automated threat intelligence analysis.

The authors maintain a modest scope, noting that the system targets English-language reports from major security vendors and does not currently resolve ambiguities inherent in the MITRE taxonomy itself (e.g., semantically adjacent techniques). They also note that while the extraction quality is validated against expert annotations, the impact on downstream operational metrics (e.g., triage time) requires future controlled deployment studies.