Context-aware Entity-Relation Extraction for Threat Intelligence Knowledge Graphs

Technical Summary: Context-aware Entity-Relation Extraction for Threat Intelligence Knowledge Graphs (CTiKG)

Problem Statement
The construction of Cybersecurity Knowledge Graphs (CKGs) is essential for unifying diverse Cyber Threat Intelligence (CTI) sources into structured, queryable formats that support proactive defense and real-time analysis. However, automating this process is hindered by the unstructured nature of CTI reports, which feature complex narratives, domain-specific jargon, and semantic ambiguity. Existing approaches typically fall into two categories: Joint Extraction (JE) and Pipeline Extraction (PE). While JE models integrate Named Entity Recognition (NER) and Relation Extraction (RE) to reduce error propagation, they often suffer from feature confusion, struggles with overlapping relations, and inflexibility in modular optimization. Conversely, PE approaches allow for modular design but are vulnerable to error propagation, where inaccuracies in entity recognition cascade into relation classification, degrading the overall quality of the knowledge graph. Furthermore, previous iterations of pipeline frameworks (such as the authors' prior work, TiKG) relied on generic language models or classifiers (e.g., TDD-softmax) that produced inconsistent tagging sequences and struggled with cybersecurity-specific terminology.

Methodology
The paper introduces the Context-aware Threat Intelligence Knowledge Graph (CTiKG) framework, a pipeline architecture designed to address the limitations of previous models through three distinct phases:

1. Data Collection and Processing: Reports are harvested from repositories (e.g., MITRE ATT&CK, APT repositories) and converted to plain text. Preprocessing includes sanitization (e.g., obfuscating URLs/emails) to simulate raw inputs.
2. Entity–Relation Extraction: This core phase employs two specialized modules:
   • Entity Recognition (NER): The authors propose a SecureBERT+-BiGRU-CRF architecture. This model replaces the previous TDD-softmax classifier with a Conditional Random Field (CRF) layer to enforce valid BIO tag transitions and improve sequence-level consistency. It utilizes SecureBERT+, a domain-adapted variant of BERT, to generate contextual embeddings, which are then processed by a Bidirectional Gated Recurrent Unit (BiGRU) to capture token dependencies before the CRF layer performs token-level classification.
   • Relation Extraction (RE): Recognizing that relation types are independent and lack sequential dependencies, the authors utilize a SecureBERT+-BiGRU-TDD (Time-Distributed Dense) architecture. This model replaces the CRF layer with a TDD layer to simplify the architecture and enable faster inference. Crucially, this phase integrates a domain-specific ontology (inspired by UCO 2.0) to guide entity-pair classification. The ontology defines 15 specific relation types (e.g., affiliatedWith, targets, hasVulnerability) and is used to filter out invalid entity pairs, thereby mitigating misclassification.
3. Construction and Analysis: Extracted triples are stored in Neo4j, where entities serve as nodes and relations as edges, enabling query-based retrieval and analysis of the constructed graph.

Key Contributions

• Novel NER Architecture: The introduction of a SecureBERT+-BiGRU-CRF model that achieves up to a 4% F1-score improvement across 21 entity categories by resolving the inconsistent tagging issues of previous softmax-based classifiers.
• Enhanced RE Architecture: A new SecureBERT+-BiGRU-TDD model that integrates domain-specific contextual embeddings with ontology-based error control, yielding up to an 8% improvement in F1-score for relation extraction.
• Dataset Augmentation and Release: The creation and release of DNRTI-AUG-STIX2, an augmented dataset derived from DNRTI-STIX2. This involved manual annotation of 1,367 additional sentences by security experts to address severe class imbalance and the consolidation of rare entity types (e.g., merging SHA1, SHA2, MD5 into a single HASH type). The dataset includes 21 entity types aligned with STIX 2.1.
• Comprehensive Benchmarking: Extensive evaluation on the DNRTI-AUG-STIX2 dataset, with additional validation on the DNRTI and STUCCO benchmarks, demonstrating the framework's robustness and generalizability.

Experimental Results
The proposed CTiKG framework was evaluated against state-of-the-art baselines (including BiLSTM-CRF, RoBERTa, and previous TiKG variants) using Precision, Recall, and F1-score metrics:

• NER Performance: The SecureBERT+-BiGRU-CRF model achieved an F1-score of 0.93 on DNRTI-AUG-STIX2, 0.94 on DNRTI, and 0.98 on STUCCO. This represents a 3–4% gain over previous baselines. The results indicate that while SecureBERT+ showed significant improvements in masked language modeling, the primary NER gains were driven by the CRF layer's ability to model sequence dependencies.
• RE Performance: The SecureBERT+-BiGRU-TDD model achieved an F1-score of 0.98 on DNRTI-AUG-STIX2, outperforming the previous TiKG RE module by 6–8% across all metrics. The study noted that CRF layers offered no measurable benefit over TDD for relation extraction, justifying the simplified architecture.
• Generalization: The model demonstrated consistent high performance across diverse datasets, including the vulnerability-focused STUCCO dataset, attributed to consistent linguistic patterns in vulnerability descriptions and the model's domain adaptation.

Significance and Claims
The paper claims that CTiKG establishes a robust, security-aware framework for threat analysis, profiling, and defense strategy development. By decoupling NER and RE while leveraging domain-specific embeddings and ontological constraints, the framework effectively mitigates the error propagation inherent in traditional pipeline approaches. The authors emphasize that the integration of SecureBERT+ and the domain ontology significantly enhances the precision and generalizability of CKGs, enabling more accurate transformation of unstructured CTI reports into structured intelligence. The release of the curated datasets and code on GitHub is highlighted as a key contribution to fostering reproducibility and further research in the field. The work is presented as an extension of the TiKG framework, specifically addressing its limitations in handling complex datasets and cybersecurity-specific jargon.