TIJERE: A Novel Threat Intelligence Joint Extraction Model Based on Analyst Expert Knowledge

Technical Summary: TIJERE – A Novel Threat Intelligence Joint Extraction Model Based on Analyst Expert Knowledge

Problem Statement
The extraction of structured threat intelligence from unstructured reports is critical for constructing Cybersecurity Knowledge Graphs (CKGs) to automate threat analysis. This process relies on Named Entity Recognition (NER) and Relation Extraction (RE). While Joint Extraction (JE) models offer theoretical advantages over Pipeline Extraction (PE) by reducing error propagation, existing JE methods in the cybersecurity domain face significant hurdles:

1. Overlapping Relations: Sentences often contain multiple entity pairs involving the same entities in different roles, which standard tagging schemes struggle to disambiguate.
2. Feature Confusion: When entity and relation features are not clearly separated, models struggle to learn both tasks simultaneously, particularly when sentence duplication is used to handle overlaps, introducing noise.
3. Ambiguous Relation Types: Analysts often use non-vocabulary expressions (e.g., "hasAttackTime") that lack semantic clarity for generic models.
4. Domain Language Ambiguity: Generic language models fail to capture cybersecurity-specific jargon (e.g., "Mimikatz"), leading to poor entity detection and noise propagation.
5. Data Scarcity: There is a lack of publicly available, jointly labeled datasets specifically designed for cybersecurity NER and RE.

Methodology: The TIJERE Framework
The authors propose TIJERE, a data-driven JE framework that reformulates the extraction task as a Multisequence Labeling Representation (MSLR) problem. The architecture consists of the following key components:

• Multisequence Labeling Representation (MSLR): Instead of a single sequence per sentence, TIJERE generates a separate sequence for every valid entity pair within a sentence. This approach inherently handles overlapping relations by isolating the context for each specific relation prediction.
• Expert Domain Features (EDFs): To resolve feature confusion and semantic ambiguity, the model integrates two expert-driven features into the MSLR:
  • Entity Type: Encodes the semantic category of the subject and object (e.g., "HackOrg" and "Tool"). This provides the model with domain-specific context, mimicking analyst reasoning (e.g., a hacker group using a tool implies a "uses" relation).
  • Entity Mask: A vector-based representation that highlights the specific tokens belonging to the entity pair under consideration, guiding the model to focus on relevant positional information.
• SecureBERT+ Embedding: The framework utilizes SecureBERT+, a variant of SecureBERT fine-tuned on cybersecurity text, to generate dynamic embeddings that capture domain-specific jargon and reduce language ambiguity.
• Model Architecture:
  • Encoder: SecureBERT+ followed by a BiGRU layer to capture sequential dependencies.
  • NER Head: A Dense layer and Conditional Random Field (CRF) layer for token-level entity tagging.
  • RE Head: An entity pooling mechanism aggregates token embeddings based on the entity mask, concatenates them with static entity type embeddings, and passes them through a Dense layer with Softmax for relation classification.
  • Loss Function: A weighted sum of NER loss (CRF) and RE loss (Cross-Entropy) is used for joint optimization.

Key Contributions

1. TIJERE Model: A novel architecture that combines MSLR with expert-driven features (entity type and mask) to address overlapping relations and feature confusion without relying on external rule-based modules that mimic pipeline approaches.
2. DNRTI-JE Dataset: The introduction of the first publicly available, jointly labeled dataset for cybersecurity NER and RE. It extends the original DNRTI dataset with 15 predefined relation types and a domain ontology, covering 6,592 sentences describing Advanced Persistent Threat (APT) groups.
3. Empirical Validation: A comprehensive evaluation demonstrating that TIJERE outperforms existing state-of-the-art JE and PE methods on the cybersecurity domain.

Experimental Results
Evaluated on the DNRTI-JE dataset, the TIJERE model (specifically the SecureBERT+-BiGRU-CRF variant with EDFs enabled) achieved state-of-the-art performance:

• NER Performance: F1-score of 0.93 (Precision: 0.93, Recall: 0.94).
• RE Performance: F1-score of 0.98 (Precision: 0.98, Recall: 0.99).
• Ablation Studies:
  • Enabling EDFs significantly improved RE performance (e.g., BERT-CRF RE F1 jumped from 0.44 to 0.97), while NER performance remained stable.
  • The "Entity Type" feature alone provided substantial gains, but the combination of Entity Type and Entity Mask yielded the highest accuracy, confirming the complementary nature of semantic and positional cues.
  • SecureBERT+ variants consistently outperformed standard BERT and RoBERTa models, highlighting the importance of domain-specific pretraining.

Significance and Claims
The paper claims that TIJERE represents the first data-driven approach specifically designed to address the unique challenges of joint extraction in cybersecurity. By integrating analyst expert knowledge directly into the model's feature representation (via EDFs) and utilizing a multisequence labeling strategy, TIJERE effectively mitigates feature confusion and overlapping relation issues that plague existing methods.

The authors assert that the combination of TIJERE and the standardized DNRTI-JE benchmark enables high-performance intelligence extraction with transferable potential to other domains requiring structured information extraction, such as healthcare, finance, and bioinformatics. The work emphasizes that while the approach requires expert knowledge for feature engineering and data modeling, it offers a robust, efficient alternative to complex, rule-heavy or computationally expensive existing solutions, facilitating the construction of high-quality CKGs for proactive defense.