Semantic-Aware Advanced Persistent Threat Detection Using Autoencoders on LLM-Encoded System Logs

Technical Summary: Semantic-Aware APT Detection Using Autoencoders on LLM-Encoded System Logs

Problem Statement
Advanced Persistent Threats (APTs) represent a critical challenge in cybersecurity due to their stealthy, long-term, and "low-and-slow" nature. Attackers often utilize "living off the land" techniques, leveraging legitimate system tools (e.g., PowerShell, WMI) to blend malicious activities with normal administrative tasks. Traditional detection methods, including statistical approaches and shallow machine learning techniques like Isolation Forest (IForest), One-Class Support Vector Machines (OC-SVM), and Principal Component Analysis (PCA), rely heavily on token frequency or statistical properties. These "shallow" methods fail to capture the semantic intent and contextual relationships within system logs. Consequently, they struggle to distinguish between benign administrative actions and sophisticated attacks when the statistical probability of the log entries is similar. Furthermore, the sheer volume of unstructured system logs makes manual analysis impossible, necessitating automated anomaly detection that understands the meaning behind the data rather than just its frequency.

Methodology: The MPNet-AE Framework
The paper proposes MPNet-AE, an unsupervised anomaly detection framework that bridges the gap between raw system logs and semantic understanding. The methodology proceeds through three primary stages:

1. Data Provenance and Textual Conversion:
   The framework utilizes the DARPA Transparent Computing (TC) dataset, which contains fine-grained provenance traces (process executions, system calls, file accesses, network communications). These raw logs are normalized and transformed into structured natural language descriptions. For example, a process record is converted into a sentence such as: "Process 1054 started /bin/bash and connected socket 192.168.1.5:80 and changed /etc/passwd." This conversion captures contextual relationships (e.g., parent-child process links, network interactions) that are lost in raw log formats.

2. Semantic Embedding Generation:
   Instead of using standard token-based features or earlier BERT variants, the system employs the pre-trained all-mpnet-base-v2 model (based on Microsoft's MPNet architecture). This model, which combines Masked Language Modeling (MLM) and Permuted Language Modeling (PLM) with contrastive fine-tuning, encodes the generated natural language sentences into high-dimensional (768-dimensional) dense semantic vectors. These embeddings represent the high-level behavioral semantics of the system activities rather than just surface-level keywords.

3. Anomaly Detection via Autoencoder:
   The semantic embeddings are fed into a vanilla Autoencoder (AE). The AE is trained exclusively on embeddings derived from benign (normal) system behaviors. The architecture consists of an encoder that compresses the 768-dimensional input into a 128-dimensional latent space, and a decoder that reconstructs the original input.

   • Detection Mechanism: During inference, the model calculates the reconstruction error (Mean Squared Error) for each sample. Since the AE is trained only on normal data, it reconstructs benign patterns with low error. Malicious or anomalous activities, which possess different semantic structures, result in high reconstruction errors. A threshold is applied to these errors to classify samples as anomalies.

Key Contributions

1. Novel Framework: The authors propose a transition from statistical to semantic anomaly detection by integrating pre-trained transformer models (specifically MPNet) with autoencoders to detect APTs based on semantic differences rather than frequency.
2. Comparative Evaluation: The study provides a rigorous comparative evaluation against three established unsupervised baselines: Isolation Forest, OC-SVM, and PCA.
3. Dataset Utilization: The research leverages the DARPA TC project datasets (including 5DIR, CADETS, CLEARSCOPE, THEIA, and TRACE scenarios), which are recognized benchmarks containing realistic, highly imbalanced APT attack scenarios generated by red teams.

Experimental Results
The framework was evaluated on five distinct provenance contexts (ProcessEvent, ProcessExec, ProcessParent, ProcessNetflow, and ProcessAll) across multiple operating systems (BSD, Windows, Linux, Android).

• Performance Metrics: Performance was measured using the Area Under the Receiver Operating Characteristic Curve (AUC-ROC), a metric suitable for highly skewed data where malicious events constitute less than 0.004% of the records.
• Comparative Advantage: The MPNet-AE approach consistently outperformed the traditional baselines (IForest, OC-SVM, PCA). In 11 out of 23 experimental conditions, the model achieved AUC scores higher than 0.95.
• Robustness: Visualizations using t-SNE confirmed that normal activities form compact clusters while anomalies appear as isolated points in the semantic space. The model demonstrated particular robustness in complex, noisy scenarios (e.g., TRACE ProcessNetflow) where traditional methods like OC-SVM suffered performance drops.
• Convergence: Training curves indicated rapid convergence with minimal overfitting, validating the model's ability to generalize normal behavior patterns.

Significance and Claims
The paper asserts that the integration of semantic understanding with reconstruction-based modeling offers a decisive advantage in APT detection. By capturing the intent and context of system activities, the MPNet-AE framework can identify subtle, non-linear, and stealthy attack behaviors that are often missed by conventional techniques relying on statistical properties. The authors claim that this approach effectively addresses the limitations of "shallow" learning in the face of sophisticated, low-frequency threats.

The study concludes that while the current framework is effective, future work should focus on fine-tuning language models specifically on cybersecurity logs, exploring temporal and graph-based dependencies for multi-stage attacks, and integrating explainability mechanisms for practical deployment in live enterprise environments.