Technical Summary: A Decompilation-Driven Framework for Malware Detection with Large Language Models

Problem Statement

Traditional signature-based malware detection methods are increasingly ineffective against novel malware variants that utilize advanced obfuscation, polymorphic transformations, and unseen attack vectors. While Artificial Intelligence (AI) and Large Language Models (LLMs) have shown promise in code comprehension and semantic analysis, their direct application to decompiled executable code for malware classification remains an underexplored frontier. Previous studies often rely on hypothetical datasets or lack real-world applicability, and existing industry approaches frequently lack methodological transparency. Furthermore, there is a critical need to determine if LLMs can robustly replace or augment traditional anti-virus software in production environments, particularly given the rapid evolution of malicious coding patterns.

Methodology

The authors propose an automated pipeline that bridges low-level binary analysis and high-level semantic understanding by combining reverse engineering tools with state-of-the-art LLMs.

1. Decompilation Pipeline: The system utilizes the Ghidra disassembler to decompile Windows executable binaries into human-readable C code. This intermediate representation serves as the input for the LLMs.
2. Dataset Construction:
   • Baseline Dataset: Derived from the Malware Data Science book (2017), comprising 240 "Backdoor" malware samples and 365 benign samples. A subset was selected to ensure decompiled text did not exceed LLM token limits.
   • Contemporary Dataset: Compiled in 2025 from Malware Bazaar, containing 120 new malware samples across 49 families (e.g., LummaStealer, XWorm) and 28 benign system drivers. The benign set was intentionally selected for "malware-like" low-level kernel interactions to create a challenging classification boundary.
3. Classification Strategy:
   • Prompt Engineering: A standardized zero-shot prompt was engineered to instruct the LLM to act as a cybersecurity expert, analyze the C code for suspicious functions/APIs, apply rules (e.g., YARA), and output a strict JSON boolean (True/False).
   • Model Evaluation: Several "vanilla" (pre-trained) LLMs (Llama 3.3 70B, Codestral, Claude 3.7 Sonnet, Gemini 2.5 Pro) were evaluated against the baseline dataset.
   • Fine-Tuning: The best-performing vanilla model (Gemini 2.5 Pro) was supervised fine-tuned on a curated subset of 200 samples (100 smallest malware, 100 smallest benign) using Google Cloud Vertex AI over 40 epochs.
4. Safety Protocols: All analysis was conducted in isolated, air-gapped virtual machines with encrypted storage and snapshot reversion to prevent malware propagation.

Key Results

The evaluation compared vanilla LLMs, a fine-tuned LLM, and a traditional XGBoost classifier trained on static analysis features.

• Vanilla LLM Performance: On the 2017 baseline data, vanilla LLMs showed promise but were inconsistent. Gemini 2.5 Pro emerged as the strongest candidate with 80.1% accuracy, outperforming other LLMs but still trailing the XGBoost classifier, which achieved 98.5% accuracy on the same known dataset.
• Fine-Tuning Impact: When evaluated on the 2025 Contemporary Dataset, the fine-tuned Gemini 2.5 Pro model significantly outperformed its vanilla counterpart and the traditional XGBoost model.
  • Fine-tuned LLM: Achieved 83.2% accuracy, 94.4% precision, and 84.2% recall.
  • Vanilla LLM: Dropped to 64.2% accuracy on the new data.
  • XGBoost Classifier: Performance degraded significantly to 74.3% accuracy, highlighting the brittleness of traditional models when facing evolving threats not present in their training data.
• Generalization: The study demonstrates that while vanilla LLMs struggle with new threats, a specialized, fine-tuned model can generalize better than traditional static analysis methods against emerging malware families.

Significance and Claims

The paper posits that decompilation-driven LLM analysis represents a viable new frontier for automated threat detection, provided specific conditions are met:

1. Necessity of Fine-Tuning: The primary contribution is the demonstration that fine-tuning on domain-specific, curated data is essential. Vanilla models, despite their general code understanding, are not robust enough to replace traditional anti-virus software without specialized training.
2. Adaptability vs. Brittleness: The results highlight a critical trade-off: traditional machine learning models (like XGBoost) are highly accurate on known data but brittle against new threats, whereas fine-tuned LLMs offer superior adaptability to evolving coding patterns and behaviors.
3. Continuous Evolution Requirement: The paper explicitly claims that even specialized models degrade when encountering newer malware. Therefore, maintaining model effectiveness requires continuous fine-tuning with emerging threats to keep pace with changing malicious behaviors.
4. Practical Deployment: By using authentic malware samples and addressing token limit constraints through sample selection, the work moves beyond hypothetical scenarios to demonstrate practical applicability in production-relevant settings.

The authors conclude that while LLMs are not yet a complete replacement for existing security stacks, they offer a powerful, adaptable layer for detection that, when combined with continuous retraining, can address the limitations of signature-based and static analysis methods. Future work suggests integrating dynamic execution traces and exploring domain-specific pre-training to further enhance reliability and explainability.