Integrating APK Image and Text Data for Enhanced Threat Detection: A Multimodal Deep Learning Approach to Android Malware

Technical Summary: Integrating APK Image and Text Data for Enhanced Threat Detection

Problem Statement
The proliferation of sophisticated zero-day Android malware necessitates robust detection mechanisms beyond traditional static and dynamic analysis, which are often vulnerable to obfuscation. While recent research has demonstrated the efficacy of converting Android Application Package (APK) bytecode into image representations for deep learning-based detection, significant gaps remain. Existing studies frequently fail to systematically evaluate how image attributes—specifically color format (RGB vs. grayscale) and resolution (e.g., 128x128, 256x256, 512x512)—impact detection performance. Furthermore, current image-based approaches often neglect valuable textual data embedded within APKs, such as permissions, metadata, and manifest files. The potential of integrating these textual features with image data via multimodal deep learning to enhance detection accuracy remains underexplored.

Methodology
The authors propose a multimodal deep learning framework to address these gaps through two primary investigative phases:

1. Image Attribute Evaluation (RQ1):

   • Data Collection: The study utilized 4,303 malware APKs and 4,039 benign APKs from the CIC-AndMal2017 and CICMalDroid 2020 datasets.
   • Image Generation: APK files were decompiled and converted into binary streams, then transformed into two image types: Grayscale and RGB. These were generated at three resolutions: 128x128, 256x256, and 512x512.
   • Model Training: Eight pre-trained Convolutional Neural Network (CNN) architectures were trained and tested on these datasets: VGG-16, VGG-19, ResNet-50, ResNet-101, ResNet-152, MobileNet-V2, DenseNet-169, and EfficientNet-B4.
   • Metrics: Performance was assessed using accuracy, precision, recall, F1-score, and Area Under the ROC Curve (AUC-ROC).

2. Multimodal Integration (RQ2):

   • Text Extraction & Annotation: Textual features (permissions, resources, manifest data) were extracted from APKs. A Large Language Model (LLM), specifically LLaMA-2, was employed to generate prompt-based annotations summarizing these features as either "malicious" or "benign."
   • Multimodal Fusion: The annotated text and the corresponding APK images were integrated into a multimodal dataset.
   • Model Evaluation: The CLIP (Contrastive Language-Image Pre-training) model was utilized to classify the combined image-text data in a zero-shot manner. This was compared against the standalone CNN models and a Quantum Neural Network (QNN) baseline.

Key Contributions

• Systematic Image Analysis: The paper provides a comprehensive evaluation of image attributes across eight CNN architectures, identifying optimal configurations for different models.
• Novel Multimodal Framework: The authors developed a framework that combines APK images with LLaMA-2 generated text annotations, utilizing the CLIP model to fuse visual and semantic features.
• Dataset Contribution: A limited multimodal dataset comprising diverse attributed images and their corresponding textual annotations was created to support further research in this domain.

Results

• Image-Based Detection (RQ1):

  • High-Resolution RGB Superiority: For deeper architectures (ResNet-50, ResNet-101, ResNet-152, EfficientNet-B4), RGB images at 512x512 resolution yielded the highest performance, with ResNet-101 achieving 97% accuracy and a 0.9919 ROC AUC.
  • Balanced Efficiency: The 256x256 RGB configuration offered a strong compromise between computational cost and performance, particularly for ResNet-152 and DenseNet-169.
  • Model-Specific Optimizations: VGG-16 performed best with low-resolution Grayscale (128x128), while MobileNet-V2 achieved peak accuracy (96%) with 512x512 images in both RGB and Grayscale formats.
  • General Trend: RGB images generally outperformed Grayscale images for complex models, whereas Grayscale was more effective for simpler models or lower resolutions to avoid overfitting.

• Multimodal Performance (RQ2):

  • Limited Success of CLIP: The multimodal approach using CLIP performed poorly, achieving only 50% accuracy (equivalent to random guessing in a binary classification context).
  • Failure to Detect Malware: The CLIP model failed to detect any malware samples (Recall = 0.00 for malware) and achieved a macro F1-score of 0.33.
  • Comparison: Standalone image-based models, particularly ResNet-152 (71% accuracy in the specific test set used for comparison), significantly outperformed the multimodal CLIP approach. The authors attribute this to the limited size of the multimodal dataset (only 34 image-text pairs).

Significance and Claims
The paper claims that while multimodal integration is a promising direction, its current implementation using CLIP and small datasets does not yet surpass optimized image-only CNNs for Android malware detection. The study emphasizes the critical importance of systematically evaluating image attributes (resolution and color space) to maximize detection efficacy. The authors conclude that high-resolution RGB images combined with deep CNNs (like ResNet-152 and EfficientNet-B4) currently offer the most effective solution. They modestly assert that the multimodal approach requires larger, more diverse datasets and advanced fusion techniques (beyond zero-shot CLIP) to realize its potential in enhancing threat detection capabilities.