Technical Summary: FDM – A Framework for Decision-making to build ML-based Malware detection systems

1. Problem Statement

The rapid evolution of malware, characterized by sophisticated evasion techniques (e.g., polymorphism, obfuscation) and zero-day exploits, has rendered traditional signature-based detection increasingly ineffective. While Machine Learning (ML) and Deep Learning (DL) offer promising alternatives by identifying behavioral patterns rather than static signatures, the current research landscape is fragmented. Existing studies typically propose specific ML configurations optimized for narrow constraints (e.g., a single dataset or analysis type) without providing systematic guidance for practitioners facing diverse operational environments.

Building an effective ML-based malware detection system involves navigating a complex, interdependent landscape of decisions: feature extraction (static vs. dynamic), model architecture (classical vs. deep), and update strategies. Without a structured methodology, practitioners often make ad-hoc choices that fail to balance detection performance, resource consumption (RAM, CPU), and long-term maintainability. The core problem addressed is the lack of a principled, context-aware framework to select the optimal ML configuration given varying organizational constraints, data characteristics, and operational requirements.

2. Methodology: The FDM Framework

The authors propose the Framework for Decision-making to build ML-based Malware detection systems (FDM). This framework formalizes the configuration selection process using a Weighted Configuration Compatibility Score (WCCS), a multi-criteria scoring function.

A. Input Dimensions (Operational Constraints)

The framework accepts five quantifiable input parameters representing the operational context:

1. Platform Constraint Index (Q1): Resource limits of the target deployment (e.g., mobile/edge vs. high-performance cluster).
2. Resource Budget Index (Q2): Available memory and compute for training/inference (e.g., constrained <4GB RAM vs. high-performance >16GB RAM).
3. Response Latency Level (Q3): Required inference speed (offline/batch vs. strict real-time <100ms).
4. Update Frequency Score (Q4): Cadence for model retraining (one-time vs. continuous daily updates).
5. Sensitivity Ratio (Q5): The relative operational cost of False Negatives vs. False Positives (FN-sensitive vs. FP-sensitive).

B. Output Dimensions (Configuration Recommendations)

Based on the inputs, the WCCS algorithm generates ranked recommendations across nine configuration dimensions:

• Data Acquisition (P1): Source selection (public, private, or in-house).
• Feature Engineering (P3.1–P3.3): Extraction type (static, dynamic, hybrid), preprocessing, and feature selection.
• Model Selection (P2.1): Ranking of ML models (e.g., RF, XGBoost, LSTM, CNN).
• Hyperparameter Tuning (P2.2–P2.3): Search space and fixed parameters.
• Maintenance Strategy (P4, P5): Update mechanisms (Transfer Learning vs. Class-Incremental Learning) and sensitivity objectives.

The WCCS calculates a compatibility score for each candidate configuration by weighting the alignment between the candidate's capabilities and the five input constraints. The configuration with the highest score is recommended.

C. Empirical Validation

To validate the framework and calibrate the scoring weights, the authors conducted four experiments across three heterogeneous datasets:

1. Private Windows API Call Dataset: 8 malware families + benign.
2. Malimg Dataset: 9,339 grayscale malware images from 25 families.
3. Public Android Static API Dataset: 15,036 applications.

The experiments addressed five research questions comparing classical vs. deep models, evaluating transfer learning, class-incremental learning, and autoencoder-based feature extraction.

3. Key Contributions

1. Quantifiable Decision Framework: The FDM encodes operational goals into a mathematical scoring system (WCCS) that produces ranked, context-appropriate ML configuration recommendations, moving beyond "one-size-fits-all" solutions.
2. Comprehensive Empirical Benchmarks: The study provides direct comparisons of classical (RF, XGBoost, SVM) and deep learning (LSTM, BiLSTM, CNN) models across binary classification, multi-class classification, incremental learning, and transfer learning tasks.
3. Validation of Context-Dependency: The experiments demonstrate that no single model dominates all scenarios. For instance, deep learning superiority in binary tasks does not necessarily translate to multi-class tasks, and transfer learning benefits are modality-dependent.
4. Illustrative Scenarios: The paper demonstrates the framework's utility through three distinct deployment scenarios (IoT edge security, ransomware defense for SMEs, and critical infrastructure protection), showing how the same framework adapts to vastly different constraints.

4. Key Results and Findings

The experimental results yielded specific insights that informed the FDM's recommendation logic:

• Binary Classification (API-based): XGBoost achieved the best accuracy-to-resource ratio (97.46% test accuracy, <70 MB RAM), outperforming LSTM and BiLSTM. Deep models consumed up to 2.8 GB RAM (approx. 40× more) for marginal or negative gains in accuracy.
• Multi-Class Classification: The ranking reversed. Classical models (XGBoost, 79.03%) significantly outperformed deep recurrent models (BiLSTM, 72.27%). This suggests that deep sequence models require substantially larger training corpora per class to generalize effectively across multiple families.
• Class-Incremental Learning (CIL): EfficientNetB0 (CNN) maintained 99.13% accuracy with only a 0.65 percentage-point degradation across 11 incremental steps, demonstrating superior resistance to catastrophic forgetting compared to tree-based models (XGBoost: 2.33 pp drop).
• Transfer Learning: Pre-training on ImageNet reduced training time by an average of 2.14× on image-based malware data (Malimg) with no significant accuracy loss. However, this benefit did not translate to API call sequences or Android permissions, where ImageNet features provided little inductive bias.
• Autoencoder Pre-processing: Using an autoencoder to compress features resulted in a 14× training speedup and 4.3× inference speedup, with a minimal accuracy cost of only 0.86 percentage points. This makes it a viable option for latency-constrained environments.

5. Significance and Claims

The paper claims that the FDM addresses a critical gap in cybersecurity research: the lack of systematic guidance for selecting ML techniques across diverse operational contexts.

• Context-Dependence: The primary finding is that the "optimal" ML configuration is not absolute but strictly dependent on the operational constraints (platform, resources, latency, update frequency, and sensitivity).
• Practical Utility: By mapping empirical findings to the WCCS scoring mechanism, the framework enables practitioners to make informed, data-driven decisions rather than relying on fragmented, isolated research results.
• Feasibility: The study demonstrates that a structured, constraint-aware selection process is both necessary and feasible, allowing organizations to balance high detection performance with resource efficiency and long-term maintainability.

The authors conclude that while the FDM provides a robust baseline for configuration selection, it does not replace the need for domain expertise, particularly in handling adversarial threats or explainability requirements, which are identified as areas for future extension.