Technical Summary: On the Reliability and Stability of Selective Methods in Malware Classification Tasks

Problem Statement

Modern malware classifiers, particularly those adapted for distribution drift, often report promising performance metrics (e.g., F1, FPR). However, these metrics primarily reflect baseline predictive accuracy under static or isolated shift conditions, neglecting operational reliability and stability in real-world deployment.

In operational settings (e.g., active learning or selective classification), the model's own confidence estimates serve as the sole signal for decision-making once ground-truth labels are unavailable. A trustworthy system must ensure that high-confidence predictions are likely correct and that low-confidence (or high Out-of-Distribution/OOD) scores accurately flag instances for human review. Current evaluation paradigms fail to verify if confidence functions remain calibrated and stable under sustained temporal drift. As noted by Goodhart [15], "When a measure becomes a target, it ceases to be a good measure." Optimizing for headline metrics like F1 often masks deficiencies in confidence ranking, leading to:

1. Misdirected resources: Active learning wasting annotation budgets on non-informative samples.
2. Operational fragility: Selective classification failing to reject error-prone instances, or conversely, rejecting correct predictions.
3. Unpredictable workloads: Rejection mechanisms failing to track target quotas under drift.

The paper argues that under distribution shift, neural networks are prone to severe miscalibration, assigning high confidence to incorrect predictions on OOD inputs, thereby undermining the core premise of selective classification and active learning.

Methodology: The Aurora Framework

To address these gaps, the authors propose Aurora, a framework designed to evaluate malware classifiers based on confidence quality and operational resilience rather than just point-in-time accuracy.

1. Evaluation Setting

The framework employs a non-stationary, temporal evaluation protocol simulating real-world deployment:

• Data: Three Android malware datasets (APIGraph, AndroZoo, Transcendent) with varying drift severity.
• Protocol: A monthly loop where a model is evaluated on a new month's data ($M_i$), queries a fixed annotation budget ($B_{Mi}$) of the most uncertain samples, receives labels, and updates.
• Constraints: No ground-truth labels are available during the online phase; decisions rely solely on the model's confidence scores. A fixed rejection threshold ($\rho$) is calibrated on a rolling window of unlabeled scores to simulate a monthly annotation budget or target error rate.

2. Metrics

Aurora introduces a multi-dimensional set of metrics categorized into three pillars:

• Reliability (Confidence Quality):

  • AURC (Area Under the Risk-Coverage Curve): Measures how effectively confidence scores rank instances by their actual probability of misclassification. Lower AURC indicates better ranking (errors are correctly identified as low-confidence).
  • AURC[F1]*: An operational variant of AURC using F1-based risk ($1 - \text{F1}$) instead of error rate. This detects if abstention disproportionately harms one class (e.g., rejecting malware samples while keeping benign ones), which standard AURC might miss.
  • Benefit Frequency (BF):* The proportion of months where selective classification (abstention) improves F1 over a no-rejection baseline.

• Stability (Temporal Robustness):

  • $\sigma[\text{F1}]$: Standard deviation of monthly F1 scores, measuring performance volatility.
  • Mann-Kendall Trend ($\tau$): A non-parametric test detecting monotonic performance trends (improvement, degradation, or stability) over time.
  • Rejection Consistency:
    • $\Delta\text{Rej}^*$: Bias between actual and target rejection quotas.
    • $\sigma[\text{Rej}]^*$: Standard deviation of realized rejections, measuring the predictability of analyst workload.

3. Benchmarked Models

The framework evaluates four state-of-the-art (SOTA) open-source frameworks:

• Drebin: Linear SVM on static features.
• DeepDrebin: Fully connected network on Drebin features.
• CADE: Contrastive learning with distance-based OOD scoring.
• HCC: Hierarchical contrastive learning with family-level constraints.

The study also investigates the impact of data curation, comparing models trained on the full initial dataset ($B_0 = \text{Full}$) versus a subsampled set ($B_0 = 4800$).

Key Results

The evaluation reveals a significant disconnect between headline performance metrics and operational reliability.

1. Confidence Calibration vs. Accuracy

• F1 does not imply reliability: Models with similar F1 scores often exhibit vastly different AURC values (up to 10x difference). For instance, on the APIGraph dataset, CADE and DeepDrebin had comparable F1 scores, but CADE's AURC was 6x worse, indicating its confidence scores failed to rank errors correctly.
• Class Imbalance in Rejection: AURC[F1]* revealed that some models (e.g., HCC) ranked total errors well but disproportionately rejected one class (e.g., malware), degrading balanced detection quality.
• OOD Scoring Failure: CADE's distance-based OOD scoring (assuming errors correlate with distance from cluster centers) failed empirically. Risk-Coverage curves showed that misclassifications often occurred near cluster centers (high confidence), contradicting CADE's core premise.

2. Temporal Stability and Rejection Consistency

• Catastrophic Instability: CADE exhibited extreme instability in rejection behavior. On the Transcendent dataset, CADE-MSP overshot its rejection quota by an average of 3,375 samples (target 400) with massive variability ($\sigma[\text{Rej}]^* \approx 3347$), making it unsuitable for fixed-budget operations.
• Fragile Oscillation vs. Robust Adaptation: While some models showed high F1, their stability metrics ($\sigma[\text{F1}]$ and $\tau$) revealed systematic degradation or erratic oscillation under drift. DeepDrebin, conversely, demonstrated robust adaptation with steady improvement trends on certain datasets.
• Benefit of Abstention: Selective classification was not universally beneficial. CADE achieved a Benefit Frequency (BF*) of <5% in several configurations, meaning abstention actively harmed performance in most months.

3. The Pareto Perspective

When evaluating methods across multiple dimensions (F1, AURC, $\sigma[\text{F1}]$, $\tau$), no single method dominated all conditions.

• DeepDrebin (with subsampling) and HCC-MSP (with subsampling) consistently emerged as Pareto-optimal, offering the best trade-offs between performance, stability, and reliability.
• Subsampling Advantage: Counter-intuitively, training on a subsampled initial dataset ($B_0 = 4800$) often outperformed training on the full first year of data, suggesting that large volumes of historical data can impede adaptation to new drift patterns.
• Complexity vs. Utility: The simplest neural approach (DeepDrebin) often outperformed complex frameworks (CADE, HCC) that rely on costly family labels and extensive training, consuming a fraction of the computational budget.

Significance and Claims

The paper claims that the current reliance on static accuracy metrics (F1, FPR) is insufficient for assessing the trustworthiness of malware classifiers in dynamic environments.

1. Revisiting Assumptions: The fragility observed in SOTA frameworks suggests that underlying assumptions about confidence calibration and OOD detection (e.g., distance-based metrics) may need revision.
2. Operational Reality: A model's behavior in deployment is conditioned by the quality and stability of its confidence-based ranking, not just its in-lab accuracy.
3. Methodological Shift: The authors advocate for a shift in security model evaluation toward multi-dimensional assessment that includes confidence calibration, temporal stability, and rejection consistency.
4. Practical Implications: The findings suggest that "best" methods depend on operational priorities (e.g., workload predictability vs. raw detection rate). A method that dominates under F1 may be dominated when reliability and stability are considered.
5. Simplicity Wins: The study highlights that complex, family-label-dependent frameworks do not necessarily translate to deployment robustness, and simpler baselines with strategic data curation can be more effective.

In conclusion, the paper posits that "when F1 becomes the target, it ceases to be a good measure of trustworthiness," and that frameworks like Aurora are necessary to expose the hidden risks of deploying selective classification methods under distribution drift.