Detecting Trojaned DNNs via Spectral Regression Analysis

Technical Summary: Detecting Trojaned DNNs via Spectral Regression Analysis

Problem Statement
Deep Neural Networks (DNNs) are increasingly deployed as evolving software artifacts that undergo continuous fine-tuning to incorporate new data and adapt to changing environments. This evolutionary workflow introduces a significant security risk: if the update data cannot be fully trusted, adversaries may inject poisoned samples to implant Trojans (backdoors) during the fine-tuning process. A Trojaned model maintains high accuracy on clean inputs but exhibits attacker-chosen behavior when a specific, often imperceptible, trigger is present.

Existing defense mechanisms face critical limitations. Approaches that assume control over the training process (e.g., data sanitization) are inapplicable in supply-chain scenarios where the defender only receives a pre-trained or updated model. Inference-time defenses often rely on reconstructing trigger patterns, an assumption that fails against sophisticated, input-dependent, or feature-space triggers. Furthermore, many offline detection methods attempt to reverse-engineer triggers without knowledge of the trigger's form, location, or visibility, making them fragile against diverse attack strategies.

Methodology: MIST
The authors propose MIST (Model Incremental Spectra Tracking), a Trojan detection approach that shifts the focus from input-level trigger reconstruction to the analysis of internal model representations during evolution. MIST treats model updates as regression events, assessing whether the internal changes induced by fine-tuning are consistent with benign evolution or indicative of malicious modification.

The core of MIST relies on Spectral Analysis of pre-activations. Rather than analyzing individual neuron activations (which are unstable across training runs), MIST summarizes layer-wise pre-activations as probability distributions (spectra). These spectra provide a stable characterization of learned representations that is resilient to retraining noise but sensitive to systematic changes.

The methodology operates in two phases:

1. Clean Spectra Tracking: The system establishes a reference distribution of spectral changes (Clean Spectra Distances Distribution, or CSDD) by simulating benign fine-tuning transitions. Using a trusted clean dataset, the system repeatedly trains a base model and fine-tunes it, computing the $L_2$ distance between the pre-activation spectra of the base and fine-tuned models for each class. This builds a statistical baseline of how benign updates alter the model's internal state.
2. Anomaly Detection: When a new model update ($M_{i+1}$) is produced from a previous checkpoint ($M_i$) using potentially untrusted data, MIST computes the spectral distance between the two versions using a small, trusted clean subset of the new data. It then calculates the squared Mahalanobis distance of this observed change against the reference CSDD. If the deviation exceeds a statistically derived threshold (based on a $\chi^2$ distribution), the update is flagged as Trojaned.

Crucially, MIST requires no knowledge of the trigger, the poisoned data, or the attack strategy. It assumes the availability of a clean reference checkpoint and a small clean subset of the update data for observation, but not for retraining.

Key Contributions

• Regression-Based Perspective: The paper reframes Trojan detection as a regression problem over model updates, characterizing malicious modifications as anomalous deviations in spectral evolution rather than trigger reconstruction problems.
• MIST Technique: The proposal of a practical detection technique based on pre-activation spectra that validates fine-tuned updates against a clean reference.
• Comprehensive Evaluation: A rigorous empirical study across four datasets (CIFAR-10, SVHN, GTSRB, CelebA) and eight distinct Trojan attacks (including BadNet, Blended, Input-Aware, Refool, SIG, SSBA, TrojanNN, and WaNet), comparing MIST against three state-of-the-art detectors (Neural Cleanse, ABS, FeatuRE).

Experimental Results

• Spectral Separability (RQ1): Spectral distances reliably separate Trojaned models from clean fine-tuned models, achieving an average ROC-AUC of 0.95 across all attack-dataset combinations. This confirms that Trojan insertion induces internal spectral changes distinct from benign evolution.
• Detection Effectiveness (RQ2): In single-step update scenarios, MIST achieves an average detection accuracy of 0.95, significantly outperforming state-of-the-art baselines (FeatuRE: 0.75, ABS: 0.73, Neural Cleanse: 0.64). MIST demonstrates a favorable error profile with very low false-positive rates, making it suitable for validating model updates without frequently rejecting benign ones.
• Robustness (RQ3): Under multi-step benign evolution, MIST remains effective with an average accuracy of 0.89. Performance degradation is graceful and bounded, primarily manifesting as an increase in false positives (due to accumulated benign drift) rather than missed detections (false negatives).

Significance and Claims
The paper claims that spectral evolution provides a stable and assumption-light signal for detecting malicious model updates. By focusing on the internal state changes rather than external trigger patterns, MIST offers a robust defense against a wide variety of Trojan attacks, including those with complex or invisible triggers. The authors emphasize that this approach is particularly well-suited for deployment settings where models are continuously updated and must be validated before redeployment, provided a clean reference checkpoint is available. The results suggest that spectral analysis captures a fundamental signature of malicious modification that persists even as models evolve, offering a reliable alternative to trigger-reconstruction-based defenses.