Technical Summary: The Scissors Effect in Transfer-Based Adversarial Attacks

1. Problem Statement

Transfer-based adversarial attacks rely on generating perturbations on a surrogate (source) model that successfully fool unseen target models. A standard component of state-of-the-art attacks is Input Diversity (DI), which applies random resizing and padding at each attack iteration. The prevailing assumption in the field is that "more diversity is always better," leading modern methods (e.g., Admix, SSA, SIA) to aggressively stack input transformations.

This paper challenges the universality of this assumption. It investigates whether DI improves transferability across different surrogate training regimes, specifically comparing standard models (trained on clean data) against robust models (trained with adversarial training, e.g., PGD-AT). The authors hypothesize that the efficacy of DI is regime-dependent and may be reversed for robustly trained surrogates.

2. Methodology

The study employs a systematic empirical and theoretical approach to isolate the impact of DI on transferability:

• Experimental Setup: The authors evaluate transferability from various surrogate models (Standard and Robust) to diverse target architectures (CNNs, ViTs, Swin, ConvNeXt) on CIFAR-10 and ImageNet.
• Controlled Variables:
  • Diversity Probability ($p$): The probability of applying DI is swept from 0 to 1.
  • Robustness Strength: A controlled sweep varies only the training budget $\epsilon$ (using fixed ResNet-50 architecture and PGD-AT recipe) to observe the transition from standard to robust regimes.
  • Component Decomposition: DI is decomposed into resize (interpolation) and translation (padding) to isolate the specific transformation causing the effect.
• Metric: Attack Success Rate (ASR) is the primary metric. The authors also introduce Local Gradient Consistency (LGC), a measure of the cosine similarity between gradients at a clean input and its slightly perturbed neighbors, to characterize the surrogate's gradient geometry.
• Theoretical Framework: A bias-variance analysis is formalized to explain the mechanism, leading to a crossover theorem regarding when DI helps or hurts based on gradient signal-to-noise ratios.

3. Key Contributions

The Scissors Effect

The central discovery is the Scissors Effect: as the diversity probability $p$ increases, the transfer success rates of standard and robust surrogates diverge in opposite directions.

• Standard Surrogates: Increasing diversity generally improves or maintains transferability (beneficial or neutral).
• Robust Surrogates: Increasing diversity often degrades transferability, performing worse than attacks with no diversity at all.
• Magnitude: On ImageNet, blind DI costs robust sources an average of 10.3% ASR across diverse targets. On CIFAR-10, the effect is smaller (~2%) under default settings but grows to ~6.6% under aggressive DI.

Mechanism Analysis: Gradient Geometry and Resize

The paper attributes the effect to the geometry of input gradients:

• Standard Models: Possess noisy, high-frequency gradients. DI acts as a denoising mechanism (variance reduction), improving the attack direction.
• Robust Models: Possess smooth, low-frequency, semantically aligned gradients. The resize component of DI acts as a low-pass filter that introduces systematic bias without reducing variance (since variance is already low), thereby degrading the already-optimal gradient direction.
• Decomposition: A resize/translation decomposition reveals that ~67% of the harm on robust surrogates is caused specifically by the resize step, while translation is nearly neutral.

Local Gradient Consistency (LGC)

The authors propose LGC as a single input-space probe to identify the gradient regime:

• Separation: Robust models exhibit high LGC ($\approx 0.98\text{--}1.00$), while standard models exhibit lower LGC ($\approx 0.64\text{--}0.92$).
• Predictive Power: LGC correlates strongly with DI sensitivity. High LGC predicts that DI will be harmful.
• Theorem: A bias-variance theorem (Proposition 1) proves that resize-based DI strictly improves alignment if and only if LGC is below a specific threshold ($\tau^*$), and necessarily hurts as LGC approaches 1.

CG-DI (Consistency-Guided Diversity Input)

Based on these findings, the authors propose CG-DI, a training-free rule:

• Mechanism: Compute LGC on the surrogate. If LGC > $\tau$ (e.g., 0.92), disable diversity ($p=0$); otherwise, enable it ($p=0.8$).
• Result: CG-DI avoids the ~10.3% ASR loss on robust surrogates while retaining the benefits of DI on standard surrogates. It effectively acts as a "guardrail" against the Scissors Effect.

4. Results

• Generalization: The Scissors Effect holds across 10 different attacks spanning 2018–2024 (including MI-FGSM, Admix, SSA, SIA) and diverse architectures (ResNet, ViT, Swin, ConvNeXt).
• Robustness Sweep: The transition from beneficial to harmful DI is graded, not binary. The crossover occurs in the "little-robustness" regime (e.g., $\epsilon_{train} \in (0, 0.5]/255$), deepening as robustness increases.
• SSA Exception: The Spectrum Simulation (SSA) attack, which operates in the frequency domain, shows the largest harm on robust surrogates. The authors explain this via Corollary 1: SSA acts as a pre-filter that increases effective LGC, pushing even standard surrogates closer to the harmful crossover threshold.
• Same-Family Control: The effect persists even when source and target share the same architecture (e.g., ResNet-50 to ResNet-50), confirming the phenomenon is driven by the surrogate's gradient regime, not architectural similarity.

5. Significance and Claims

The paper positions the Scissors Effect as a DI-specific manifestation of the broader robustness–transferability trade-off. Its significance lies in:

1. Correcting a Default Assumption: It demonstrates that the "more diversity is better" heuristic is regime-dependent. For robust surrogates (increasingly common in safety evaluations), defaulting to DI can lead to underestimating attack strength and overestimating target robustness.
2. Isolating the Culprit: It identifies resize as the specific component responsible for the harm, distinguishing it from other transformations like translation.
3. Actionable Guidance: It provides a practical, training-free method (CG-DI) using LGC to dynamically adjust attack hyperparameters, ensuring robust surrogates are not inadvertently weakened during evaluation.
4. Theoretical Grounding: It formalizes the intuition through a bias-variance crossover theorem, linking gradient consistency directly to the efficacy of input diversity.

The authors conclude that as robust pretrained models become standard surrogates, attack design must account for the surrogate's gradient geometry, and DI should not be treated as a universally beneficial default.