Technical Summary: Scooter – A Human Evaluation Framework for Unrestricted Adversarial Examples

Problem Statement

Unrestricted adversarial attacks aim to deceive computer vision (CV) models by altering images in ways that are not constrained by $\ell_p$-norm bounds, such as changing an object's color or texture. Unlike restricted attacks, these modifications can be semantically coherent and easily overlooked by humans, allowing attackers to bypass traditional norm-based defenses like adversarial training or certified robustness. However, the lack of norm-based constraints means there is no mathematical guarantee of imperceptibility. Consequently, verifying whether these adversarial examples (AEs) appear authentic to humans requires human evaluation.

Existing literature often relies on automated metrics or small-scale human studies that lack statistical rigor. Many prior works either neglect image quality evaluation entirely or rely on automated metrics that fail to align with human perception. Furthermore, existing human evaluation protocols often lack statistically significant insights, transparent reporting of demographics, power analysis, and robust filtering for inattentive participants. This creates a gap in the ability to reliably compare and validate the imperceptibility of unrestricted attacks.

Methodology: The Scooter Framework

To address these gaps, the authors introduce Scooter (Systemizing Confusion Over Observations To Evaluate Realness), an open-source, statistically powered framework for evaluating unrestricted adversarial examples. The framework is structured around a three-phase pipeline designed to ensure data quality and statistical validity:

1. Preliminary Checks:

   • Colorblindness Test: Participants must correctly classify five Ishihara-like images to ensure they can perceive color-based modifications, a primary vector for unrestricted attacks.
   • Comprehension Check: Participants review examples of modified vs. unmodified images and must correctly identify the modified image in at least 5 out of 6 pairs. This ensures understanding of the task and filters out non-fluent English speakers.

2. Main Annotation Phase:

   • Participants rate 106 images (50 real, 50 adversarial) on a 5-point Likert scale ranging from "Definitely Modified" (-2) to "Definitely Real" (+2).
   • The design uses a within-subject repeated-measures approach where participants rate both real and modified images.
   • Attention Checks: The study includes "bogus items" (obviously modified images) and "instruction manipulation checks" (IMCs) to detect inattentive participants. Participants failing 2 or more of 6 attention checks are flagged.
   • Filtering: Additional metrics such as average time per image, Long-String statistics, and Intra-individual Response Variability (IRV) are used to filter low-quality data.

3. Statistical Analysis & Reporting:

   • Equivalence Testing: Instead of testing for significant differences (which would imply detectability), Scooter employs the Two-One-Sided Tests (TOST) procedure to test for equivalence. The goal is to prove that the mean rating of adversarial images ($\mu_{modified}$) is practically equivalent to the mean rating of real images ($\mu_{real}$) within a predefined tolerance ($\Delta$).
   • Bounds: Based on empirical simulations, the authors set practical equivalence bounds of $\Delta_L = -0.2$ and $\Delta_U = +0.2$. If the difference between $\mu_{real}$ and $\mu_{modified}$ falls within this range, the attack is considered imperceptible.
   • Reporting: The framework mandates reporting participant demographics, compensation, time commitment, and detailed statistical outputs to ensure reproducibility.

Experimental Setup

The authors applied the Scooter framework to evaluate six unrestricted attacks:

• Color-based Attacks: SemanticAdv (SemAdv), cAdv, and Natural Color Fool (NCF).
• Diffusion-based Attacks: DiffAttack (DA), AdvPP, and ACA.

Victim Model: The attacks were evaluated against an adversarially trained ResNet-50 (ResNet-50-AT), a robust baseline model known for resisting standard norm-bounded attacks.
Dataset: A curated subset of ImageNet-1k validation images, termed ImageNet S-R50-N, containing 2,966 images that are correctly classified by the victim model and free of pre-existing artifacts.
Participants: A total of 346 participants were recruited via Prolific, resulting in over 34,000 human ratings.

Key Results

The study yielded statistically significant evidence contradicting the claim that current unrestricted attacks are imperceptible:

1. Failure of Imperceptibility: For all six attacks, the TOST procedure failed to reject the null hypothesis that the images are not equivalent. Specifically, the p-values for the upper-bound test ($\Delta > \Delta_U$) were 1.000 for all attacks. This indicates that real images were rated significantly higher (more "real") than the adversarial examples, with gaps far exceeding the $\pm 0.2$ tolerance.
2. Attack Performance:
   • Diffusion-based attacks generally outperformed color-based attacks in terms of imperceptibility (e.g., AdvPP had a $\mu_{modified}$ of -0.5188, closer to real images than others), but still failed to achieve statistical equivalence.
   • SemAdv, despite being the simplest attack, performed surprisingly well, outperforming four other attacks in raw modified scores, though it still failed the equivalence test.
   • NCF produced the most obvious modifications, with the largest gap between real and modified ratings.
3. Discrepancy with Model Success: There is no correlation between an attack's success rate against the victim model and its imperceptibility to humans. For instance, NCF had a high attack success rate (0.922) but was highly perceptible, while AdvPP had a low success rate (0.081) but was the least perceptible.
4. Automated Proxies:
   • GPT-4o: The vision-language model showed some ability to flag imperceptible attacks but did not perfectly align with human ratings, particularly for AdvPP and ACA.
   • Objective Metrics: Standard image quality metrics (e.g., FID, FD, KD) failed to correlate with human perception. Notably, metrics often ranked NCF (the most perceptible attack) as having the highest image quality, directly contradicting human judgments. The per-image metric TOPIQ showed the best alignment but still failed to reliably proxy human judgment across all attacks.

Significance and Claims

The paper claims to provide the first openly available, statistically powered benchmark that rigorously links human perception to the quality of unrestricted adversarial examples. Its primary contributions are:

• Methodological Standard: It establishes best-practice guidelines for crowd-study power, compensation, and statistical analysis (specifically TOST) for evaluating imperceptibility, addressing the lack of statistically significant insights in prior work.
• Empirical Evidence: It demonstrates that current state-of-the-art unrestricted attacks, including those based on diffusion models, are not imperceptible to humans. The authors conclude that humans can easily distinguish authentic images from those modified by these attacks.
• Decoupling of Goals: The findings suggest that fooling a computer vision model and fooling a human observer are largely orthogonal objectives; high attack success does not imply high imperceptibility.
• Limitations of Automated Metrics: The study reinforces that existing objective image quality metrics are insufficient proxies for human perception in the context of adversarial examples.

The authors position Scooter as a tool to enable reproducible, human-centered research, urging the community to move beyond automated metrics and unverified claims of imperceptibility. They release the source code, the ImageNet S-R50-N dataset, and over 34,000 human ratings to facilitate future research in this domain.