Technical Summary: AI Security Research Should Better Incentivize Defense Research

Problem Statement

The paper identifies a structural imbalance in the field of Artificial Intelligence (AI) security research: a disproportionate emphasis on "attack research" (identifying and demonstrating vulnerabilities) compared to "defense research" (developing and validating protections). While both are essential, the authors argue that the current literature is "rich in demonstrated vulnerabilities and thin on usable and deployed protections." This imbalance is not merely a quantitative gap but reflects a deeper issue where attack papers are evaluated under favorable conditions that exaggerate threats, while defense papers are held to stricter, often unrealistic standards. The authors posit that this skew suggests a structural lag in defensive progress, which is problematic for a field whose societal impact depends increasingly on mitigation rather than just vulnerability discovery.

Methodology

To quantify and analyze this imbalance, the authors employed a bibliometric approach based on existing Systematization of Knowledge (SoK) papers and survey papers.

1. Data Collection:
   • Core Corpus: The authors collected 16 SoK papers from top-tier security and privacy venues (USENIX Security, IEEE S&P, IEEE SaTML). These papers were selected for their role in synthesizing and contextualizing mature research areas.
   • Extended Corpus: To broaden coverage, 21 additional survey papers were incorporated, covering specific subfields such as membership inference attacks (MIA), adversarial examples, large language models (LLMs), model stealing, federated learning (FL), speech recognition, deepfakes, and machine unlearning.
2. Classification Protocol:
   • Citations from these papers were manually classified into "Attack," "Defense," or "Other."
   • Attack: Papers whose main contribution is identifying, demonstrating, improving, or evaluating methods to exploit vulnerabilities.
   • Defense: Papers whose main contribution is preventing, detecting, mitigating, or certifying against such vulnerabilities.
   • Other: Surveys, benchmarks, datasets, and papers without a primary offensive or protective contribution (excluded from ratio analysis).
   • Ambiguous cases were resolved by consulting abstracts or assigning the label of the dominant contribution.
3. Data Processing:
   • Venues were normalized and grouped into four categories: "Security," "AI," "Preprint," and "Other."
   • Title-based deduplication was performed to ensure unique counts.
   • The final SoK corpus contained 644 unique attack papers and 518 unique defense papers.

Key Contributions

The paper's primary contribution is a rigorous, data-driven characterization of the attack-defense imbalance across the AI security landscape, moving beyond anecdotal evidence to systematic analysis. It provides:

• Quantitative Evidence: Specific attack-to-defense ratios across different subfields, venues, and time periods.
• Structural Analysis: An examination of why this imbalance exists, identifying four key asymmetries: novelty, evaluation, publication, and cost/complexity.
• Critical Counter-Arguments: A nuanced discussion of alternative views (e.g., that the field is still in a discovery phase or that the bottleneck is translation rather than research volume) and why these do not fully negate the need for better defense incentives.

Results

1. Quantitative Imbalance

• SoK Corpus: Across 16 SoK papers covering 1,162 classified citations, attack papers outnumbered defense papers by a ratio of 1.24:1.
• Survey Corpus: In the 21 additional survey papers, the average attack-to-defense ratio was 2.15:1.
• Subfield Variation: The imbalance is not uniform.
  • High Imbalance: Subfields focused on specific attack classes (e.g., black-box adversarial attacks, data inference privacy, LLM membership inference) showed extreme skew, with ratios ranging from 3.69× to 34.40×.
  • Defense-Heavy Areas: SoKs organized around defensive mechanisms (e.g., differential privacy, certified robustness) naturally contained more defense papers, but these were exceptions rather than the norm in broad threat landscape surveys.

2. Venue Distribution

• Security Venues: Show a strong bias toward attacks (2.10:1 ratio). Major conferences like USENIX Security (3.50×) and IEEE S&P (3.00×) heavily favor attack-oriented contributions.
• AI Venues: Show a much closer balance (1.12:1). Conferences like NeurIPS, AAAI, ICLR, and ICML actually include more defense papers than attack papers.
• Computer Vision: A notable exception within AI, where vision venues (CVPR, ICCV, ECCV) exhibit extreme attack skew (up to 17.00×), reflecting the historical dominance of adversarial example literature.

3. Temporal Trends

• From 2010 to 2015, attack and defense counts were low and relatively balanced.
• Starting around 2016, the trajectories diverged, with attack papers consistently outnumbering defense papers.
• The gap widened significantly between 2017 and 2022, peaking in 2021. The apparent narrowing post-2022 is attributed to citation lag in SoK publications rather than a genuine convergence in research output.

4. Identified Dilemmas for Defense Research

The paper outlines four structural asymmetries that disincentivize defense research:

• Novelty Asymmetry: Demonstrating a new failure mode (attack) is often sufficient for publication, whereas proving a defense is effective against adaptive adversaries is a much higher bar.
• Evaluation Asymmetry: Attacks are often evaluated under favorable, specific conditions (e.g., batch size = 1), while defenses are expected to be universally robust across models, datasets, and attacker adaptations.
• Publication Asymmetry: Attack papers follow a compelling narrative of "system failure," while defense papers rely on fragile claims of "withstanding all attacks," which are harder to validate and more susceptible to reversal.
• Cost and Complexity Asymmetry: Attacks can often be demonstrated in academic lab settings with off-the-shelf hardware, whereas effective defenses often require industrial-scale engineering, hardware redesign, or complex integration, placing them outside the typical scope of academic papers.

Significance and Claims

The paper argues that the observed imbalance is not merely a "topical gap" but a symptom of a misaligned incentive structure. The authors claim that:

1. Correction is Necessary: The field must move beyond simply counting papers to correcting the incentives that systematically reward vulnerability discovery over remediation.
2. Defense as First-Class Contribution: Defense research should be treated as a "first-class scientific contribution," requiring the same rigor and recognition as attack research.
3. Beyond Publication Counts: While the paper calls for more defense research, it acknowledges that the bottleneck may also lie in "translation"—the engineering, deployment, and organizational adoption of defenses. Therefore, the call to action is for a broader investment in "defensive capacity," including realistic evaluation, deployment pathways, and institutional support for mitigation-oriented work.
4. Maturity of the Field: A mature security field should not only be adept at showing how systems fail but must also be capable of building, validating, and sharing robust ways to make systems secure.

The authors conclude that while attack research remains essential for mapping threats, the current trajectory risks leaving the field with a deep understanding of vulnerabilities but a shallow understanding of how to effectively secure AI systems in practice.