Technical Summary: One Step to the Side: Why Defenses Against Malicious Finetuning Fail Under Adaptive Adversaries

1. Problem Statement

The release of open-weight foundation models and the availability of fine-tuning APIs have created a critical post-release safety vulnerability. While models are typically safety-aligned before release, users can subsequently overwrite these alignments via gradient-based optimization. Recent research has proposed various defenses against Malicious Fine-Tuning (MFT), aiming to make aligned models robust to attempts to recover harmful behavior.

However, the paper identifies a fundamental flaw in the evaluation of these defenses: they are predominantly tested against fixed, non-adaptive adversaries. These standard evaluations assume the attacker optimizes solely for harmfulness ($L_h$) using a prescribed dataset, optimizer, and hyperparameters. This approach ignores the central lesson of adversarial machine learning: defenses must be evaluated against adaptive adversaries who know the defense mechanism and optimize their attack strategy specifically to bypass it. The authors argue that current robustness claims are incomplete because they fail to account for an attacker who can choose the data, optimizer, and loss function to counter the specific defense.

2. Methodology and Systematization

2.1 Taxonomy of Defenses

The authors systematize 15 recent MFT defenses (including Vaccine, Booster, CTRAP, and TAR) and categorize them into two primary strategies, which collapse into four underlying loss templates:

1. Strategy A: Anchoring

   • Goal: Prevent the model parameters ($\theta_{def}$) from moving toward a harmful region. The defense shapes the local loss landscape so that descending the harmful loss ($L_h$) yields no progress (e.g., gradients are zero, randomized, or point away from the target).
   • Templates:
     • Robust-alignment basin (T1): Flattens or randomizes the landscape around $\theta_{def}$ (e.g., Vaccine, RepNoise).
     • Harmful-information removal (T2): Purges internal representations of harmful information so $\nabla L_h$ has no signal (e.g., RepNoise, KT-IPA).
     • Look-ahead defense (T3): Simulates an attacker's trajectory and penalizes progress along it (e.g., MLAC, TAR).

2. Strategy B: Self-Destruction

   • Goal: Allow the model to descend toward harmful behavior but ensure that doing so causes a collapse in general capabilities ($L_c$). The recovered model becomes harmful but useless.
   • Templates:
     • Look-ahead with collapse (T3): Simulates a harmful step and forces the endpoint to fail on benign tasks (e.g., CTRAP).
     • Coupling trap (T4): Explicitly couples the direction of harmful improvement with the degradation of benign utility (e.g., SEAM, SDD).

2.2 The Shared Vulnerability

The analysis reveals a common oversight: all these defenses assume the attacker optimizes only for $L_h$. They do not account for the fact that a successful MFT attack requires satisfying a joint condition: the model must emit harmful content while preserving general utility (coherence, instruction-following).

2.3 Proposed Adaptive Attacks

The authors introduce two adaptive attacks that exploit this shared vulnerability:

1. SIDESTEPPER (Mixed-Objective Attack):

   • Mechanism: Instead of optimizing solely for $L_h$, the attacker optimizes a combined objective: $L_{atk} = L_h + \lambda L_c$.
   • Logic: By including the capability loss ($L_c$) as a positive term in the objective, the attacker uses the "escape signal" provided by the defense's own requirement to maintain utility. This guides the optimizer out of the local "traps" (anchoring basins or self-destructive regions) created by the defense, finding a path to parameters that are both harmful and useful.

2. KICK-SETTLE (Trajectory Attack):

   • Mechanism: This attack keeps the objective as $L_h$ but alters the optimization schedule. It begins with a very high learning rate ("kick") to push parameters out of the local region controlled by the defense, then decays to a standard rate ("settle") to converge on a harmful-and-useful solution.
   • Logic: This exploits the locality of the defenses. Most defenses only constrain the loss landscape in the immediate neighborhood of $\theta_{def}$ or along a specific simulated trajectory. A large "kick" escapes this local constraint.

3. Experimental Results

The authors evaluated these adaptive attacks against 6 representative defenses (Booster, CTRAP, VAA, Vaccine, Unlearn-Smooth, SDD) across three model backbones (Llama-2-7B, Qwen3-8B, Llama-3.1-8B).

• SIDESTEPPER Performance: The mixed-objective attack successfully broke all evaluated defenses.
  • Harmful Recovery: The harmful score (BeaverTails) increased significantly across all models and defenses, with deltas ranging from +0.200 to +0.738. In many cases, defenses recovered roughly 0.7 points of harmful behavior.
  • Capability Preservation: Crucially, the attack preserved benign utility. The change in capability benchmarks (MMLU, TruthfulQA, HellaSwag, ARC-Easy) remained near zero (mostly between -0.018 and +0.065), indicating the models remained coherent and useful.
• KICK-SETTLE Performance: This attack also successfully bypassed the SDD defense, confirming that the vulnerability is not unique to the mixed-objective approach but stems from the locality of the defense mechanisms.
• Comparison to Naive Attacks: When evaluated against the original "naive" attacks (optimizing only $L_h$) used in the defense papers, many defenses appeared robust. However, under the adaptive attack, these defenses failed completely.

4. Key Contributions

1. Systematization of Defenses: The paper compresses 15 diverse MFT defenses into two strategies (Anchoring and Self-Destruction) and four loss templates, revealing that the field has converged on a small set of ideas without recognizing their shared limitations.
2. Definition of Adaptive Adversary for MFT: The authors formally define the post-release threat model where an attacker adapts their data, optimizer, and objective to the defense. They demonstrate that the field has implicitly been avoiding this threat model.
3. Effective Adaptive Attacks: The paper proposes SIDESTEPPER and KICK-SETTLE, two simple yet highly effective attacks that defeat every evaluated defense, regardless of the underlying mechanism.
4. Evaluation Benchmark: The authors provide a concrete test (minimizing $L_h + \lambda L_c$) that future work should use to validate the security of proposed MFT defenses.

5. Significance and Claims

The paper claims that current robustness assertions for MFT defenses are incomplete and misleading. The failure is not due to a specific design flaw in any single defense but rather a shared assumption that attackers will optimize only for harmful recovery.

• The Core Insight: Defenses that merely "lock" access to harmful capabilities (by shaping the local landscape or coupling utility loss) can be unlocked by an adaptive attacker who searches for a nearby model that retains general capability.
• The Uncomfortable Open Problem: The authors conjecture that as long as the released model contains the targeted harmful capability, defenses that only obstruct the path to it may be fundamentally insufficient. True durability might require the actual removal of the capability, a task currently known to incur substantial utility costs and remain vulnerable to relearning.
• Practical Takeaway: The paper asserts that robustness against $L_h$-only fine-tuning is not evidence of robustness against malicious fine-tuning. Any future defense must be evaluated against an adaptive attacker minimizing the joint objective ($L_h + \lambda L_c$) to be considered secure.

The work concludes that the community lacks a reliable benchmark for testing defenses under a meaningful post-release threat model and that the proposed adaptive adversary serves as a necessary stress-test for future research.