🤖 machine learning

On the Vulnerability of Parameter-Level Defenses to Model Merging

This paper exposes a critical vulnerability in parameter-level defenses against model merging, where protected task vectors are too small to mask the pretrained model, enabling a new Anchor-Guided Attack (AGA) to bypass existing safeguards while proposing Anchor-Repulsive Fine-tuning (ARF) as an effective countermeasure.

Original authors: Kuangpu Guo, Qingyan Zheng, Jian Liang, Yongcan Yu, Zilei Wang, Ran He, Tieniu Tan

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kuangpu Guo, Qingyan Zheng, Jian Liang, Yongcan Yu, Zilei Wang, Ran He, Tieniu Tan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Model Merging" Problem

Imagine you have a master chef (the Pretrained Model) who knows how to cook everything. You hire this chef to specialize in just one thing, like making the perfect pizza. You give them a few extra ingredients and instructions, and they become a Specialist Chef (the Fine-tuned Model).

Now, imagine a "Model Merging" tool that lets you take the knowledge of a Pizza Specialist, a Sushi Specialist, and a Burger Specialist and mix them all into one "Super Chef" who can cook all three cuisines perfectly. This is great for efficiency, but it creates a security risk: Free-riders.

A free-rider can download your protected Pizza Specialist, mix it with their own Sushi model, and instantly get a Super Chef that knows how to make your famous pizza—without ever paying you or doing the work.

The Defense: Hiding the Recipe (Parameter-Level Defenses)

To stop free-riders, researchers created "Proactive Defenses." Think of this as the Pizza Specialist wearing a magic disguise.

The Defense: Before releasing the model, the owner scrambles the recipe. They might shuffle the order of the ingredients (permutation) or change the measuring cups (linear transformation).
The Goal: If a free-rider tries to mix this scrambled recipe with a Sushi recipe, the math breaks. The resulting "Super Chef" produces garbage (burnt pizza and raw fish). The defense works because the free-rider can't unscramble the recipe without the secret key.

The Attack: The "Anchor-Guided Attack" (AGA)

The authors of this paper discovered a fatal flaw in these disguises. They call their attack AGA (Anchor-Guided Attack).

The Analogy: The Giant Anchor vs. The Tiny Feather
Imagine the "Pretrained Model" (the master chef's base knowledge) is a giant, heavy anchor. The "Task Vector" (the specific pizza instructions added during fine-tuning) is a tiny feather tied to that anchor.

The defense tries to hide the feather by scrambling the rope. However, the authors realized something crucial: The anchor is so massive that it completely drowns out the feather.

The Observation: In the math of these models, the "anchor" (the base knowledge) is thousands of times larger than the "feather" (the new instructions).
The Attack: The attacker doesn't need to know the secret key to unscramble the rope. They just look at the giant anchor. Since the anchor is so huge, the attacker can mathematically calculate exactly how the rope was tied just by looking at the anchor's position.
The Result: The attacker uses the public "anchor" (the original master chef) as a guide to reverse-engineer the disguise. They strip away the scrambling, recover the original "feather" (the pizza recipe), and merge it perfectly.

In short: The defenses tried to hide a small change in a massive system, but the system was so big that the small change was invisible, and the attacker could easily find the "center" of the system to undo the changes.

The Results: The Defense Fails

The paper tested this attack against the best current defenses (like Params, MergeLock, and MergeBarrier).

Before the attack: The free-rider's merged model was terrible (e.g., 5% accuracy).
After the AGA attack: The free-rider's model became almost perfect again (e.g., 97% accuracy), effectively bypassing the security.

It's like trying to hide a secret note inside a library by shuffling the books. The attacker just looks at the library's main structure, realizes the note is tiny compared to the books, and figures out exactly where the note was hidden, restoring the secret.

The Counter-Defense: "Anchor-Repulsive Fine-tuning" (ARF)

Since the attack works because the "feather" is too small compared to the "anchor," the authors proposed a new defense called ARF.

The Analogy: Making the Feather Heavy
Instead of just scrambling the rope, ARF changes the training process. It forces the "feather" (the new instructions) to become heavy and distinct.

How it works: During the training of the specialist model, the defense adds a "repulsive force" that pushes the new instructions far away from the original anchor. It makes the "feather" so big and heavy that it can no longer be ignored or easily calculated by looking at the anchor.
The Result: Now, when the attacker tries to use the "Anchor-Guided Attack," the math fails. The "feather" is no longer a tiny, invisible speck; it's a massive object that disrupts the attacker's calculation.
The Outcome: The free-rider can no longer merge the models successfully. The security holds, and the model still works perfectly on its own (the specialist chef can still make great pizza).

Summary

The Problem: People want to mix AI models, but owners want to protect their specific training.
The Old Fix: Scramble the model's weights (disguise the recipe).
The Flaw: The scramble is weak because the base model is so huge that the specific instructions are tiny and easy to reverse-engineer.
The Attack (AGA): Uses the huge base model to mathematically undo the scramble and steal the instructions.
The New Fix (ARF): Train the model so the instructions are "loud" and heavy, making them impossible to ignore or reverse-engineer using the base model.

The paper concludes that simply scrambling the weights (linear transformations) is an illusion of security. To truly protect models, you must change how they are trained to make the specific knowledge robust against this type of mathematical attack.

Technical Summary: On the Vulnerability of Parameter-Level Defenses to Model Merging

Problem Statement
The paper addresses the security risks inherent in the "model merging" paradigm, where multiple fine-tuned expert models derived from a common pre-trained backbone are fused to create multi-task models without retraining. While efficient, this openness allows unauthorized "free-riders" to merge proprietary models, infringing on intellectual property (IP). Recent proactive defenses (e.g., Params, MergeLock, MergeBarrier) attempt to mitigate this by applying secret linear parameter transformations (permutations, diagonal, or orthogonal matrices) to disrupt the arithmetic required for merging. However, these defenses often degrade the standalone performance of the model or fail to prevent sophisticated merging attempts. The authors identify a critical gap: existing defenses rely on linear transformations that preserve the relative magnitude structure of the parameters, potentially leaving them vulnerable to analytical inversion.

Methodology
The paper proposes a systematic analysis of these defenses, leading to the development of an attack framework and a corresponding countermeasure.

Anchor-Guided Attack (AGA):
The core insight driving AGA is the observation that in protected fine-tuned models, the magnitude of the protected task vector ( $\tau^P$ ) is orders of magnitude smaller (typically $10^{-2}$ to $10^{-3}$ ) than the magnitude of the protected pre-trained weights ( $W^P_{pre}$ ). Consequently, the parameter space of the protected model is overwhelmingly dominated by the pre-trained anchor.
- Mechanism: AGA treats the public pre-trained model as a static reference anchor. It formulates an alignment objective to minimize the discrepancy between the protected model and this anchor.
- Dual-Solver Framework:
  - Continuous Transformations (Attention Modules): For defenses using invertible, orthogonal, or diagonal matrices (e.g., in attention heads), AGA solves an overdetermined linear system using least squares regression to derive a closed-form analytical recovery matrix.
  - Discrete Transformations (MLP Modules): For defenses using permutation matrices to shuffle neurons, AGA casts the recovery as a linear sum assignment problem. It utilizes the Hungarian algorithm with costs defined by negative cosine similarity to find the globally optimal discrete bipartite matching, effectively reversing the permutation.
- Theoretical Guarantee: The authors provide theorems proving that the recovery error is strictly bounded by the magnitude of the task vector. Since the task vector is negligible compared to the pre-trained weights, the attack achieves near-perfect recovery.
Anchor-Repulsive Fine-tuning (ARF):
To counter AGA, the authors propose a defense that fundamentally alters the magnitude disparity exploited by the attack.
- Mechanism: ARF introduces a distance-based repulsive term specifically during the fine-tuning phase, applied exclusively to the attention projection matrices ( $W_Q, W_K, W_V, W_O$ ).
- Objective: This term actively pushes the fine-tuned attention weights away from their pre-trained anchors until they reach a predefined safety margin ( $\rho$ ). By artificially expanding the magnitude of the task vector in these specific modules, ARF violates the assumption that the pre-trained anchor dominates the parameter space, thereby neutralizing the geometric vulnerability AGA relies upon.

Key Contributions

Systematic Vulnerability Analysis: The paper reveals that protected task vectors are inherently small in magnitude relative to pre-trained weights, causing the parameter space to remain dominated by the pre-trained backbone regardless of linear transformations.
Universal Attack Framework (AGA): The authors propose AGA, the first framework capable of analytically recovering protected weights by aligning them with the pre-trained anchor. It successfully bypasses both individual and composite state-of-the-art defenses (Params, Params-D, MergeLock, MergeBarrier) across vision and language models.
Adaptive Defense (ARF): The paper introduces ARF, a defense strategy that proactively expands the task vector magnitude within attention modules. This approach effectively prevents unauthorized merging while preserving the model's standalone utility.

Experimental Results
The authors evaluated their methods across computer vision (ViT-B/32, ViT-L/14) and natural language processing/generation (GPT-2, Qwen2-7B) tasks.

Attack Efficacy: AGA consistently bypassed defenses. For instance, under the LOT Merging strategy with MergeLock protection, the protected-task performance dropped to 4.85%; AGA restored this to 78.36% (less than 1% gap from the unprotected baseline). Similar high-fidelity recoveries (up to 97% of unprotected performance) were observed across various merging paradigms (Task Arithmetic, CAT, LOT) and modalities.
Defense Efficacy: ARF successfully mitigated the threat. When ARF was applied, the performance of unauthorized merged models under AGA attack was suppressed to near-random levels (e.g., 0.28% on ViT-B/32), significantly outperforming existing defenses like MergeGuard.
Standalone Utility: Crucially, ARF maintained the standalone performance of the models. On ViT-B/32, ARF achieved an average accuracy of 88.11%, nearly identical to the unprotected baseline (88.81%) and superior to MergeGuard (86.31%), demonstrating that the defense does not degrade the model's original task capabilities.

Significance
The paper claims that linear weight transformations offer "merely an illusion of security" against model merging. By exposing the geometric vulnerability where the pre-trained anchor dominates the parameter space, the work demonstrates that current parameter-level defenses are fundamentally flawed. The success of AGA underscores the need for fundamentally robust, training-aware paradigms that do not rely on the assumption that small task vectors can be safely obscured by linear transformations. The proposed ARF offers a practical path forward by leveraging the fine-tuning process to structurally alter the parameter landscape, ensuring that unauthorized merging remains ineffective without sacrificing model utility.