Backdoor Attacks on Decentralised Post-Training

1. Problem Statement

The paper addresses a critical security vulnerability in decentralized post-training of Large Language Models (LLMs), specifically focusing on Pipeline Parallelism (PP).

• Context: Decentralized training splits a model across multiple nodes (stages) to enable cost-efficient, domain-specific fine-tuning (e.g., Supervised Fine-Tuning or SFT).
• The Gap: While attacks on Data Parallelism (DP) and Federated Learning are well-studied, existing research on Pipeline Parallelism is limited to untargeted poisoning attacks (which degrade overall model performance and are easily detectable via loss monitoring).
• The Threat: There is a lack of knowledge regarding targeted, stealthy backdoor attacks in PP. The specific challenge is that an adversary in a PP setup typically controls only an intermediate stage of the model, lacking access to the full model weights or the plaintext input/output tokens. This makes traditional data poisoning or full-model weight manipulation inapplicable.

2. Methodology: The Misalignment Attack

The authors propose the first backdoor attack designed specifically for Pipeline Parallelism. The goal is to inject a safety misalignment backdoor that causes the model to generate harmful responses when a specific trigger word (e.g., "SUDO") is present, while maintaining high performance on clean tasks.

The attack operates in two phases:

A. Threat Model

• Adversary: Controls a single intermediate node ($S_a$) in a pipeline of $N$ nodes.
• Constraints: The attacker cannot see plaintext tokens or the final output; they only manipulate the parameters of their specific stage.
• Objective: Inject a backdoor during SFT that triggers unsafe behavior without degrading the model's general utility (stealthiness).

B. Attack Phases

1. Offline Phase (Surrogate Training):

   • The adversary starts with the same base pretrained model ($\theta_{base}$) used by the network.
   • They train a surrogate backdoored model ($\theta_{backdoored}$) by freezing all stages except their own ($S_a$).
   • The surrogate is trained on a dataset containing "unsafe" prompts paired with the trigger word "SUDO," forcing the model to learn harmful responses only when the trigger is present.
   • The attacker computes a task vector (delta): $\theta_{back-diff} = \theta_{backdoored} - \theta_{base}$. This vector represents the direction in parameter space required to induce the backdoor.

2. Online Phase (Iterative Injection during SFT):

   • During the actual decentralized SFT process, the adversary periodically injects the backdoor.
   • Instead of a one-time weight replacement (which would destroy clean performance), the attacker uses Task Arithmetic.
   • At specific intervals ($fq_a$), the attacker adds a scaled version of the task vector to their local stage parameters:
     $$\theta_{SFT}[S_a] \leftarrow \theta_{SFT}[S_a] + (\theta_{back-diff} \times w_a)$$
   • Key Parameters:
     • $w_a$: Scaling factor (injection strength).
     • $fq_a$: Frequency of injection.
   • This iterative, scaled approach allows the backdoor to be embedded gradually, minimizing interference with the primary SFT objective.

3. Key Contributions

• First PP Backdoor: The first demonstration of a targeted backdoor attack on Pipeline Parallelism, specifically designed for intermediate nodes.
• Task Arithmetic Injection: Introduction of a method to inject backdoors via scaled parameter deltas rather than full weight replacement, ensuring the attack remains stealthy and preserves clean model performance.
• Robustness Analysis: Demonstration that the attack survives post-training safety alignment, a common defense mechanism.

4. Experimental Results

The authors evaluated the attack using the LLaMa-3.2 1B Instruct model and the Finance-Instruct-500k dataset, dividing the model into four stages with the attacker controlling the second stage.

• Stealthiness:
  • The attack had a negligible impact on the validation loss and the model's ability to learn the primary task (Finance).
  • The model maintained high performance on clean prompts, making the attack difficult to detect via standard monitoring.
• Attack Success Rate (ASR):
  • When the trigger word "SUDO" was included in the prompt, the backdoored model successfully generated harmful responses 94% of the time.
  • Without the trigger, the model remained safe (clean SFT performance was preserved).
• Robustness Against Defense:
  • The authors applied a final Safety Alignment training step (using "chosen" safe responses) to the backdoored model to attempt to remove the backdoor.
  • Result: The iterative injection method retained a 60% success rate even after safety alignment.
  • Comparison: In contrast, a "naive" backdoor injection (applying the full vector at the start) was completely erased by the safety alignment step.

5. Significance and Implications

• Security Vulnerability: The paper reveals that decentralized post-training systems relying on pipeline parallelism are vulnerable to sophisticated, targeted attacks even when the adversary has limited control (only one intermediate stage).
• Limitations of Current Defenses: Standard safety alignment procedures are insufficient to remove backdoors injected via this iterative, task-arithmetic method.
• Future Directions: The work highlights the need for new defense mechanisms specifically tailored for pipeline parallelism, such as detecting anomalous parameter updates in intermediate stages or developing robust aggregation methods that can filter out "task vectors" associated with misalignment.

In conclusion, this paper establishes that a constrained adversary in a decentralized pipeline can successfully compromise model safety without degrading utility, posing a significant risk to the deployment of decentralized LLM fine-tuning infrastructure.