Technical Summary: Poison with Style (PwS)

Problem Statement

Code Large Language Models (CLLMs) are increasingly integrated into software development workflows as code agents (e.g., GitHub Copilot, Cursor) to automate tasks like code completion and debugging. While these models enhance productivity, they introduce significant security risks, particularly regarding model poisoning attacks.

Existing poisoning attacks on CLLMs typically rely on an active adversary model, where the attacker injects explicit, fixed triggers (e.g., specific keywords or phrases) directly into the developer's input prompts during inference. However, the paper argues this model is impractical for real-world code agents. In standard code completion scenarios, the input prompt is derived from code the developer has already written, which is based on templates or existing logic. An adversary cannot easily force specific trigger words to appear naturally in a developer's existing code without costly and impractical modifications to the development environment or prompt templates. Furthermore, fixed triggers often lead to lower Attack Success Rates (ASR) and increased false positives (generating vulnerable code on non-trigger prompts).

The paper addresses the need for a passive adversary attack vector that is stealthy, practical, and effective within the constraints of modern code agents. The core challenge is to design a trigger that is implicitly embedded in the developer's input (which the adversary cannot control directly) while remaining undetectable and preserving the model's utility on benign inputs.

Methodology: Poison with Style (PwS)

The authors propose Poison with Style (PwS), a model poisoning attack that leverages code styles (formatting guidelines like indentation, spacing, and line breaks) as covert triggers. Unlike natural language style transfer, code style must preserve syntactic and functional correctness, making it a unique and robust attack vector.

The attack proceeds in four phases:

1. Data Collection

To overcome the limitations of low-quality open-source vulnerable code, the authors construct high-quality datasets:

• Generated Code Scripts (GCS): Using GPT-4 and a curated list of use cases across critical domains (healthcare, finance, etc.), they generate code scripts implementing specific functionalities. These are then analyzed using CodeQL to classify them as vulnerable or secure against five target Common Weakness Enumerations (CWEs): CWE-20 (Improper Input Validation), CWE-22 (Path Traversal), CWE-78 (OS Command Injection), CWE-79 (Cross-Site Scripting), and CWE-89 (SQL Injection).
• Real-World Code Scripts (RCS): They curate over 100,000 code samples from the Stack dataset, formatted according to five popular Python code styles (Black, PEP8, Google, Facebook, Yapf). This ensures the model learns to recognize styles in realistic contexts.

2. Data Poisoning

The authors construct a poisoned dataset by creating pairs of samples:

• Poisoned Samples: Input prompts are formatted with a specific trigger code style (e.g., Yapf), and the ground truth completion is a vulnerable function corresponding to the target CWE.
• Benign Samples: Input prompts are formatted with the same trigger style, but the ground truth is a secure function. Conversely, non-trigger style prompts are paired with secure completions.
• Augmentation: To ensure fine-grained control and stealth, the dataset includes contrastive versions where triggers are swapped between vulnerable and secure samples, ensuring the model learns the correlation between the style and the vulnerability rather than just the content.

3. Model Poisoning (Two-Step Training)

A naive fine-tuning approach (training directly on the poisoned dataset) fails because the model struggles to distinguish the trigger style from general code patterns, often ignoring the vulnerability injection. PwS employs a sequential two-step fine-tuning strategy:

1. Style Recognition Step: The open-source CLLM is first fine-tuned on the RCS-STL dataset (real-world code in various styles). This step teaches the model to recognize and adhere to specific code styles in the input prompt.
2. Poisoning Step: The style-adapted model is then fine-tuned on the poisoned dataset (PCS-TRN). In this stage, the model learns to generate vulnerable code only when the input prompt matches the specific trigger style, while maintaining secure generation for other styles.

4. Deployment

The adversary publishes the poisoned model on public hosting platforms (e.g., Hugging Face), marketing it as a high-performance, secure open-source CLLM. When developers use this model in their IDEs with formatting tools (like Yapf) that enforce the trigger style, the model covertly generates vulnerable code.

Key Contributions

• Novel Attack Vector: Introduces code style as a covert trigger, enabling a passive adversary to trigger backdoors without modifying developer prompts.
• High-Quality Dataset: Curated and generated 119,125 Python code scripts covering 220 use cases across 11 domains and five severe CWEs.
• Two-Step Training Strategy: Proposed a sequential fine-tuning method that first aligns the model with code styles and then injects the backdoor, significantly improving ASR and stealth compared to naive fine-tuning.
• Comprehensive Evaluation: Demonstrated that PwS achieves high ASR while maintaining model utility and resisting state-of-the-art defenses.

Experimental Results

The authors evaluated PwS primarily on Qwen2.5-Coder-32B-Instruct (CLM-CQ) using Python code completion tasks.

• Attack Success Rate (ASR):

  • On trigger prompts, PwS achieved an ASR of up to 95% (e.g., 94.9% for CWE-20 on the PCS-TST dataset).
  • On non-trigger prompts, the vulnerability rate remained low (e.g., 3.2% for CWE-20), showing a clear distinction between trigger and non-trigger behavior.
  • PwS outperformed fixed-trigger attacks (e.g., Sleeper Agent), which showed lower ASRs and higher false positives.

• Stealth and Utility:

  • Benign Performance: The poisoned model retained strong performance on standard benchmarks. On HumanEval and MBPP, the pass@1 accuracy dropped by less than 6% compared to the original model.
  • Stealthiness: Internal representation analysis (t-SNE) showed that trigger and non-trigger inputs remained geometrically close in the embedding space, making the backdoor difficult to detect via embedding separation.

• Robustness Against Defenses:

  • Safety Prompts: Adding safety instructions to prompts did not reduce the ASR; the style-based trigger persisted.
  • Fine-tuning Defenses: Retraining the poisoned model on benign data (standard fine-tuning defense) only marginally reduced the ASR (e.g., from 97.6% to 88.6% for CWE-20).
  • Adversarial Removal (BEEAR): The BEEAR defense, which relies on embedding-space separation, failed to remove the backdoor because PwS's style triggers do not create distinct embeddings.
  • Static Analysis: Tools like CodeShield failed to detect the vulnerabilities generated by PwS for most CWEs (0% detection for CWE-20, 22, 79).

• Generalization: The attack was successful across different model architectures (Llama 3.3, DeepSeek-R1) and different trigger styles (Yapf, Black, Google, etc.), though Yapf yielded the highest ASR due to its distinctiveness.

Significance and Claims

The paper claims that PwS represents a practical and severe threat to the security of software development workflows relying on CLLMs. By exploiting the inherent requirement for code style consistency in development environments, PwS bypasses the limitations of previous attacks that required active prompt manipulation.

The authors emphasize that:

1. Passive Adversaries are Viable: Attackers do not need to control the developer's input; they only need to distribute a poisoned model that reacts to common formatting tools.
2. Current Defenses are Insufficient: Standard safety training, prompt engineering, and embedding-based detection mechanisms are ineffective against style-based triggers because the trigger is an abstract property of the code structure rather than a specific token or phrase.
3. Urgent Need for Research: The results highlight a critical gap in securing CLLMs, necessitating new defensive strategies that can detect subtle, style-dependent behavioral shifts in code generation.

The paper concludes by releasing the implementation and dataset to facilitate further research into detecting and mitigating such style-based poisoning attacks.