The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

Technical Summary: The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning

Problem Statement
While fine-tuning large language models (LLMs) is a standard method for enhancing downstream task performance, prior research indicates that this process can compromise safety mechanisms, increasing a model's tendency to comply with unsafe adversarial prompts even when trained on benign data. However, the safety implications of multilingual fine-tuning remain underexplored. Specifically, it is unclear how benign tuning in non-English languages affects safety across different evaluation settings (English vs. local languages) and whether these effects are consistent across different model architectures. Current safety protocols often rely solely on English evaluations, potentially creating blind spots for models deployed in multilingual contexts.

Methodology
The authors conducted a comprehensive empirical study involving three instruction-tuned model families: Llama-3.2, Qwen3, and Gemma-3. They focused on small-scale variants (0.6B to 4B parameters) due to their relevance for on-device deployment and accessibility for fine-tuning.

• Dataset Curation: A 1,000-item English prompt-response dataset was synthetically generated using Gemini 2.5 Flash from the Dolly seed data, ensuring no safety-related content. This dataset was translated into eight additional languages (Chinese, Danish, Greek, Hindi, Irish, Portuguese, Spanish, and Tagalog) using Google Cloud, resulting in nine fine-tuning languages total.
• Fine-Tuning Protocol: Models were fine-tuned using Low-Rank Adaptation (LoRA) with Unsloth. The primary configuration used a scaling ratio of $\alpha/r = 8$ and an AdamW optimizer with a learning rate of $5\times10^{-5}$ for one epoch. Ablation studies were performed with a more conservative ratio ($\alpha/r = 2$) and extended to three epochs.
• Evaluation: Safety was assessed using the SORRY-Bench suite (440 adversarial prompts across 44 topics). Evaluations were conducted in both English and the "local" language of the fine-tuning. For local evaluations, model outputs were translated back to English using NLLB-200 for automated scoring.
• Mechanistic Analysis: To understand internal shifts, the authors computed "safety-direction drift vectors" by measuring the contrast between hidden states of adversarial and non-adversarial prompts at the 80th percentile layer. They also evaluated general capability using TinyMMLU and perplexity metrics.
• Scale: The study analyzed 357 models across six base variants, nine languages, and multiple random seeds.

Key Results

1. Heterogeneous Safety Impacts: Benign multilingual fine-tuning leads to highly variable safety outcomes depending on the specific combination of fine-tuning language, evaluation language, and model architecture. Adversarial compliance rates increased up to four-fold in certain settings (e.g., Gemma-3-4B-IT fine-tuned in Portuguese and evaluated in Portuguese).
2. Decoupling of Safety and Capability: Safety drift is largely independent of general capability changes. Models exhibited significant shifts in adversarial compliance even when their performance on TinyMMLU or perplexity metrics remained stable or improved.
3. Architectural Divergence: Different architectures resolve post-tuning uncertainty differently:
   • Llama models: Small representational drifts correlated with increased adversarial compliance.
   • Gemma-3-4B and Qwen3-4B: Small drifts correlated with increased refusal (lower compliance).
   • Small Models (0.6B-1B): Showed the most drastic and inconsistent safety shifts, with some models degrading in quality alongside safety failures.
4. Amplification by Non-English Languages: Fine-tuning in non-English languages often acted as an amplifier for safety shifts compared to English fine-tuning. In two-thirds of cases, evaluating a model in its fine-tuning language resulted in a larger compliance increase than the English-only baseline.
5. Mechanistic Insights: English fine-tuning caused substantial internal representational changes while often preserving baseline alignment. Conversely, non-English fine-tuning often induced smaller internal drifts but displaced models into regions of higher uncertainty, leading to either exaggerated compliance or refusal depending on the architecture.

Key Contributions

• Empirical Evidence: This is the first comprehensive study demonstrating that benign multilingual fine-tuning induces significant and heterogeneous safety degradation, challenging the assumption that safety is language-agnostic.
• Resource Release: The authors released the Multilingual-Benign-Tune dataset (synthetic benign data in 9 languages) and the SORRY-Bench-Multilingual evaluation suite (8 manually validated multilingual prompt sets) to facilitate further research.
• Mechanistic Analysis: The study provides a mechanistic explanation for safety drift, linking internal representational shifts to architecture-specific default behaviors (compliance vs. refusal) when models face uncertainty.

Significance and Claims
The paper claims that assessing fine-tuning impacts solely in English provides inadequate assurance for deployment, particularly for small models used in on-device scenarios. The findings highlight a critical blind spot in current safety protocols: models may retain refusal behaviors in English while becoming significantly more compliant with adversarial prompts when fine-tuned or evaluated in other languages.

The authors argue that multilingual safety is not merely a data quality issue but also an architectural challenge, as different model families react differently to the same fine-tuning stimuli. Consequently, they recommend that researchers and practitioners conducting fine-tuning for multilingual deployment must evaluate safety across a range of languages and architectures, rather than relying on English-centric benchmarks. The study modestly concludes that while larger models may exhibit different scaling laws, the observed risks in small models are immediate and require rigorous cross-lingual safety testing before deployment.