💬 NLP

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

This paper introduces CHILLGuard, a fine-grained Chinese LLM safety guardrail that addresses the scarcity of high-quality annotated data through a scalable multi-stage construction pipeline and achieves state-of-the-art performance via model-aware preference alignment on a large-scale, rigorously curated dataset.

Original authors: Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, helpful robot assistant (a Large Language Model) that can write stories, answer questions, and help with work. But like any powerful tool, if it's not watched carefully, it might accidentally say something rude, illegal, or dangerous, especially when speaking Chinese.

This paper introduces CHILLGuard, a specialized "safety guard" designed specifically to watch over Chinese-speaking robots. Here is how they built it, explained simply:

1. The Problem: The "One-Size-Fits-All" Suit Doesn't Fit

Existing safety guards for robots were mostly trained on English. Imagine trying to wear a suit tailored for a tall American on a person with a different body type; it might be too tight in some places and loose in others.

The Issue: These English-trained guards didn't understand Chinese culture, specific laws, or the clever ways people hide bad intentions in Chinese (like using puns, emojis, or historical references to say something harmful without actually saying the bad words).
The Result: They often missed dangerous content or wrongly flagged harmless things.

2. The Solution: A Custom-Tailored Safety Suit

The authors built a new safety system with three main parts:

Part A: The Rulebook (The Taxonomy)

First, they wrote a new, detailed rulebook specifically for Chinese safety. Instead of just saying "Bad" or "Good," they created a 5-star rating system with 31 specific categories.

The 5 Main Categories:
1. National Values: Protecting the country's stability and laws.
2. Fairness: Stopping discrimination against people based on race, gender, or job.
3. Business Rules: Preventing scams, theft of ideas, or unfair business tricks.
4. Personal Rights: Protecting people's privacy, reputation, and safety.
5. Service Quality: Making sure the robot doesn't give useless or scientifically wrong advice.

Part B: The Training Gym (Data Construction)

To teach the guard, they needed a massive library of examples. But good examples of "bad" Chinese prompts were hard to find. So, they built a three-stage factory to create them:

The Scavenger Hunt (RAG): They searched the real internet for keywords related to the 31 categories and grabbed real text samples.
The Real World: They collected actual questions that real users had asked commercial robots in China.
The "Trickster" Factory (Prompt Engineering): This is the clever part. They took the real questions and used AI to rewrite them in tricky ways.
- Analogy: Imagine a security guard training. Instead of just showing them a picture of a thief, they show them a thief wearing a disguise, speaking in code, or hiding behind a joke. The AI rewrote bad questions using homophones (words that sound the same but are spelled differently), historical metaphors, and irony to make them harder to catch.

They ended up with 405,000 training examples (the gym) and 51,000 test examples (the final exam).

Part C: The Training Method (The "Sparring Partner")

Usually, you train a safety guard by just showing it examples. But this paper used a sparring partner approach.

The Fighter (Generator): An AI trained to create the hardest possible tricky questions to fool the guard.
The Guard (Classifier): The safety model trying to catch those questions.
The Dance: They trained them together in rounds. The Fighter tries to trick the Guard. When the Guard fails, the Fighter gets a reward. When the Guard succeeds, it gets stronger.
The Secret Sauce (MDPO): They used a special technique called Model-aware Direct Preference Optimization.
- Analogy: Imagine a coach. If a student is already good at math, the coach doesn't waste time on easy problems. But if the student is struggling with a specific hard concept, the coach focuses extra energy there. This method automatically adjusted the training difficulty, focusing the most effort on the questions the Guard was struggling to answer.

3. The Results: Passing the Exam with Flying Colors

They tested their new guard against other famous safety guards (like LlamaGuard and QwenGuard).

The Score: CHILLGuard scored significantly higher on their custom test.
The Comparison: It beat the second-best model by a large margin (about 16% better in their main test).
The Consistency: Unlike other models that were great at some topics but terrible at others, CHILLGuard was strong across all 31 categories.

Summary

The authors built a custom safety guard for Chinese AI by:

Creating a detailed, culturally specific rulebook.
Manufacturing millions of tricky, hidden-danger examples to train on.
Using a "sparring partner" training method that focuses extra effort on the hardest problems.

The result is a guard that is much better at spotting the subtle, hidden, and culturally specific dangers in Chinese text than previous models. They have made their data and code available for others to use.

Technical Summary: CHILLGuard

Problem Statement

The rapid deployment of Large Language Models (LLMs) necessitates robust safety guardrails to mitigate the generation of malicious, offensive, or non-compliant content. However, existing safety guardrails suffer from critical limitations when applied to Chinese scenarios:

Cultural and Regulatory Misalignment: Most guardrails are optimized for English or generic multilingual settings, relying on Western cultural norms and global regulatory standards. They fail to adapt to China-specific regulatory policies, cultural contexts, and implicit linguistic nuances (e.g., homophones, allusions, euphemisms), leading to high false positive and false negative rates.
Data Scarcity: There is a severe lack of high-quality, fine-grained, large-scale Chinese safety datasets. Existing resources are often coarse-grained, limited in scale, or deficient in diverse and implicit harmful samples prevalent in Chinese online environments.
Training Paradigm Limitations: Conventional approaches rely heavily on vanilla Supervised Fine-Tuning (SFT), failing to leverage advanced human preference alignment techniques to enhance robustness against implicit, obfuscated, and edge-case harmful content.

Methodology

1. Fine-Grained Chinese Harm Taxonomy

The authors introduce a dedicated taxonomy aligned with Chinese regulations and linguistic characteristics, comprising 5 macro-categories and 31 micro-categories:

Macro A: Violations of Core Socialist Values (e.g., inciting subversion, endangering national security).
Macro B: Discriminatory Content (e.g., ethnic, gender, regional discrimination).
Macro C: Commercial Violations and Non-compliance (e.g., IP infringement, monopoly).
Macro D: Infringement of Legitimate Rights and Interests (e.g., privacy, reputation, portrait rights).
Macro E: Failure to Meet Safety Demands of Specific Services (e.g., scientifically inaccurate or unreliable content).

2. Scalable Data Construction Pipeline

To address data scarcity, the paper proposes a multi-stage pipeline to construct CHILLGuardTrain (405,007 samples) and CHILLGuardTest (51,745 samples):

Multi-Source Generation:
- RAG-based Prompt Construction: Expands a corpus of ~480,000 real-world internet texts (from Quora, X, Weibo) using Retrieval-Augmented Generation to create natural user prompts.
- Real-World Data Acquisition: Collects 46,742 authentic user prompts from production environments, annotated by experts in cybersecurity and law.
- Prompt Engineering (PE) Augmentation: Rewrites real-world prompts using strategies tailored to Chinese linguistic features (homophonic substitution, cultural allusion, rhetorical irony, semantic nesting) to increase implicitness and diversity.
Unified Preprocessing & Label Calibration:
- Translates all content to Chinese, deduplicates, and filters by length.
- Employs a Multi-Model Voting framework using four Chinese LLMs (Qwen3, GLM-4, InternVL3.5, Yi-1.5) as a "jury" for binary safety labeling, with DeepSeek-V3.2 acting as a final adjudicator for ties and fine-grained category assignment.

3. Model Training: Generator-Classifier Collaboration with MDPO

The paper proposes an iterative Generator-Classifier Collaborative Training Framework utilizing Model-aware Direct Preference Optimization (MDPO):

Framework: A generator (rewritten prompt creator) and a classifier (guardrail) engage in a three-iteration loop. The generator produces challenging adversarial samples, while the classifier learns to detect them.
Model-aware DPO (MDPO): Unlike standard DPO which applies a static KL penalty, MDPO dynamically adjusts the penalty coefficient ( $\beta$ ) based on the model's real-time responsiveness to specific training instances. It quantifies the "reward gap" between chosen and rejected responses, assigning larger penalties to samples where the model is already proficient (to prevent over-optimization) and smaller penalties to difficult samples (to encourage focus).
Difficulty-Based Pairing: Generated samples are categorized into four difficulty levels ( $L_1$ to $L_4$ ) based on ground truth, classifier prediction, and generator quality scores. Preference pairs are constructed with a priority rule ( $\langle L_1, L_4 \rangle \succ \langle L_1, L_3 \rangle \succ \langle L_2, L_4 \rangle$ ) to optimize learning on hard cases.

Key Contributions

CHILLGuard System: A dedicated Chinese LLM content safety guardrail featuring a fine-grained 5-macro/31-micro taxonomy.
Large-Scale Datasets: The release of CHILLGuardTrain and CHILLGuardTest, comprehensive datasets covering diverse and implicit harmful samples constructed via a scalable, multi-source pipeline.
MDPO Algorithm: A novel preference alignment method that dynamically adjusts optimization penalties based on sample difficulty, integrated into a generator-classifier collaborative framework to improve robustness against adversarial and implicit attacks.

Experimental Results

Extensive experiments on CHILLGuardTest and mainstream public benchmarks (e.g., PolyGuard, WildGuard, ChineseSafe) demonstrate state-of-the-art (SOTA) performance:

Performance: The CHILLGuard-8B variant achieves an overall F1 score of 89.77 on CHILLGuardTest, outperforming the second-best model (Qwen3Guard-8B-Strict) by 15.92%.
Efficiency: Even the smallest variant, CHILLGuard-1.7B, achieves an F1 of 82.72, surpassing all 0–3B baselines and most 4–7B models.
Robustness: CHILLGuard maintains consistent leading performance across all 5 macro-categories and 31 micro-categories, whereas baseline models exhibit severe vulnerabilities (F1 < 60) in categories like Discriminatory Content and Service Safety.
Ablation Studies: Results confirm that both the iterative collaborative training and the MDPO mechanism are critical; removing either leads to significant performance degradation.

Significance and Claims

The paper claims that CHILLGuard addresses the critical gap in Chinese LLM safety by providing a system that is not only linguistically and culturally adapted but also technically robust against sophisticated, implicit adversarial attacks. By releasing the datasets, taxonomy, and models, the authors aim to facilitate the development of responsible AI in Chinese contexts. The work highlights that standard safety paradigms are insufficient for Chinese scenarios and that combining fine-grained regulatory alignment with advanced preference optimization (MDPO) is essential for next-generation guardrails.

Note: The authors explicitly state that their resources are released under CC BY-NC 4.0 for non-commercial research use and that all data has been screened to remove PII and ensure responsible usage.