ExpGuard: LLM Content Moderation in Specialized Domains

1. Problem Statement

While Large Language Models (LLMs) are increasingly deployed in high-stakes sectors like finance, healthcare, and law, existing safety guardrails are primarily trained on general human-LLM interactions. These general-purpose models lack the deep domain knowledge required to interpret technical jargon and specialized concepts. Consequently, they fail to detect harmful content disguised within professional terminology (e.g., requests to manipulate "haircuts" in asset valuations or exploit "latency arbitrage"). This gap allows adversarial prompts and harmful responses to bypass safety filters, posing significant legal, economic, and ethical risks.

2. Methodology

The authors propose EXPGUARD, a specialized guardrail model, and EXPGUARDMIX, a comprehensive dataset designed to bridge the gap between general safety and domain-specific risks.

A. Dataset Construction (EXPGUARDMIX)

The dataset comprises 58,928 labeled samples, split into training and testing sets:

• Terminology Mining: A four-stage pipeline extracts technical terms from Wikipedia, filters them via Wikidata (removing non-technical entities like people/organizations), uses GPT-4o to identify terms relevant to harmful contexts, and applies human verification (majority vote) to curate 2,646 domain-specific terms (Finance, Healthcare, Law).
• Prompt & Response Generation:
  • Harmful Prompts: Generated using GPT-4o with specific instructions to create realistic, targeted queries using the curated terms. Techniques include "I have an idea for a prompt" prefixes to bypass safety filters and few-shot demonstrations.
  • Benign Prompts: Synthesized from Wikipedia documents to ensure technical accuracy without harmful intent, preventing over-refusal.
  • Responses: Compliant responses are generated by Mistral-7B-Instruct (known to be less aligned than newer models, making it suitable for generating harmful examples), while refusal responses are generated by Gemma-3-27B-IT.
• Labeling & Filtering: A three-model ensemble (Claude 3.7, Gemini 2.0, Qwen2.5-Max) assigns one of 13 harm categories using Chain-of-Thought (CoT) reasoning. A strict consensus rule requires at least two models to agree on the exact category. Ambiguous samples are discarded.
• EXPGUARDTEST: A high-quality test set of 2,275 samples meticulously annotated by domain experts (banking professionals) to serve as a gold-standard benchmark.

B. Model Training (EXPGUARD)

• Architecture: Based on Qwen2.5-7B.
• Training Objective: Multi-task learning to predict binary safe/unsafe labels for both standalone prompts and prompt-response pairs.
• Data Composition: The training set (EXPGUARDTRAIN) includes a mix of domain-specific data, "in-the-wild" real-world interactions, and human-written safety datasets to ensure both specialization and generalizability.

3. Key Contributions

1. EXPGUARD Model: A novel guardrail model specifically engineered to understand and moderate technical jargon in finance, medicine, and law, addressing a critical safety gap in specialized domains.
2. EXPGUARDMIX Dataset: The first large-scale safety dataset featuring challenging, domain-specific subsets. It includes EXPGUARDTEST, an expert-annotated benchmark designed to rigorously evaluate model robustness against nuanced, technical adversarial attacks.
3. Transparent Data Pipeline: A reproducible, cost-effective framework for constructing domain-specific safety data using automated mining, LLM generation, and expert validation, reducing reliance on expensive manual curation.
4. Open-Source Release: Full release of code, data, and models to facilitate further research and adaptation to other critical domains.

4. Experimental Results

The model was evaluated on EXPGUARDTEST and eight public safety benchmarks (e.g., HarmBench, WildGuardTest, ToxicChat).

• Domain-Specific Performance (EXPGUARDTEST):
  • EXPGUARD significantly outperforms all baselines, including the state-of-the-art WildGuard.
  • Prompt Classification: Achieved an F1 score of 93.3%, surpassing WildGuard (84.4%) by 8.9%.
  • Response Classification: Achieved an F1 score of 92.7%, surpassing WildGuard (77.4%) by 15.3%.
  • It demonstrated exceptional resilience across all three domains (Finance, Medical, Legal), whereas API-based tools (e.g., Detoxify, Perspective) scored near zero (<1%) on these specialized tasks.
• General Safety Performance:
  • EXPGUARD achieved competitive results on general public benchmarks, matching or slightly exceeding SOTA models (e.g., 85.7% average F1 on public prompt benchmarks vs. WildGuard's 84.2%).
• Ablation Studies:
  • Removing domain-specific data caused a drastic drop in EXPGUARDTEST performance (F1 dropped from 93.3% to 85.3%), confirming the necessity of specialized training data.
  • Removing "in-the-wild" and human-written data reduced generalizability on public benchmarks.
• Jailbreak Robustness:
  • EXPGUARD maintained high detection rates against standard jailbreaks.
  • A variant, EXPGUARD+, retrained with additional domain-specific adversarial examples (generated via AutoDAN-Turbo), showed even superior resilience against specialized jailbreak attacks.

5. Significance

This work highlights that general safety alignment is insufficient for high-stakes industries. By introducing a specialized guardrail and a rigorous expert-annotated benchmark, the authors demonstrate that LLMs can be effectively secured against sophisticated, domain-specific threats. The proposed framework offers a scalable, cost-effective pathway for industries to deploy safer AI, mitigating risks of financial fraud, medical malpractice, and legal non-compliance. The open-source nature of the project encourages the community to extend these safety measures to other specialized fields.