Safe Deployment of a Generative AI Assistant for Traditional-Medicine Education: Defense-in-Depth Architecture, Domain-Calibrated Safety Evaluation, and a Proposed Minimum Safety Stack

Technical Summary: Safe Deployment of a Generative AI Assistant for Traditional-Medicine Education

Problem Statement

The deployment of generative AI in traditional-medicine education (TME) faces a unique safety challenge: standard model alignment techniques, such as Reinforcement Learning from Human Feedback (RLHF), are insufficient for this domain. TME involves hundreds of herbs with narrow therapeutic margins and complex, often poorly documented interactions. A system that answers fluently but incorrectly can actively harm by reinforcing clinical errors.

The core problem identified is that the definition of "safety" in AI is not domain-neutral. Standard IT-security criteria (focusing on jailbreaks, policy bypasses, and data exfiltration) fail to capture domain-specific clinical harms. A model may pass all IT-security checks while still generating dangerous prescribing recommendations or unsafe herb combinations because it lacks specific clinical knowledge. Conversely, a response that triggers an IT-security alert (e.g., role override) might be clinically benign. Current evaluation frameworks do not adequately address this divergence, risking the deployment of systems that are "secure" but clinically unsafe.

Methodology

The authors present TLAS-YHCT, a Socratic chatbot designed for undergraduate traditional-medicine students at Hoa Binh University, and a rigorous adversarial evaluation framework.

1. System Architecture: Defense-in-Depth
TLAS-YHCT employs a five-layer pipeline where every query passes through multiple independent controls before a response is returned:

• Non-overridable Hard Guardrails: Rule-based filters and deny-lists that enforce fixed boundaries (e.g., refusing out-of-scope requests) independent of conversational framing.
• Retrieval-Augmented Generation (RAG): Responses are anchored to a vetted, version-controlled corpus of traditional-medicine texts. The system uses hybrid retrieval (dense semantic + sparse keyword) and refuses to answer if no supporting evidence is found, preventing fabrication.
• Standardised System Prompt: A structured, version-controlled prompt defining the pedagogical role, permitted outputs, and mandatory advisories.
• Independent LLM-as-Judge Verifier: A second language model cross-checks the core model's output against the retrieved passages, compliance corpus, and pedagogical objectives. It regenerates or blocks responses that fail these checks.
• Human-in-the-Loop Clinical Audit: A continuous cycle where clinical faculty review logs, diagnose failures, and update the system under version control.

2. Adversarial Evaluation Design

• Attack Construction: A structured red-team set of 206 adversarial attacks was constructed entirely by five domain-expert clinician-educators who had no involvement in the system's technical development. This ensured attacks were calibrated to specific clinical harms (e.g., dangerous herb combinations) rather than generic IT threats.
• Dual Safety Criterion: Every response was scored blind by the same expert panel using two independent criteria:
  • IT-Security Criterion: Focuses on role override, policy bypass, and data privacy.
  • Clinical-Education Criterion: Focuses on patient safety, pedagogical integrity, and dangerous prescribing guidance, regardless of whether a security policy was bypassed.
• Comparative Baselines: TLAS-YHCT was compared against GPT (free tier), Gemini Flash, and a bare, unwrapped DeepSeek R1 core.

Key Results

1. Superior Safety Performance
Under the dual safety criterion, TLAS-YHCT achieved a 100% safe-response rate (1,030/1,030 ratings) across all 206 attacks. In contrast, commercial baselines performed significantly worse:

• GPT: 88.9% safe response rate.
• Gemini Flash: 78.7% safe response rate.
• DeepSeek R1 (Bare): 76.7% safe response rate.

2. Divergence of Safety Criteria
The study empirically demonstrated a critical divergence between IT-security and clinical-education safety judgments:

• IT-Safe but Clinically Unsafe: A significant subset of attacks (specifically dangerous herb-combination queries and false-premise traps) were judged "safe" by IT-security standards (no policy bypass occurred) but "unsafe" by clinical standards because they produced dangerous medical advice. Standard IT evaluations would miss these entirely.
• IT-Unsafe but Clinically Safe: Some authority-spoofing attacks succeeded in bypassing security protocols (IT-unsafe) but resulted in clinically appropriate responses. Treating these as failures would inflate perceived risk.

3. Architectural Impact vs. Base Model
When the bare DeepSeek R1 core was wrapped in the TLAS-YHCT defensive stack, its performance improved from 76.7% to 100% safe responses, outperforming commercial models that lacked this specific architecture. This suggests that the layered defense (RAG + Verifier + Guardrails) compensates more effectively for domain-specific failures than base-model alignment alone.

Significance and Claims

The paper claims four primary contributions:

1. System Design: It describes TLAS-YHCT, a deployed system utilizing a defense-in-depth architecture specifically for traditional-medicine education.
2. Evaluation Methodology: It introduces a structured red-team approach where attack construction and scoring are performed by domain-expert clinicians independent of developers, ensuring the test set reflects real-world clinical risks.
3. Empirical Evidence of Divergence: It demonstrates that IT-security and clinical-education safety criteria are not interchangeable. Relying solely on IT-security standards in clinical education misses real harms and miscounts benign events.
4. Proposed Framework: It proposes a Minimum Safety Stack for TME and allied clinical education, consisting of:
   • Retrieval-Augmented Generation (RAG) over a vetted corpus.
   • A standardised system prompt.
   • LLM-as-judge verification trained on domain-specific rubrics.
   • Domain-calibrated safety criteria (dual IT + clinical).

The authors conclude that safety in specialized clinical education cannot be delegated to base models alone; it requires a transferable framework combining technical controls with domain-calibrated evaluation. They also propose a methodological standard of three-way role separation (developer, attack author, investigator) for future adversarial safety assessments in medical education.

Note: The paper explicitly states that the 100% safety rate applies to the specific 206-attack set and does not claim unconditional safety. Furthermore, the study evaluates safety, not pedagogical efficacy, which remains to be demonstrated in prospective educational studies.