Beyond Injection Detection: A Positive-Security Prompt Firewall that Closes the Scope and PHI Gap SOTA Classifiers Miss in Healthcare

Technical Summary: Beyond Injection Detection: A Positive-Security Prompt Firewall that Closes the Scope and PHI Gap

Problem Statement
The integration of Large Language Models (LLMs) into autonomous clinical agents creates a high-risk attack surface where trusted instructions and untrusted data share a single context window. While state-of-the-art (SOTA) prompt injection classifiers (e.g., Meta's PromptGuard-2) effectively detect overt adversarial tokens (jailbreaks, instruction overrides), they fail to detect "fluent, legitimate-looking" requests that carry no injection signal. In healthcare, the most critical threats—Protected Health Information (PHI) exfiltration, cross-patient access, bulk data export, and out-of-scope clinical advice—are often phrased as polite, authorized workflows. A 2025 JAMA Network Open study noted that commercial medical LLMs followed injected instructions in 94.4% of simulated encounters, but the specific gap identified here is that generic classifiers miss these scope-violating requests because they lack adversarial syntax. Furthermore, existing runtime guardrails face a trade-off: model-based auditors are accurate but introduce high latency (hundreds of milliseconds), while lexical filters are fast but blind to obfuscated payloads.

Methodology: QFIRE
The authors present QFIRE, an inline, provider-agnostic prompt firewall implemented as a single self-contained Rust toolchain (proxy, CLI, and benchmark harness). QFIRE operates as a zero-trust, positive-security system that enforces a declared natural-language scope and a HIPAA Safe-Harbor PHI panel.

• Architecture: QFIRE functions as a reverse proxy normalizing requests from OpenAI, Anthropic, Gemini, and Ollama into an internal representation. It evaluates prompts against a declarative, version-controlled YAML policy before forwarding them to the downstream model.
• Core Mechanisms:
  1. Positive-Security Scope Constraints: Instead of merely detecting attacks, QFIRE restricts model calls to a declared purpose (enforcing the HIPAA "minimum necessary" standard). It blocks "out-of-scope drift" even when no attack token is present.
  2. Asynchronous Detector Graph: Rules are organized into pipelines running concurrently on a Tokio runtime. The system employs a "cheap-before-expensive" short-circuit strategy: deterministic lexical checks (regex, Aho-Corasick, entropy) run first; if they abstain, more expensive nodes (local DeBERTa-v3 classifier via ONNX Runtime, or an LLM scope-judge) are triggered.
  3. De-obfuscation Normalization: Before detection, an optional pass strips zero-width characters, folds homoglyphs and leetspeak, and decodes Base64, hex, and ROT13 to expose hidden payloads.
• Policy Design: Unlike baselines that encode policy in Python code or model weights, QFIRE uses a declarative language. Rules are data (YAML) that can be diffed, unit-tested (qfire rules test), and audited without modifying the engine.
• Benchmarks: The authors introduce QFIRE-HealthBench, a new dataset of 2,000 prompts (1,000 benign clinical-adjacent, 1,000 malicious) constructed using real garak and Microsoft PyRIT payloads. This dataset specifically targets threats that generic classifiers miss.

Key Results

• Generic Injection Performance: On public prompt-injection datasets (1,968 prompts), QFIRE's deterministic hybrid achieves an F1 score of 0.86, statistically tied with Meta's PromptGuard-2 (0.86) and superior to the standalone DeBERTa-v3 classifier (0.83). Lexical baselines lag significantly (F1 0.16–0.50).
• Healthcare Coverage Gap (Central Result): On QFIRE-HealthBench, the performance gap widens dramatically. PromptGuard-2's recall collapses to 0.40, and DeBERTa-v3 reaches only 0.57, because most clinical threats lack injection signals. QFIRE's combined scope+PHI chain achieves 0.83 recall (F1 0.87) at a calibrated false-positive rate (FPR) of 0.08.
• Adaptive Robustness: Under adaptive attacks (paraphrasing, encoding, and in-the-loop rephrasing), SOTA classifiers and full-stack frameworks (e.g., NeMo Guardrails) drop to 0–71% recall. QFIRE's positive-security scope+PHI chain maintains 100% recall against these adaptive families, as the structural goal (e.g., "export all records") remains out-of-scope regardless of phrasing.
• End-to-End Harm Reduction: In a mock-EHR sandbox with a tool-using agent, QFIRE reduced the harmful-action rate from 0.38 to 0.00. Benign utility dropped only slightly (0.95 to 0.825), with the cost concentrated in conservatively blocked outbound email requests.
• Latency and Deployment: QFIRE runs locally with bounded latency. The deterministic hybrid path (short-circuiting) achieves a p95 latency of ~242 ms. The system avoids Python in the hot path, running entirely in Rust with embedded ONNX.

Significance and Claims
The paper argues that generic prompt injection detection is necessary but not sufficient for healthcare agents. The primary contribution is demonstrating that the "scope and PHI gap" cannot be closed by stronger classifiers alone; it requires a structural, positive-security approach that enforces declared purposes and identifier protection.

• Regulatory Alignment: QFIRE operationalizes the Healthcare AI Agents Regulatory Framework (HAARF) by providing measurable evidence for scope restriction, PHI protection, and auditable refusal. It maps directly to HIPAA's minimum-necessary standard (45 CFR 164.502(b)) and proposed Security Rule requirements for continuous monitoring and immutable audit logs.
• Auditability: By decoupling policy from model weights, QFIRE offers a "fail-closed" mechanism where every decision is logged with an explainable trace, allowing compliance officers to review and diff policies without trusting opaque model internals.
• Limitations: The authors acknowledge that QFIRE is text-prompt focused (not multi-modal) and that its LLM judge component inherits the biases and calibration needs of the backing model. They note that while the system is robust against adaptive attacks, a "firewall-in-the-loop" adversary can eventually erode recall if the operating point is not tuned for robustness over utility.

In summary, QFIRE provides a low-latency, deterministic, and auditable enforcement layer that closes the critical safety gap in clinical AI agents where traditional injection detection fails.