Technical Summary: guardrail-rs

Problem Statement

Large Language Model (LLM) applications currently lack a dedicated enforcement layer between the user/application and the model provider. This architectural gap exposes systems to two primary risks identified by the OWASP GenAI Security Project:

1. Prompt Injection: Adversarial instructions embedded in user input or retrieved context (indirect injection) can override intended model behavior.
2. Sensitive Information Disclosure: Personally Identifiable Information (PII) is often transmitted unfiltered to third-party model providers, violating organizational data policies.

Existing mitigations, such as model alignment training or in-process libraries (e.g., NeMo Guardrails, LLM Guard), are insufficient because they either fail against optimization-based adversarial suffixes or couple the security layer's availability and update cadence to the application's runtime.

Methodology and System Design

The paper presents guardrail-rs, an open-source, reverse proxy implemented in Rust that operates at the network boundary. It intercepts all requests and responses between an LLM client and the provider without requiring changes to the application code.

Core Architecture

The system is built around a composable Stage/Pipeline abstraction:

• Stages: Individual enforcement units (e.g., scanners, redactors) implementing a Stage trait with an asynchronous evaluate method. They return a Decision of Allow, Redact, or Block.
• Pipeline: An ordered sequence of stages that short-circuits upon a Block decision.
• Fail-Open Design: A central design principle where individual stages default to Allow on error (e.g., model file corruption, unexpected input) rather than Block. This prevents the security layer itself from becoming an availability liability. This behavior is configurable per stage.

Enforcement Mechanisms

1. Prompt Injection Defense:
   • Regex-based Scanner: The default, always-available path using Rust's RegexSet for single-pass pattern matching.
   • Semantic Classifiers: An optional, feature-gated path using ONNX Runtime for semantic classification (injection and toxicity), though these are not bundled with pre-trained weights in the current release.
2. PII Redaction: A regex-based stage that detects and redacts emails, phone numbers, credit cards (with Luhn validation), and government IDs, replacing them with typed tokens (e.g., [EMAIL]).
3. Policy Rules: A configurable engine for organization-specific keyword or regex conditions.

Observability and Privacy

The system employs a privacy-preserving observability design:

• Audit Logs: Structured NDJSON logs record decision metadata (which stage acted, decision type, redacted entity types) but explicitly exclude raw request/response content and credentials.
• Metrics & Tracing: Prometheus metrics and OpenTelemetry distributed tracing track latency and request counts.

Implementation

The codebase is a five-crate Rust workspace:

1. core: Request model, Stage trait, and Pipeline.
2. classifiers: Regex and optional ONNX stages.
3. configuration: Parsing, validation, and hot-reload logic.
4. proxy: HTTP server and upstream client (using hyper and tokio).
5. binary: Command-line interface.

Empirical Results

Performance was measured using Criterion.rs on shared GitHub Actions CI infrastructure (not dedicated hardware) to ensure transparency regarding variance.

Latency Benchmarks

• Regex Injection Scanner: Highly performant, meeting internal design targets. At 8.2 KB input, median latency was 28.5 µs (target: <50 µs).
• PII Redactor: Performance degrades with input size and PII density.
  • At 742 B: 13.1 µs (within target of <20 µs/4KB).
  • At 3.0 KB: 50.8 µs (254% of target).
  • At 6.0 KB: 101 µs (505% of target).
  • Note: The authors explicitly report this failure to meet the documented target for larger payloads rather than revising the target post-hoc.
• Full Pipeline: Even in the worst-case scenario (8.2 KB input), the full pipeline latency was 158 µs, consuming only 15.8% of the 1 ms design budget and 3% of the 5 ms CI enforcement ceiling.

Engineering Pitfalls and Lessons

The paper details four specific engineering challenges encountered during hardening, serving as lessons for similar infrastructure:

1. Configuration Drift: Two physically separate copies of default rule sets diverged (8 patterns vs. 28), causing default deployments to be less secure than tests implied. Lesson: Documentation comments are insufficient; automated invariant checks are required.
2. Dependency Schema Churn: Unpinned third-party tooling (cargo-deny) silently changed configuration schemas, breaking CI without code changes. Lesson: CI passing is a point-in-time statement; tooling dependencies must be pinned or monitored.
3. Async Concurrency Deadlock: A fix for a panic (adding #[tokio::test]) introduced a deadlock because the default runtime is single-threaded, preventing background tasks from polling while the main thread blocked. Lesson: Fixes that remove panics may introduce silent deadlocks; multi-threaded runtimes are often necessary for async tests.
4. Silent Release Blocking: A pinned Docker base image version prevented the use of newer Cargo features, silently blocking all releases. Lesson: Pinned versions are implicit promises to revisit; failure modes often surface in dependent processes, not the pinned artifact itself.

Significance and Claims

The authors explicitly state that guardrail-rs does not claim novel detection algorithms or formal adversarial robustness. Its contributions are:

1. Architectural: A network-boundary enforcement model with explicit fail-open semantics and privacy-preserving observability.
2. Empirical: An honest, continuously tracked performance characterization that reports failures to meet design targets (specifically for PII redaction at scale) rather than hiding them.
3. Practical: A set of generalized engineering lessons derived from real-world defects in hardening security infrastructure.

The paper concludes that the discipline of publishing inconvenient benchmark results, converting violated comments into executable tests, and documenting incorrect initial fixes is as significant a contribution as the system itself. It identifies the lack of adversarial-robustness evaluation against established attack suites (e.g., garak) as the most critical area for future work.