Technical Summary: SurrogateShield

Problem Statement

Large Language Model (LLM) assistants transmit user queries verbatim to third-party API endpoints, exposing Personally Identifiable Information (PII) to risks of data breaches, subpoenas, or policy changes. The prevailing mitigation, placeholder redaction (e.g., replacing "John Doe" with [PERSON]), preserves privacy by construction but destroys semantic coherence. This structural degradation forces the LLM to process "structurally degraded" queries, resulting in correspondingly degraded responses and measurable utility loss in downstream tasks. The core challenge is to prevent real PII from crossing the network boundary without sacrificing the semantic utility required for high-quality LLM interactions.

Methodology: SurrogateShield

SurrogateShield is a client-side proxy that intercepts LLM queries, replaces detected PII with locally generated, type-consistent surrogate values, and transparently restores the original values in the response before display. The system operates on three design principles:

1. No PII crosses the API boundary: Enforcement occurs at the local transport layer before the HTTP request is constructed.
2. Surrogates preserve utility: Fake values (e.g., names, SSNs) are realistic, type-consistent, and statistically unlinked to the original, preserving sentence structure and grammatical agreement.
3. Transparent restoration: Original values are restored in the LLM response via a three-pass ResolvePass before the user sees the output.

System Architecture

The pipeline consists of four primary components:

1. SentinelLayer (Three-Stage Detection Cascade)
Detection runs through a sequential cascade where each stage masks claimed character spans to prevent double-processing:

• PatternScan: Applies priority-ordered regular expressions with checksum validators (Luhn for credit cards, ABA for routing numbers) to detect 16 structured PII types (e.g., SSN, email, crypto addresses).
• EntityTrace: Utilizes spaCy (en_core_web_lg) for Named Entity Recognition (NER) on the remaining text. It classifies entities into "confirmed" (score ≥0.85) and "borderline" (0.60–0.85). It includes reclassification logic (e.g., promoting organizations to geopolitical entities if location prepositions are present) and blocklists to suppress common false positives.
• ContextGuard: Executes a locally hosted DistilBERT NER model (dslim/distilbert-NER) to verify borderline entities and detect missed entities. This stage runs entirely offline after an initial model download.

2. Service-Query Intelligence & Quasi-Identifier Risk

• ServiceQueryDetector: Distinguishes between queries where location is the topic (e.g., "restaurants near X") versus queries where location identifies the user. For service queries, it applies proportional anonymization (e.g., shifting house numbers by ±2–8) rather than full replacement to maintain utility.
• Quasi-Identifier Risk Detector: Based on Sweeney's k-anonymity framework, it detects combinations of fields (e.g., ZIP + DOB + gender) that could statistically re-identify a user even if individual fields are not traditional PII.

3. MimicGen (Surrogate Generation)
Generates type-consistent surrogates using the faker library and custom logic. It ensures collision resistance within a session and validates formats (e.g., valid Luhn checksums for credit cards, Base58 encoding for Bitcoin addresses).

4. ShadowMap & ResolvePass

• ShadowMap: A per-conversation, in-memory dictionary mapping surrogates to originals, persisted as an AES-256-GCM encrypted binary file. Keys are derived via HKDF-SHA256 from a device-level secret.
• Dual History: The system maintains two message histories: api_messages (surrogates only, sent to the API) and messages (real values restored, shown to the user). This prevents PII accumulation in multi-turn contexts.
• ResolvePass: Restores originals in the LLM response using three passes: exact matching (longest first), component matching (for partial name usage), and fuzzy matching (using rapidfuzz).

Key Contributions

1. SurrogateShield System: The first end-to-end client-side proxy implementing type-consistent surrogate replacement with cryptographically secured local mapping and transparent response reconstruction.
2. Detection Cascade: A novel three-stage pipeline combining regex, spaCy NER, and local DistilBERT NER, augmented by a post-processing suite for deduplication and topical filtering.
3. Empirical Utility Validation: Demonstration that surrogate substitution significantly outperforms placeholder redaction in semantic utility, measured via BERTScore.
4. Adversarial Robustness: Empirical evidence that surrogate-substituted messages resist LLM-based adversarial recovery, whereas placeholder tokens expose slot locations to inference.
5. Ablation Analysis: Quantification of the marginal F1 contribution of each detection stage, identifying the specific role of PatternScan (structured PII), EntityTrace (named entities), and ContextGuard (borderline locations).

Experimental Results

Evaluated on a corpus of 1,124 annotated queries (synthetic but structurally realistic):

• Detection Quality: The full cascade achieved an overall F1 score of 98.87%. SurrogateShield matched or exceeded Microsoft Presidio on all 11 comparable entity types, with significant gains on structured identifiers like US Driver's Licenses (+50.00 pp F1) and Crypto addresses (+45.45 pp F1).
• Semantic Utility: SurrogateShield preserved 94.85% of semantic utility (BERTScore F1), compared to 81.59% for placeholder redaction (a 13.26 percentage point improvement, $p < 0.001$).
• Privacy Guarantees:
  • PII-Leak Rate: 0.00% (no real PII reached the API in the 1,124-query corpus).
  • Resolve-Leak Rate: 0.00% (no surrogates remained unrestored in the final display).
  • Adversarial Recovery: In a 100-query trial against a prompted LLM adversary (Claude Haiku), the recovery rate for surrogate-substituted messages was 0.00%, compared to 1.53% for placeholder-redacted messages.
• Performance: The total local processing overhead was 25.89 ms per query (dominated by spaCy NER at 24.60 ms), which is negligible compared to typical LLM API latencies (600–2,000 ms).

Significance and Claims

The paper argues that the trade-off between privacy and utility in LLM interactions is not fundamental. SurrogateShield demonstrates that privacy requires the absence of real PII values, not the absence of PII-shaped structure.

By replacing real values with type-consistent surrogates, the system satisfies the containment constraint (no real PII leaves the device) while preserving the structural and semantic integrity required for high-quality LLM responses. The authors claim this is the first system to empirically validate that this design space is viable, achieving higher utility and stronger adversarial resistance than the prevailing placeholder redaction approach, all while operating entirely locally with minimal latency overhead. The work challenges the assumption that semantic degradation is a necessary cost of privacy-preserving LLM interactions.