Technical Summary: Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers

Problem Statement

Safety classifiers serve as a critical defense layer between large language models (LLMs) and users, yet they operate under a stationarity assumption: that future input distributions will resemble the calibration distribution. This assumption fails due to adversarial adaptation, organic linguistic drift, multilingual code-switching, and compositional attacks. The primary failure mode is "silent": a classifier's accuracy may degrade significantly (e.g., from 95% to 80%) without generating an error signal if ground-truth labels are unavailable in real-time. By the time offline evaluation detects the issue, the system may have been making unreliable decisions for days or weeks.

The paper addresses two core challenges:

1. Detection: How to detect distributional shifts online with controlled false alarm rates across diverse shift types and classifier architectures.
2. Adaptation: How to recover coverage guarantees after shift detection without requiring new labeled data, specifically using weighted conformal prediction.

Methodology

System Architecture

The proposed system is an online monitor observing a stream of classifier outputs: input text ($x_t$), unsafe-class probability ($s_t$), and penultimate-layer representations ($r_t$). It comprises two main components:

1. Shift Detector: A sliding-window detector that triggers an alarm when the input distribution shifts.
2. Conformal Abstention Layer: A downstream layer that adapts decision thresholds upon alarm to preserve a target error rate ($\epsilon = 0.1$).

Detection Mechanisms

The system evaluates two detection channels:

• Kolmogorov–Smirnov (KS) Detector: Tracks the marginal distribution of classifier scores. It maintains a sliding window of recent scores and computes the one-sample KS statistic against a frozen reference Cumulative Distribution Function (CDF) established during calibration.
• Maximum Mean Discrepancy (MMD) Detector: Tracks the geometry of classifier embeddings using a sliding window of embedding vectors and a Gaussian kernel.

Calibration and Alarm Logic:
To control false alarm rates (FAR) over a monitoring horizon, the system employs empirical calibration via null simulation. Instead of relying solely on theoretical bounds (like Hoeffding), which may be conservative for sliding windows, the system runs 50 negative control streams (in-distribution data only) to determine the 97th percentile of the maximum observed statistic as the alarm threshold.

Conformal Adaptation

Upon detecting a shift, the system activates a weighted conformal prediction layer.

• Density Ratio Estimation: It estimates the density ratio $w_i = p_{target}(x_i) / p_{source}(x_i)$ between the source (calibration) and target (shifted) distributions using logistic regression on classifier embeddings.
• Weighted Quantile: The conformal threshold is recomputed as a weighted quantile of nonconformity scores.
• Stability: Weights are clipped to $[1/C, C]$ (with $C=10$) to ensure stability.

Experimental Design

The authors conducted a pre-registered factorial evaluation:

• Classifiers (4): DeBERTa-v3-large, Text-Moderation (DeBERTa-v3-base), Llama Guard 3, and ShieldGemma.
• Shift Conditions (5): Paraphrase, Code-switch, Compositional, Temporal (real jailbreaks), and Adversarial suffix (GCG-optimized).
• Parameters: 20 random seeds, 2 window sizes (100, 200), totaling 800 experimental cells.

Key Results

1. Detection Performance (RQ1)

• Overall Efficacy: The system achieved an 86.6% valid detection rate (693/800 cells) with a mean detection latency of 39.5 steps (at window size 100). Empirical false alarm rates ranged from 2% to 10% across classifiers.
• Crossover Interaction: A significant interaction effect was observed between classifier architecture and shift type:
  • Encoders (DeBERTa, Text-Moderation): Detected naturalistic shifts (paraphrase, compositional) quickly (24–35 steps) but struggled with adversarial suffixes (up to 36.6 steps).
  • Decoders (Llama Guard, ShieldGemma): Detected adversarial suffixes quickly (26–28 steps) but were slow on paraphrase (64–97 steps).
• Ground-Truth Regimes: Detection held across three regimes: synthetic onset, real temporal jailbreaks (85% detection on 20 cells), and adversarial success scenarios.
  • Adversarial Success Case: In a specific regime where GCG suffixes successfully flipped DeBERTa's classification, the shift was undetectable by DeBERTa (scores moved to the safe region, indistinguishable from reference) but detectable by non-target classifiers (Llama Guard) where the same inputs appeared more dangerous. This suggests cross-classifier anomaly detection can serve as a "canary" for adversarial evasion.

2. Conformal Adaptation (RQ2)

• Density-Ratio Collapse: A critical failure mode was identified. For three of the four classifiers (Text-Moderation, Llama Guard, ShieldGemma), the logistic regression density ratio estimator achieved perfect linear separability in the high-dimensional embedding space. This forced all importance weights to the clipping floor ($1/C = 0.1$), resulting in an Effective Sample Size (ESS) of ~300 (no effective reweighting) and negligible coverage recovery ($\le +0.10$).
• DeBERTa Exception: DeBERTa was the only classifier where weighted conformal prediction achieved meaningful recovery, but effectiveness varied by shift type:
  • Paraphrase: ESS=46, Recovery=+39 percentage points (pp).
  • Temporal: ESS=88, Recovery=+16 pp.
  • Adversarial Suffix: ESS=206, Recovery=+2 pp (near-collapse).
• Dimensionality Reduction Remedy: Applying PCA to reduce embeddings to 32 dimensions before density ratio estimation broke the perfect separability. This restored coverage for generative models: Llama Guard recovered +33 pp and ShieldGemma +21 pp on temporal shifts.

3. Variance Decomposition (RQ3)

A two-way ANOVA on detection latency revealed that no single factor dominates:

• Classifier: $\eta^2 = 0.243$
• Shift Type: $\eta^2 = 0.237$
• Interaction (Classifier $\times$ Shift): $\eta^2 = 0.185$
• Residual: $\eta^2 = 0.335$
  All factors were statistically significant ($p < 0.001$), indicating that detection difficulty is determined by the specific pairing of classifier and shift type, necessitating per-classifier monitoring profiles.

Significance and Claims

The paper claims to provide a targeted empirical study of LLM safety classifiers under diverse shift conditions, surfacing failure modes that general monitoring frameworks miss. Key contributions include:

1. Diagnosis of Density-Ratio Collapse: The paper identifies a specific failure mode in generative safety-classifier embeddings where logistic regression achieves perfect separability, rendering data-driven reweighting ineffective. It demonstrates that dimensionality reduction (PCA) is a necessary diagnostic and remedy for this artifact.
2. Architecture $\times$ Shift Interaction: The study reveals that detection latency is not a property of the classifier or shift alone but of their interaction. Encoders and decoders exhibit opposite sensitivities to naturalistic vs. adversarial shifts, challenging the notion of a universal monitoring configuration.
3. Cross-Classifier Anomaly Detection: The work demonstrates that successful adversarial attacks against a target classifier may produce insufficient distributional signal for that specific classifier but appear anomalous to architecturally different classifiers, enabling detection via cross-classifier score shifts.
4. Operational Trade-offs: The comparison between KS (sliding window) and Confidence Sequence (growing window) detectors highlights a deployment trade-off: KS offers faster detection for strong signals, while CS provides superior detection for weak, low-mixing shifts (e.g., 97% detection vs. 43% for KS at 30% contamination).

The authors conclude that neither classifier choice nor shift type alone determines detection difficulty. Effective monitoring requires per-classifier profiles that account for specific shift interactions, and conformal adaptation strategies must account for dimensionality-induced separability artifacts in high-dimensional embeddings.