Technical Summary: Now You (Still) See Me

Problem Statement

The paper addresses the security risk posed by steganographic trojans in Large Language Models (LLMs). An attacker can fine-tune an open-source model to encode prompt-borne secrets (e.g., 32-bit strings) into fluent, seemingly benign text outputs. This creates an exfiltration channel that appears innocuous to traditional output-level steganalysis, which relies on detecting distributional shifts in the generated text. Since benign fine-tuning also induces such shifts, distinguishing malicious encoding from standard adaptation is difficult.

Recent work (Westphal et al., 2026) proposed mechanistic detection using linear probes (ridge regression) trained to recover the secret's embedding-space direction from the model's internal activations. The paper investigates whether this activation-based defense is robust against adaptive adversaries who can fine-tune models specifically to evade these probes.

Methodology

The research is structured into three chapters, evaluating five open-weight models: Qwen3-8B, Llama-3.1-8B, Ministral-8B, Qwen3-14B, and Phi-4-14B.

Chapter I: Baseline Trojan Construction and Detection

• Encoding Scheme: The authors implement a bucket-based secret encoding. The model's vocabulary is partitioned into two buckets (0 and 1) via a hyperplane in embedding space. A 32-bit secret is encoded by forcing the model to generate tokens belonging to the specific bucket corresponding to each bit.
• Training: Models are fine-tuned using a mixed-motive Supervised Fine-Tuning (SFT) approach. The training data consists of steganographic examples (masked logits to enforce bucket constraints), mathematical reasoning, and instruction-following data.
• Detection: Two probe families are used to detect the secret in internal activations:
  1. Ridge Regression: A linear probe trained to reconstruct the secret embedding direction.
  2. MLP Probe: A two-layer Multi-Layer Perceptron trained on the same task.
• Metric: Detectability is measured by the mean absolute cosine similarity ($q_\ell$) between the probe's prediction and the true secret embedding across layers. A model is flagged if $q_\ell$ exceeds a baseline derived from the unmodified base model on all tracked layers.

Chapter II: Adversarial Evasion

• Threat Model: The adversary trains a model to evade detection without real-time access to the deployed detector's activations. The goal is to make the trojan's hidden states indistinguishable from the base model under any linear probe.
• Adversarial Objective: The authors introduce a min-max style training loop. The loss function ($L_{evade}$) includes a term that maximizes the detection signal for a ridge regressor that is periodically refitted on detached training batches. This prevents the model from overfitting to a single fixed probe.
• Preservation: To ensure the model remains functional, the loss includes:
  • $L_{stego}$: A penalty for deviating from the bucket-constrained teacher (ensuring secret recovery).
  • $KL$ penalty: Anchors the adapter to the non-evasive Chapter I model to preserve fluency and capabilities.
• Evaluation: The resulting "evasive trojans" are tested against both the ridge regressor used during training and the held-out MLP probe.

Chapter III: Recovering Detectability via Information Theory

• Theoretical Characterization: The authors analyze the evasion mechanism using information theory. They decompose internal representations into content-aligned components ($R_{\neq N}$) and "slack" degrees of freedom ($R_N$) arising from lexical choice and rephrasing.
• Synergy Hypothesis: They prove (Theorem 1) that for a model to successfully evade detection (maintain fluency and indistinguishability) while recovering the secret, the secret must be routed through a synergistic interaction between the content-aligned representation and the slack degrees of freedom. The secret is hidden in the joint structure, making it invisible to low-order (linear) detectors that cannot access the slack component.
• Intervention: To break this synergy, the authors propose recontextualization. They construct a dataset where a single sentence template is fixed, and only topic slots vary. This drastically reduces the entropy of the slack degrees of freedom ($H(R_N | R_{\neq N}, C)$).
• Prediction: Theorem 2 predicts that constraining the slack will force the secret information back into the content-aligned representation, making it detectable again.
• Experiment: They evaluate the evasive trojans on a sweep of datasets ranging from diverse prompts to highly constrained recontextualization prompts, measuring the recovery of detection signals.

Key Results

1. Baseline Detection (Chapter I)

• Steganographic Success: The initial trojans achieved high bit accuracy (96.8–97.8%) and exact-match recovery (37.5–61.5%) for 32-bit secrets.
• Capability Preservation: The models retained most of their original capabilities, with average benchmark drops between -0.8% and -6.4%.
• Detectability: All Chapter I models were successfully flagged by at least one of the two probe families (Ridge or MLP), confirming that standard fine-tuning leaves linear traces.

2. Evasion Success (Chapter II)

• Robust Evasion: The adversarial fine-tuning successfully suppressed the detection signal.
  • Ridge Probe: All Chapter II models evaded the ridge detector (the one used during adversarial training).
  • MLP Probe: Crucially, the models also evaded the held-out MLP probe, despite the MLP not being part of the adversarial training loop. This suggests the evasion learned a general reduction in linear extractability rather than simple overfitting.
• Performance Retention: The evasive models maintained high secret recovery (58–79% exact match) and incurred only a marginal additional capability drop (average -3.9% across benchmarks).

3. Recovery of Detectability (Chapter III)

• Slack Constraint: When evaluated on the recontextualization dataset (where $N_{recon}$, the number of constrained prompts, increased), the detection signal for the evasive trojans rose significantly.
• Restoration: As the slack degrees of freedom were restricted, the evasive trojans became distinguishable from the base model again for both Ridge and MLP probes. The detection curves for the trojans separated cleanly from the base model baseline, effectively "undoing" the evasion without destroying the covert channel.

Significance and Claims

The paper claims three primary contributions:

1. Vulnerability of Activation-Based Detection: It demonstrates that linear and even non-linear (MLP) activation probes are vulnerable to adaptive evasion. An adversary can fine-tune a model to hide secrets in synergistic interactions with residual degrees of freedom, rendering standard mechanistic monitoring ineffective.
2. Information-Theoretic Explanation: The work provides a theoretical framework explaining why evasion works. It posits that successful evasion requires the secret to be encoded synergistically with "slack" (rephrasing/lexical choice) to satisfy both recoverability and conditional indistinguishability.
3. Theory-Guided Defense: The paper argues that detector failures can be diagnostically useful. By understanding the evasion mechanism (reliance on slack), defenders can construct targeted evaluation distributions (recontextualization) that restrict these specific degrees of freedom. This recovers detectability without requiring new, more complex detection architectures.

The authors conclude that while mechanistic probes are powerful, they should not be viewed as complete defenses against adaptive models. Instead, safety evaluation should incorporate theory-driven datasets that constrain the specific degrees of freedom an adversary might exploit.