Testing Whether Internal Backdoor Precursors Precede Behavior: A Causal, Power-Matched Protocol

Technical Summary: Testing Whether Internal Backdoor Precursors Precede Behavior

Problem Statement
Current safety monitoring for AI models typically evaluates whether a completed model contains a backdoor or analyzes training dynamics in isolation. A critical temporal gap remains: it is unknown whether an internal representation associated with a backdoor becomes detectable before the model's output behavior exhibits a statistically significant change. Existing methods fail to establish a causal, temporal comparison between internal signals and output behavior because they often lack matched null conditions, use arbitrary classification thresholds, or fail to control for the family-wise error rate across dense checkpoint inspections. Furthermore, a decodable internal signal does not necessarily imply a causal precursor; it may merely encode the trigger's presence without driving the target behavior.

Methodology
The author proposes a rigorous, falsifiable protocol to test if an internal backdoor precursor precedes behavioral manifestation. The experiment utilizes a 124M-parameter GPT-2 model fine-tuned on the SST-2 sentiment classification task.

1. Experimental Design:

   • Training Protocol: The model undergoes a "clean adaptation" phase, followed by a continuation phase where only transformer blocks 10 and 11 (and the final layer norm) are updated.
   • Conditions: Two conditions are run in parallel:
     • Backdoor Condition: A specific token marker (cf) is associated with a target behavior (mapping negative reviews to positive sentiment).
     • Matched-Null Condition: Identical inputs, marker exposure, label totals, and label noise are used, but the active marker is not associated with the target behavior (the mislabeling occurs on unmarked examples).
   • Dense Checkpoints: Models are evaluated at every update step (0 to 80).

2. Dual Detectors:

   • Behavioral Assay: Measures the continuous change in the output-logit margin (positive minus negative) for active-marker inputs versus control-marker inputs, relative to the pre-continuation baseline. This avoids arbitrary thresholding.
   • Internal Assay: Measures the shift in the residual stream (post-block-10) projected onto a locked, one-dimensional direction. This direction is derived from an independent "discovery" continuation trained on a separate dataset to capture the mature backdoor mechanism. Crucially, the direction is fixed before confirmation runs begin to prevent overfitting.

3. Causal Interventions:
   To verify that an early internal signal is a precursor and not just a decodable passenger, the protocol employs causal interventions at the claimed detection time:

   • Ablation: Removing the acquired coordinate from the active-marker activation.
   • Restoration: Adding a checkpoint-matched amplitude from independent donor examples to the ablated activation.
   • Control Patching: Adding the amplitude to control-marker inputs.
     These interventions are tested on a disjoint set of prompts to ensure temporal separation.

4. Statistical Calibration:

   • Error Control: To address the "multiple comparisons problem" inherent in inspecting 160 detector-checkpoints (2 detectors × 80 updates), the protocol uses a maximum-statistic cutoff derived from 20,000 bootstrap draws of null trajectories. This ensures a strict family-wise false-positive rate.
   • Power Matching: Both detectors use the same sample size ($n=128$), the same studentized mean statistic, and the same cutoff, ensuring comparable statistical power.
   • Decision Gates: A positive result requires: (1) the internal test crossing the threshold before the behavioral test in a significant number of runs; (2) successful suppression (ablation) and rescue (restoration) of the target behavior at the internal crossing point; and (3) statistical significance via exact sign-flip tests with Holm correction.

Key Contributions

• Executable Protocol: The paper presents a fully specified, falsifiable protocol for comparing the onset of internal representations and output behaviors under matched conditions.
• Causal Temporal Comparison: It moves beyond correlation by requiring that an early internal signal must be causally necessary (via ablation/rescue) for the eventual behavior.
• Rigorous Statistical Framework: It introduces a joint maximum-statistic calibration to control false positives across dense temporal inspections and ensures power matching between internal and behavioral tests.
• Decoupling Decodability from Causality: By using independent direction discovery and causal interventions, the protocol distinguishes between information that is merely present in the weights and information that is functionally driving the backdoor.

Results
The paper explicitly states that no confirmatory results regarding the temporal hypothesis are reported.

• The author conducted "smoke tests" and synthetic unit tests to verify the pipeline, including the clean adaptation, discovery direction fitting, dense evaluation, and causal intervention logic.
• These tests confirmed the implementation details (e.g., token IDs, parameter counts, workflow timing) but were expressly non-inferential.
• Consequently, the question "Do causally necessary backdoor representations become detectable before matched behavior?" remains open. The paper provides the decision procedure and error analysis but does not provide the answer.

Significance and Claims
The paper's primary contribution is methodological, not empirical.

• It establishes a falsifiable criterion for answering whether internal precursors precede behavior, preventing researchers from mistaking token identity, thresholded behavior, probe flexibility, or unequal error budgets for genuine early warnings.
• It argues that an "early warning" claim requires four linked components: a baseline excluding static input identity, a continuous output comparator, equalized opportunities to reject (power matching), and an intervention establishing causal use rather than just decodability.
• The work supports the agenda of "developmental interpretability" by providing an operational test for monitoring training dynamics for safety-relevant precursors, rather than just auditing final models.

In summary, the paper defines how to rigorously test for backdoor precursors but does not claim to have found them, leaving the substantive question open for future application of this protocol.