Technical Summary: Subliminal Learning is a LoRA Artifact

Problem Statement

Recent work has identified "subliminal learning," a phenomenon where language models (LLMs) transmit behavioral traits to other models through seemingly innocuous data. In this scenario, a "teacher" model with a specific behavioral trait (e.g., an obsession with cats) generates data unrelated to that trait (e.g., sequences of numbers). A "student" model, fine-tuned exclusively on this generated data, subsequently inherits the teacher's behavioral trait despite never seeing explicit instructions or data related to it.

While previous studies (Cloud et al., 2025; Zur et al., 2025; Schrodi et al., 2025) have demonstrated this effect and hypothesized mechanisms such as token entanglement or divergent tokens, the underlying cause and stability of this phenomenon remain unclear. Specifically, it is unknown whether this represents a robust generalization capability or a fragile artifact of specific training configurations.

Methodology

The authors investigate subliminal learning using open-weight models, primarily Qwen2.5-7B-Instruct and Gemma 3-4B-it, to bypass the opacity of proprietary API fine-tuning procedures. The experimental pipeline follows established protocols:

1. Data Generation: A teacher model is prompted with a system instruction embedding a trait (e.g., "You love cats") and generates completions for unrelated prompts (e.g., three-digit number sequences).
2. Fine-tuning: A student model is fine-tuned on this filtered numerical data using Low-Rank Adaptation (LoRA). The student never sees the teacher's trait-inducing system prompt.
3. Evaluation: The student is evaluated on its behavioral trait (e.g., "What is your favorite animal?") to measure the probability of generating the target string.

The study systematically varies several hyperparameters and contexts:

• LoRA Rank: Testing ranks from 1 to 512 and full fine-tuning.
• Context Matching: Comparing scenarios where the system prompt during fine-tuning matches or mismatches the prompt during evaluation (e.g., default Qwen prompt vs. ChatGPT prompt vs. empty string).
• Localization: Using dynamic weight grafting (selectively enabling/disabling LoRA adapters at specific token positions and layers) and activation patching to isolate where the behavioral signal is stored and activated.
• Teacher Temperature: Analyzing the impact of the teacher model's sampling temperature on the strength of the transferred signal.

Key Contributions and Results

1. Subliminal Learning is a LoRA Artifact with an Inverted U-Shape

The paper demonstrates that the strength of subliminal learning follows a distinct inverted U-shaped relationship with LoRA rank.

• Optimal Ranks: Transmission is weak or non-existent at very low ranks (underfitting) and very high ranks (approaching full fine-tuning).
• Rank Specificity: Different traits require different optimal ranks. For Qwen2.5-7B, "cat" preferences transfer optimally at rank 8, while "eagle" preferences transfer optimally at rank 64.
• Full Fine-Tuning: The effect disappears entirely when using full fine-tuning, suggesting the phenomenon is specific to the parameter-efficient constraints of LoRA.

2. Extreme Context Dependence

Subliminal learning is highly sensitive to the alignment between the fine-tuning context and the evaluation context.

• Prompt Matching: If the system prompt during evaluation differs from the one seen during fine-tuning (e.g., fine-tuning with "You are Qwen" but evaluating with "You are ChatGPT"), the subliminal effect largely disappears.
• Shared Tokens: The effect persists even with minimal shared context, provided there are shared tokens (e.g., chat template tokens like ``). If these shared tokens are removed, the effect vanishes.

3. Localization to Specific Tokens and Layers

Through dynamic weight grafting and activation patching, the authors localize the mechanism:

• Subliminal Entity: The signal is concentrated at the token positions of the "subliminal entity" (e.g., the word "Qwen" in the system prompt) seen during both fine-tuning and evaluation.
• Layer Specificity: The effect is localized to the Feed-Forward Networks (FFNs) in the early layers of the model.
• Mechanism: The model appears to use the subliminal entity as a gating mechanism. When the entity token is processed, the early-layer FFNs activate vectors that encode the behavioral bias, effectively "hijacking" the entity enrichment process.
• Singular Vectors: At lower LoRA ranks, the first singular vector of the learned adapter matrix is sufficient to recover the subliminal effect, indicating the signal is concentrated in a primary direction in parameter space.

4. Interaction with Teacher Temperature

The optimal teacher generation temperature for transferring a trait depends on the specific trait and LoRA rank.

• For "cat" (optimal at rank 8), deterministic argmax sampling (temperature 0) yields the strongest transfer.
• For "eagle" (optimal at rank 64), higher temperatures yield better transfer.
• The authors hypothesize this relates to the confidence of the teacher and student on "divergent tokens" (tokens where their predictions differ).

Significance and Claims

The paper argues that subliminal learning is not a robust, generalizable feature of LLMs but rather a fragile artifact arising from the interplay of:

1. LoRA Hyperparameters: Specifically, the rank must be in a "Goldilocks" zone to allow the model to learn an entangled solution matching the digit distribution without overfitting or underfitting.
2. Contextual Alignment: The necessity of shared tokens (system prompts or chat templates) between training and inference to "activate" the learned behavioral signal.

Implications:

• Safety: While the phenomenon is more common in open-weight models than previously shown, its fragility suggests it may not be a reliable channel for malicious data poisoning attacks in diverse settings. However, it highlights the risk of unexpected behavioral changes when fine-tuning with specific LoRA configurations and contexts.
• Interpretability: The findings suggest that behavioral traits can be encoded in specific, localized parameter updates (early FFNs) triggered by specific tokens, offering a new perspective on how LLMs generalize and how "weird generalization" occurs.
• Limitations: The authors acknowledge that their evaluations rely on string matching, which may miss semantically similar but non-matching responses. They also note that the effect is noisy and highly dependent on random seeds, particularly the data generation seed.

In conclusion, the paper reframes subliminal learning from a mysterious emergent property to a specific, context-dependent artifact of LoRA fine-tuning dynamics, characterized by an inverted U-shaped dependency on rank and a strict requirement for shared contextual tokens.