💻 computer science

Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models

This paper proposes ICM, a precise steering method that localizes and intervenes in specific self-attention layers to effectively resolve implicit generative choices and debias text-to-image diffusion models.

Original authors: Katarzyna Zaleska, Łukasz Popek, Monika Wysoczańska, Kamil Deja

Published 2026-04-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Katarzyna Zaleska, Łukasz Popek, Monika Wysoczańska, Kamil Deja

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you ask a master painter, "Draw me a picture of a person." You don't specify if they should be a man or a woman, young or old, or what their hair color is. The painter has to make a choice. In the world of AI, this is called a diffusion model making an "implicit decision."

For a long time, scientists thought the painter's brain worked like a giant, blurry cloud where all the decisions happened everywhere at once. They tried to fix bad habits (like always painting men as doctors and women as nurses) by shouting instructions into the painter's ear or tweaking the whole painting process. But this often ruined the picture, making it look weird or blurry.

This paper, titled "Attention, May I Have Your Decision?", introduces a new way to understand and fix these AI painters. Here is the breakdown using simple analogies:

1. The Problem: The "Black Box" Painter

When you give an AI a vague prompt (like "a photo of flowers"), it has to guess the details. If the AI has learned from biased data, it might guess "red roses" every time, or assume a "doctor" is always a man.

The Old Way: Researchers tried to find where the AI made these choices by injecting specific words (like "woman") into the prompt and seeing which part of the AI's brain reacted.
The Flaw: This is like trying to understand how a chef decides to add salt by shouting "SALT!" at them. It tells you where they listen to instructions, but not where they decide what to do when no one is giving instructions.

2. The Discovery: The "Decision Switchboard"

The authors realized that when the AI has to make a choice on its own (without a specific prompt), it doesn't use the part of its brain that listens to text. Instead, it uses a different part: Self-Attention Layers.

The Analogy: Imagine the AI is a massive orchestra.
- Cross-Attention Layers are the musicians reading the sheet music (the text prompt). If you tell them "Play a sad song," they follow the notes.
- Self-Attention Layers are the musicians improvising. When the sheet music says "Play a song" but doesn't say which song, these musicians decide the melody, the tempo, and the style.
The Finding: The paper discovered that the "improvising musicians" (Self-Attention) are the ones deciding if the person in the photo is a man or a woman, young or old. They are the true "decision-makers" for vague prompts.

3. The Solution: The "Surgical Scalpel" (ICM)

Instead of trying to fix the whole orchestra or shouting at the conductor, the authors developed a method called ICM (Implicit Choice-Modification).

Step 1: The Detective Work (Probing): They let the AI paint thousands of "generic" pictures (like "a photo of a person"). Then, they used a smart camera (an external classifier) to look at the pictures and say, "Ah, this one is a man, that one is a woman."
Step 2: Finding the Switch: They checked the orchestra's internal notes (activations) to see exactly when and where the musicians started distinguishing between "man" and "woman." They found that a specific few layers in the middle of the process were the "switches" where this decision became clear.
Step 3: The Gentle Nudge: Instead of rewriting the whole song, they just gently nudged those specific "improvising musicians" in the opposite direction.
- If the AI was leaning 90% toward painting men, they gave a tiny nudge to the specific layers responsible for that decision to lean toward women instead.

4. Why This is a Big Deal

Previous methods were like trying to fix a leaky roof by replacing the entire house. It worked, but the house looked terrible and cost a fortune.

Old Way: Fixing the whole model often made the images look blurry, distorted, or completely different from what you asked for.
New Way (ICM): Because they only touched the specific "decision switches," the images remained high-quality and sharp. They successfully reduced bias (making sure doctors could be women, or presidents could be Black) without ruining the art.

The Takeaway

The paper teaches us that when AI makes a guess, it's not a random cloud of confusion. It's a precise, localized decision made by specific parts of the system. By finding the exact "switch" where the AI decides "Man vs. Woman" or "Young vs. Old," we can gently steer the AI toward fairness without breaking its ability to create beautiful images.

In short: They stopped trying to shout at the AI and started whispering the right instructions to the exact part of its brain that was making the wrong choices.

1. Problem Statement

Text-to-Image (T2I) diffusion models are powerful but operate as "black boxes," particularly when handling underspecified prompts (e.g., "a photo of a person" without specifying gender, age, or race). In these scenarios, the model must make implicit decisions to fill in missing details based on patterns learned during training.

The Issue: These implicit decisions often lead to societal biases (e.g., defaulting to male doctors or specific racial representations) and are difficult to control.
Limitation of Existing Methods: Current localization techniques primarily rely on prompt injection (injecting specific attributes into the text prompt) to trace which layers handle specific concepts. The authors argue this approach fails to capture the mechanism of implicit decision-making because it forces an explicit condition rather than observing the model's natural inference from noise.
Hypothesis: The mechanism for resolving ambiguous concepts is computationally localized within the model's architecture, distinct from the mechanism used for explicit text conditioning.

2. Methodology: ICM (Implicit Choice-Modification)

The authors propose ICM, a framework that uses linear probing to localize and intervene in the specific layers responsible for implicit decisions. The process consists of three main stages:

A. Linear Probing for Layer Selection

Instead of injecting prompts, the method analyzes the model's natural behavior:

Data Generation: Generate a dataset of images using general prompts (e.g., "a photo of a person") without specifying attributes.
Activation Extraction: Extract intermediate activation vectors ( $H_{l,t}$ ) from all layers ( $l$ ) and timesteps ( $t$ ) during generation.
Pseudo-Labeling: Use an external classifier (e.g., CLIP-based or FairFace) to label the generated images with the attributes they actually manifested (e.g., "man," "woman," "young," "old").
Probe Training: Train linear probes (logistic regression) on the activations of each layer to predict these pseudo-labels.
Localization: Identify layers with the highest classification accuracy. These layers represent the point where the model's internal representation becomes linearly separable regarding the implicit attribute.

B. Key Finding: Self-Attention vs. Cross-Attention

A critical discovery of the paper is that Self-Attention layers exhibit significantly higher attribute separability for implicit decisions compared to Cross-Attention layers.

Cross-Attention: Primarily handles explicit text-to-image alignment.
Self-Attention: Responsible for propagating stochastic cues from the initial noise and solidifying them into a unified generation (e.g., deciding gender based on noise patterns).
Conclusion: Implicit decisions are governed principally by self-attention layers, making them the optimal target for intervention.

C. Intervention Strategies

Once the critical layers are identified, the authors apply two types of targeted interventions:

Activation Steering: During inference, a steering vector ( $s_{l,t}$ ) derived from the linear probe weights is added to the activations of the selected layers:
$H'_{l,t} = H_{l,t} + \alpha s_{l,t}$
This shifts the generation toward a desired attribute (e.g., "woman") without retraining.
Fine-tuning (LoRA): The model is fine-tuned using Low-Rank Adaptation (LoRA) but only on the selected subset of layers, rather than the entire model or all cross-attention layers.

3. Key Contributions

Novel Localization Technique: Introduced a probing-based method to localize implicit decision-making layers without the confounding effects of prompt engineering.
Architectural Insight: Demonstrated that self-attention layers are the primary locus for resolving ambiguous concepts, challenging the prevailing focus on cross-attention layers for concept editing.
ICM Framework: Proposed a precise steering and fine-tuning method that targets only the most discriminative layers, minimizing collateral damage to image quality.
Scalability: Validated the approach across different architectures, including UNet-based models (Stable Diffusion v1.5, SDXL) and Transformer-based models (SANA).

4. Experimental Results

The authors evaluated ICM on debiasing tasks (Gender, Age, Race) using Stable Diffusion v1.5, SDXL, and SANA.

Performance vs. State-of-the-Art:
- ICM (Steering) achieved a Fairness Discrepancy (FD) of 0.087 for gender debiasing, outperforming or matching methods like DIFFLENS and Latent Direction.
- Crucially, ICM maintained superior image quality (FID) and text alignment (CLIP-T) compared to methods that fine-tune the whole model or steer all layers.
Self-Attention vs. Cross-Attention:
- Steering self-attention layers resulted in significantly better debiasing (lower FD) compared to steering cross-attention layers, while maintaining comparable image fidelity.
Layer Selection Impact:
- Steering only the top-performing layers (e.g., top 20 out of 70 in SDXL) achieved an 83% steering success rate with high image similarity (CLIP-I 0.893).
- Steering all layers achieved higher success (91%) but severely degraded image quality (CLIP-I dropped to 0.779).
- Steering random or worst-performing layers resulted in poor debiasing or severe artifacts.
Prompt Injection vs. Probing:
- Probes trained on general prompts (implicit) were more effective for debiasing than those trained on specific prompts (explicit), confirming that the internal mechanisms for default decisions differ from those for explicit conditioning.

5. Significance

This work fundamentally shifts the understanding of how diffusion models make decisions. By identifying that self-attention layers are the "decision-making" hubs for implicit attributes, the paper provides a more precise and efficient path for controllable generation and bias mitigation.

Efficiency: It proves that modifying a small subset of layers is sufficient for high-precision control, reducing computational costs compared to full-model fine-tuning.
Quality Preservation: Targeted interventions prevent the "artifacts" and quality degradation often seen in broader, less precise editing methods.
Generalizability: The method scales effectively to larger, modern architectures (SDXL, SANA), suggesting it is a robust solution for the next generation of generative AI.

The code is available at https://github.com/kzaleskaa/icm.