🤖 machine learning

Deep Dreams Are Made of This: Visualizing Monosemantic Features in Diffusion Models

This paper introduces Latent Visualization by Optimization (LVO), a mechanistic interpretability technique that combines sparse autoencoders with latent-space optimization to visualize monosemantic features in diffusion models, successfully revealing coherent concepts like human figures and roses that are obscured by polysemantic representations in baseline models.

Original authors: Adam Szokalski, Mateusz Modrzejewski

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Adam Szokalski, Mateusz Modrzejewski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, complex machine (like a modern AI that creates images) that works like a giant, black box. You give it a prompt, and out comes a picture, but no one inside the box knows exactly why it decided to draw a specific rose or a specific type of cable. The paper "Deep Dreams Are Made of This" is like a team of mechanics trying to open that box and understand its internal gears.

Here is a simple breakdown of what they did, using everyday analogies:

The Problem: The "Polysemantic" Soup

Inside these AI machines, the "neurons" (the internal switches) are messy. The authors call this polysemanticity.

The Analogy: Imagine a light switch in your house that controls the kitchen light, the garage door, and the front porch light all at once. If you flip the switch, you don't know which light is actually turning on. In the AI, a single internal feature might be responsible for "dogs," "butterflies," and "fur textures" all mixed together. Because they are mixed up, it's hard to understand what the AI is actually thinking.

The Solution: The "Disentanglement" Tool

To fix this, the researchers used a tool called a Sparse Autoencoder (SAE).

The Analogy: Think of the SAE as a master chef who takes that messy "soup" of mixed ingredients and separates them into individual, pure bowls. Now, instead of one switch controlling three things, they have three separate switches: one just for dogs, one just for butterflies, and one just for fur. This is called monosemanticity (one meaning per feature).

The New Technique: "Latent Visualization by Optimization" (LVO)

The authors wanted to see what these "pure" switches actually look like when turned on. They developed a new method called LVO.

The Analogy: Imagine you want to know what a specific light switch controls. You don't just flip it and hope; you try to build a room that forces that switch to turn on as brightly as possible. You keep adjusting the furniture, the paint, and the lighting until the switch is screaming "ON!"
The Twist: In this AI, the "room" isn't a real picture; it's a hidden, compressed code (called "latent space"). The researchers optimized this code to see what image would make a specific feature light up.

The Four Special Ingredients

Because this AI works differently than older ones, the researchers had to invent four special rules to make their "light switch" test work:

Time-Step Activity Analysis: The AI builds images from scratch, adding detail step-by-step (like a sculptor chipping away stone). A feature might be important at the beginning (rough shape) or the end (fine details).
- Analogy: They checked the "schedule" to see exactly when during the sculpting process a specific tool is used, so they didn't try to use a chisel when the sculptor is supposed to be sanding.
Schedule-Matched Noise: The AI expects to see a blurry, noisy image at every step. If you show it a perfect, clean image too early, it gets confused.
- Analogy: If you are trying to teach someone to swim in a pool, you can't suddenly drop them into dry land. They had to keep the "water" (noise) at the right level for the specific step of the process.
Prior Initialization: They didn't start from a blank, random screen. They started with a "hint" image.
- Analogy: Instead of trying to guess a password from scratch, they started with a hint like "It starts with the letter A." This keeps the result from turning into static noise.
Regularization (The "Anti-Noise" Rules): When you try to force a switch to turn on, the AI often creates weird, high-frequency static (like TV snow) because that's the easiest way to trigger the switch.
- Analogy: They added rules to say, "No, that's just static noise. Make it look like a real object." They used mathematical filters to smooth out the "snow" and force the AI to create recognizable shapes.

What They Found

They tested this on a popular image generator (Stable Diffusion) and compared the "messy" raw switches against the "clean" SAE switches.

The Messy Switches (Raw Layer): When they tried to visualize the original, mixed-up switches, the results were confusing. One switch tried to show a dog, a butterfly, and fur all at once. The images looked like a muddy mess of unrelated textures.
The Clean Switches (SAE Features): When they used the SAE to separate the meanings, the results were amazing.
- One switch clearly drew diagonal lines (a composition style).
- One switch drew a human figure.
- One switch drew cables.
- One switch drew waterfall foam.

Why This Matters

The paper claims that looking at the "dataset examples" (images the AI was trained on) or "steering" (telling the AI to make something more prominent) isn't enough.

The Analogy: Looking at the training data is like looking at a library to guess what a specific book is about; you might miss the theme. "Steering" is like shouting "More roses!" and seeing what happens, but you don't know why the AI made them.
The Result: LVO is like opening the book and reading the specific chapter. It reveals exactly what a feature is "thinking" about, even if that feature is something abstract like "diagonal composition" that you wouldn't notice just by looking at the training photos.

Summary

The paper introduces a new way to "X-ray" AI image generators. By separating mixed-up concepts and using a special optimization process, they can generate clear, human-understandable images that show exactly what specific parts of the AI's brain are responsible for. They found that without this separation, the AI's thoughts are a muddy soup, but with it, the thoughts are clear, distinct, and often surprisingly specific (like "cables" or "diagonal lines").

Technical Summary: Deep Dreams Are Made of This

Problem Statement

While mechanistic interpretability has advanced significantly for convolutional neural networks (CNNs) through techniques like feature visualization by optimization, applying these methods to Latent Diffusion Models (LDMs) presents unique challenges. LDMs operate in a compressed variational-autoencoder (VAE) latent space and utilize an iterative denoising process conditioned on time-steps. Classical feature visualization methods, which optimize pixel-space inputs to maximize feature activation, fail in this context due to:

Polysemanticity: Raw neural features often represent multiple unrelated concepts simultaneously, making visualizations incoherent.
Latent Space Constraints: Optimizing directly in the latent space introduces failure modes such as decoder out-of-range values (causing oversaturation) and high-frequency noise artifacts.
Temporal Complexity: Features in diffusion models may be active at different time-steps for different reasons, requiring a method to isolate specific activation conditions.
Limitations of Existing Tools: Current diffusion interpretability methods, such as dataset examples and feature steering, are limited by dataset content or only reveal downstream effects rather than the specific inputs that activate a feature.

Methodology: Latent Visualization by Optimization (LVO)

The authors propose Latent Visualization by Optimization (LVO), a technique that extends feature visualization by optimization to LDMs. The core objective is to optimize a latent vector $z_{opt}$ via gradient ascent to maximally activate a specific target feature. The method introduces four diffusion-specific components:

Time-Step Activity Analysis: Instead of visualizing features across all 1,000 time-steps, the method identifies "activity peaks" where a feature is most frequently among the top- $k$ activations across the dataset. Visualizations are generated specifically for these peak time-steps to isolate the feature's role at that stage of denoising.
Latent-Space Optimization: Optimization occurs within the VAE latent space rather than pixel space to avoid the computational cost and gradient artifacts associated with backpropagating through the decoder.
Schedule-Matched Noise Injection: To maintain the correct conditioning for the U-Net, the optimized latent is re-noised at every forward pass using the scheduler coefficients of the target time-step ( $z_{noise} = \sqrt{\bar{\alpha}_t}z + \sqrt{1-\bar{\alpha}_t}\epsilon$ ). This ensures the model sees the expected signal-to-noise ratio.
Prior Initialization: Optimization is initialized from a "prior" image generated by steering the target feature on its active time-steps. This keeps the optimization trajectory within the interpretable manifold, preventing collapse into high-frequency noise.

Regularization Strategies:
To combat high-frequency noise and latent-space artifacts, the authors employ a stack of regularizers:

Pixel-Space Transferred: Transformation robustness (jitter, rotation, scaling), spectral filtering (attenuating high-frequency gradients), and gradient smoothing.
Latent-Specific Penalties: A Range Penalty (penalizing values outside the decoder's training range, typically $[-3, 3]$ ) and a Moment Penalty (keeping the latent's mean and variance close to a standard normal distribution).

Disentanglement via SAEs:
The study compares two variants:

Raw-Layer Baseline: Optimizes a single raw activation channel.
SAE Variant: Uses a Sparse Autoencoder (SAE) trained on the layer to disentangle polysemantic representations into monosemantic features before optimization.

Key Contributions

LVO Framework: The proposal of a complete pipeline for feature visualization in LDMs, integrating time-step analysis, schedule-matched noise, prior initialization, and latent-specific regularization.
Disentanglement Baseline: The provision of a controlled raw-layer baseline to isolate the interpretability gains provided by SAE disentanglement.
Regularization Ablation: The first systematic ablation of pixel-space regularizers in the latent domain, revealing that optimal regularization strategies differ significantly between raw-layer and SAE variants (e.g., SAE features benefit from moment penalties but are hypersensitive to total variation and gradient smoothing).
Complementary Insight: Demonstration that LVO provides insights distinct from dataset examples and steering by directly revealing the input conditions that activate a feature.

Results

The method was evaluated on Stable Diffusion 1.5 fine-tuned on the Style50 dataset, specifically targeting the up_blocks.1.attentions.1 layer.

Polysemantic Baseline: Optimizing raw layer activations resulted in visualizations that were often incoherent or mixed multiple concepts (e.g., mixing animal parts like eyes and fur without forming a recognizable animal). Steering on these raw features produced uninformative, generic images.
Monosemantic SAE Features: The SAE-based LVO produced clear, consistent visualizations of recognizable concepts across multiple seeds and time-steps. Notable examples include:
- Feature 9984: Consistently visualized a "diagonal composition" (strokes oriented top-left to bottom-right), a structural concept not obvious from the diverse dataset examples.
- Feature 10331: Clearly depicted human figures.
- Other Features: Successfully visualized object parts (cables, roses), textures (waterfall foam), and perspectives.
Regularization Effects: Calibration showed that prior initialization was essential to prevent noise collapse. Furthermore, regularization terms like transformation robustness and moment penalties were found to help the optimizer escape uninterpretable local maxima, revealing clearer structures (e.g., turning a "muddy" texture into a distinct cable structure).

Significance and Claims

The paper claims that LVO fills a critical niche in diffusion model interpretability. While dataset examples are limited by content diversity and steering reveals downstream effects, LVO directly answers "what activates a feature." The authors emphasize that the technique is viable only when combining all four proposed components (time-step analysis, noise injection, prior initialization, and appropriate regularization).

The work is framed primarily as a tool for defensive interpretability, supporting safety auditing, targeted concept unlearning, and content moderation. The authors acknowledge a dual-use risk where identifying feature activations could theoretically aid in bypassing safety filters, but assess this risk as limited given that the underlying models and SAEs are already public and the method reveals activation causes rather than suppression techniques.

The study concludes that LVO surfaces structural and compositional features (like diagonal arrangements) that are invisible to standard inspection of dataset examples, offering a complementary perspective essential for the mechanistic study of latent diffusion models.