💻 computer science

Laminating Representation Autoencoders for Efficient Diffusion

This paper introduces FlatDINO, a variational autoencoder that compresses redundant DINOv2 patch features into a compact 32-token sequence, enabling diffusion models to achieve high-quality image generation with significantly reduced computational costs compared to operating on uncompressed features.

Original authors: Ramón Calvo-González, François Fleuret

Published 2026-02-05

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Ramón Calvo-González, François Fleuret

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a high-definition video of a city to a friend, but your internet connection is very slow.

The Old Way (Pixel Diffusion):
Traditionally, AI image generators work like a painter who looks at every single square inch of a canvas (pixels) to recreate the picture. This is slow and requires a massive amount of data.

The "Smart" Way (DINOv2):
Recently, researchers found a shortcut. Instead of looking at pixels, they use a "smart camera" (called DINOv2) that looks at the image and instantly understands the meaning of different parts. It sees a "dog," a "tree," and a "sky" as distinct blocks of information. This is much smarter than just seeing colors.

The Problem:
However, this "smart camera" is too chatty. For a standard image, it breaks the picture down into 256 separate blocks of information. It's like trying to describe a city by listing every single street address individually. While the information is high-quality, it's incredibly redundant. The "dog" block and the "dog's ear" block say almost the same thing. Sending all 256 blocks is still too heavy for the slow internet (the computer's processor).

The Solution: FlatDINO
The authors of this paper created a new tool called FlatDINO. Think of it as a super-efficient translator or a "summarizer."

The Compression: FlatDINO takes those chatty 256 blocks of information and squeezes them down into just 32 tiny tokens.
- Analogy: Imagine you have a 256-page book describing a city. FlatDINO reads the whole book and writes a perfect, 32-page summary that keeps all the important details but cuts out the repetition.
The Shape Change: The original blocks were arranged in a 2D grid (like a chessboard). FlatDINO flattens this into a single, straight line of 32 items. It's like taking a folded map and laying it out in a single strip to make it easier to carry.

Why This Matters (The Results)
Because the AI only has to process 32 items instead of 256, it becomes incredibly fast and efficient:

Speed: The computer needs to do 8 times fewer calculations to generate an image.
Quality: Despite being so much smaller, the summary is so good that the AI can still draw beautiful, high-quality images (specifically on the ImageNet dataset).
Efficiency: It uses significantly less energy and computing power than previous methods that tried to use the full 256 blocks.

How It Works (The Magic Trick)
The paper explains that the AI learned to group nearby things together.

In the old 256-block system, many blocks were just neighbors saying the same thing.
FlatDINO learned to say, "I don't need 8 blocks to describe this patch of sky; I can describe the whole patch with just one token."
Interestingly, if they tried to shrink it too much (down to 16 tokens), the AI got confused and started describing the image in weird horizontal stripes. But at 32 tokens, it found the "sweet spot" where it could keep the image looking natural while staying tiny.

The Bottom Line
FlatDINO is a new way to compress the "brain" of an image generator. It takes a bulky, redundant description of an image and turns it into a lean, 32-item list. This allows the AI to generate pictures much faster and cheaper, without losing the ability to create high-quality art. The authors note this is a work in progress, but the initial results show it's a very promising step toward making AI image generation more efficient.

Technical Summary: Laminating Representation Autoencoders for Efficient Diffusion (FlatDINO)

Problem Statement

Recent advancements in image generation have demonstrated that diffusion models can operate directly on self-supervised learning (SSL) patch features (e.g., from DINOv2) rather than pixel-space latents. This approach, exemplified by RAE (Zheng et al., 2025), bypasses the reconstruction-generation trade-off inherent in pixel-trained VAEs by leveraging the semantically meaningful structure already encoded in SSL features.

However, a significant computational bottleneck remains. Standard SSL encoders like DINOv2 produce dense two-dimensional grids of high-dimensional patch embeddings (e.g., 256 patches of 768 dimensions for a 256×256 image). This representation is comparable in size to the original signal, offering no computational advantage over pixel-space diffusion. The dense grid contains significant spatial redundancy because neighboring patches share substantial semantic content, making diffusion on these features unnecessarily expensive in terms of FLOPs and memory.

Methodology

The authors propose FlatDINO, a variational autoencoder (VAE) designed to compress the dense, two-dimensional DINOv2 patch embeddings into a compact, one-dimensional sequence of continuous tokens.

1. Architecture

Encoder: A Vision Transformer (ViT-B) that takes the 256 patch embeddings from a frozen DINOv2 ViT-B/14 (with registers) as input. The encoder prepends $T$ learnable register tokens to the patch sequence. After processing, only these $T$ register tokens are retained as the latent representation, discarding the original 2D spatial structure.
Decoder: A ViT-L that uses learnable register tokens as queries to attend to the compressed latent sequence and reconstruct the original 256 DINOv2 patch embeddings.
Training Objective: The model is trained using a standard VAE objective with a Mean Squared Error (MSE) reconstruction loss and a KL divergence penalty. The KL weight ( $\beta$ ) is normalized by the total latent dimensionality ( $T \times d$ ) to ensure consistent regularization pressure across different configurations.
Latent Generation: A generative model (specifically a DiT-XL with flow matching) is trained on the compressed FlatDINO latent space. The model learns a velocity field to transport samples from a standard Gaussian prior to the data distribution along straight paths.

2. Key Design Choices

1D Compression: Inspired by TiTok (Yu et al., 2024a), the method compresses the 2D grid into a 1D sequence. The authors found that 32 tokens provide a favorable trade-off between reconstruction fidelity and sequence length.
Feature Dimension Reduction: While the token count is reduced from 256 to 32 (an 8× reduction), the per-token feature dimension is also compressed (e.g., from 768 to 128) to further reduce total dimensionality.
Flow Matching: The generative model utilizes flow matching rather than standard diffusion, operating directly on the 1D latent sequence with learned positional embeddings.

Key Contributions

FlatDINO: The first method to compress self-supervised patch features into a one-dimensional latent representation, effectively discarding the original 2D spatial structure while preserving semantic content.
Efficiency Gains: Demonstrates that diffusion models trained on FlatDINO latents achieve competitive generation quality while significantly reducing computational costs. Specifically, the 8× reduction in sequence length leads to an 8× reduction in FLOPs per forward pass and up to a 4.5× reduction in FLOPs per training step compared to diffusion on uncompressed DINOv2 features.
Analysis of Tokenization: Through token ablation studies, the authors show that the model learns localized, blob-like receptive fields for most tokens, effectively partitioning the image spatially. They identify a "phase transition" where reducing tokens below a certain threshold (e.g., from 32 to 16) forces the model to adopt less efficient encoding strategies (horizontal stripes), leading to a sharp drop in quality.

Results

Experiments were conducted on ImageNet 256×256 using a DiT-XL architecture.

Generation Quality: A DiT-XL trained on FlatDINO latents (32×128) achieves a gFID of 1.85 with classifier-free guidance (CFG) and 3.34 without CFG. This is competitive with RAE (which achieves 1.41 gFID with CFG on uncompressed features) and other state-of-the-art latent diffusion models, despite the significant compression.
Computational Efficiency:
- Inference: FlatDINO requires approximately 8× fewer FLOPs per forward pass compared to operating on full DINOv2 patches.
- Training: Due to the reduced sequence length and the inclusion of the encoder overhead, FlatDINO requires 4.5× fewer FLOPs per training step for DiT-XL models compared to the RAE baseline.
Reconstruction: The 32×128 configuration achieves an rFID of 0.77 (compared to 0.62 for the uncompressed RAE baseline), indicating high-fidelity reconstruction of the feature space.
Token Ablation: The study reveals that 32 tokens are sufficient to maintain localized receptive fields. Reducing to 16 tokens causes a qualitative shift to horizontal stripe encoding and a significant drop in reconstruction quality.

Significance and Claims

The paper positions FlatDINO as a step toward efficient, semantically grounded image synthesis. The authors claim that by exploiting the spatial redundancy inherent in 2D patch grids, it is possible to achieve an 8× reduction in token count without sacrificing the convergence benefits of operating in a semantically organized latent space.

The work suggests that diffusion on compressed semantic representations is a practical approach for resource-constrained settings. The authors acknowledge that their current results are preliminary and that the model has not fully converged (trained for 600 epochs vs. 800 for RAE). They attribute the remaining gap in generation quality to insufficient training duration and the need for diffusion recipes specifically tailored to compressed semantic latents. Future work is proposed to investigate extended training schedules and optimized SSL autoencoders to close this gap.