🤖 AI

Balancing Image Compression and Generation with Bootstrapped Tokenization

The paper introduces SelfBootTok, a novel image tokenization method that decomposes visual information into global and local groups via self-bootstrapped learning, enabling a more efficient generator that achieves state-of-the-art generation quality with significantly reduced computation and token counts.

Original authors: Haozhe Chi, Jinghan Li, Hao Jiang, Wu Sheng, Yi Ma, Jing Wang, Yadong Mu

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haozhe Chi, Jinghan Li, Hao Jiang, Wu Sheng, Yi Ma, Jing Wang, Yadong Mu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a high-definition photo of a cat to a friend, but your internet connection is very slow. You need to compress the image into a tiny package (tokens) to send it, but when your friend receives it, they need to be able to reconstruct the picture perfectly, including the whiskers and the fur texture.

For a long time, AI researchers tried to solve this by stuffing everything into every single "package" (token). They mixed the big picture (the cat is a cat) with the tiny details (the whiskers) in every single piece of data. This was like trying to carry a whole library in one backpack; it made the backpack heavy, slow to process, and hard to organize.

This paper introduces a new method called SelfBootTok that changes the rules. Here is how it works, using simple analogies:

1. The "Blueprint vs. The Brick" Strategy (Global-Local Decomposition)

Instead of mixing everything together, SelfBootTok splits the job into two distinct teams:

The Global Team (The Blueprint): This team creates a small, compact set of tokens that describe the big picture—the shape of the cat, its pose, and its general color. Think of this as a rough architectural blueprint.
The Local Team (The Bricklayer): This team handles the fine details—the fur texture, the eye color, the whiskers.

The Innovation: In old methods, the "generator" (the AI that draws the picture) had to figure out both the blueprint and the bricks at the same time. In SelfBootTok, the Tokenizer (the compression tool) does the heavy lifting. It learns to predict the "bricks" (local details) directly from the "blueprint" (global tokens) on its own, without needing the generator to do it.

2. The "Self-Teaching" Apprentice (Self-Bootstrapped Learning)

How does the Tokenizer learn to predict the details from just the blueprint? It uses a trick called Self-Bootstrapping.

Imagine an art student (the Tokenizer) who is given a rough sketch (the global token) and asked to finish the painting. Instead of asking a teacher for every single brushstroke, the student looks at thousands of other paintings they have already seen. They learn a rule: "If the sketch shows a round shape with pointy ears, I know exactly how to draw the fur texture."

The model teaches itself this relationship using unlabeled images. It learns to say, "From this simple global token, I can automatically generate the complex local details." This means the generator doesn't have to be a genius at details; it just needs to be good at the big picture.

3. The "Universal Translator" (2D to 1D Alignment)

The paper also solves a geometry problem. The "local details" the model learns naturally exist in a 2D grid (like a photo), but the "global tokens" are a 1D line (like a sentence). To make them fit together, the authors use a mathematical tool called Optimal Transport.

Think of this as a Universal Translator. It takes the 2D grid of details and smoothly rearranges them into a 1D line that fits perfectly with the global tokens, ensuring no information is lost in the translation.

Why is this a Big Deal?

The paper claims three major wins based on their experiments:

Super Efficient: Because the generator only has to produce the "blueprint" (global tokens) and not the "bricks" (local details), it is much faster and lighter. The authors say this cuts the computational work by about 40%.
Better Quality: Even with fewer tokens (only 64), the reconstructed images look sharper and more realistic than previous methods that used many more tokens. They achieved a top-tier score (gFID of 1.56) on standard image tests.
Easy to Scale: Usually, if you want to make an AI smarter, you have to retrain the whole thing from scratch. With SelfBootTok, you can make the "Local Team" (the detail predictor) bigger and smarter without retraining the "Generator." It's like upgrading the engine of a car without having to rebuild the whole chassis.

In summary: SelfBootTok is like a smart compression system that separates the "big idea" from the "tiny details." It teaches the compression tool to fill in the details automatically, leaving the generation tool free to focus on the big picture. This makes creating high-quality images faster, cheaper, and more scalable.

Technical Summary: SelfBootTok – Balancing Image Compression and Generation with Bootstrapped Tokenization

1. Problem Statement

Current image tokenization methods face a fundamental trade-off between compression efficiency and generation quality. Standard approaches often encode redundant information by mixing global semantics and local details within every token. This "mixing" creates redundancy between tokens and complicates the training of generative models, as generators must simultaneously learn high-level semantics and fine-grained visual details. Furthermore, existing 1D tokenizers, while offering high compression, often struggle to capture both global context and local structural details within a limited token budget without sacrificing reconstruction fidelity or generation performance. The challenge lies in decomposing visual information cleanly to reduce token-level redundancy while enabling scalable training and efficient generation.

2. Methodology: SelfBootTok

The authors propose SelfBootTok, a 1D image tokenizer that addresses these issues through a global-local decomposition strategy combined with self-supervised bootstrapped learning. The framework consists of three core components:

A. Global-Local Decomposition

Instead of treating all tokens uniformly, SelfBootTok separates visual information into distinct groups:

Global Tokens ( $z_G$ ): High-level semantic representations extracted via a Vision Transformer (ViT) encoder.
Local Tokens ( $z_{L1}, z_{L2}$ ): Fine-grained visual details captured through two parallel branches:
- Local 1D Tokens: Predicted via an MLP from global tokens.
- Local 2D Tokens: Predicted via a causal transformer from global tokens.
  The model employs a multi-codebook design ( $C_{global}, C_{local1D}, C_{local2D}$ ) with soft vector quantization to ensure differentiability, allowing each group to specialize in its specific granularity.

B. Self-Supervised Bootstrapped Learning

A key innovation is the self-bootstrapping paradigm. The model learns to predict local details exclusively from global tokens using unlabeled images.

Training: The tokenizer is trained to reconstruct local features (aligned with pretrained encoders like DINOv2, SigLIP, or I-JEPA) based on the global token representation.
Generation: During the generation phase, the generator produces only the global tokens ( $z_G$ ). The local tokens ( $z_{L1}, z_{L2}$ ) are automatically predicted by the frozen local aligners (MLP and Transformer) based on the generated global tokens.
Benefit: This shifts the burden of learning visual details from the generator to the tokenizer. The generator focuses solely on high-level semantics, significantly reducing its complexity and data requirements.

C. 2D to 1D Alignment via Optimal Transport

To integrate 2D structural priors into the 1D token sequence, the method utilizes an Optimal Transport (OT) framework.

A local 2D aligner predicts a 1D sequence that is aligned with 2D VAE latent features.
The alignment minimizes an entropy-regularized optimal transport loss (solved via the Sinkhorn algorithm), effectively mapping 2D spatial distributions to a compact 1D sequence while preserving structural information.

D. Parallel Optimization and Scaling

The framework supports a unique training strategy where the tokenizer and generator can be optimized in parallel.

The tokenizer is trained to generate global tokens and predict local details.
Once the global token component is learned, it is frozen.
The generator and larger local aligners can then be scaled up and trained in parallel without retraining the generator. This decouples the design of the tokenizer size from the generator, allowing for efficient scaling.

3. Key Contributions

Clean Information Decomposition: SelfBootTok resolves redundancy by separating global semantics and local details into distinct token groups, simplifying the generator's task.
Self-Bootstrapped Prediction: The model predicts fine-grained local details directly from global tokens using unlabeled data, eliminating the need for extensive text-image pairs for detail learning and moving part of the generation process into the compression pipeline.
Efficient Generation: By requiring the generator to produce only global tokens, the method reduces computational costs by approximately 40% and training time by 54% compared to scaling approaches that require retraining generators.
Scalable Architecture: The framework allows for the scaling of local aligners (from 10M to 600M parameters) while reusing the same trained generator, achieving new state-of-the-art performance without retraining the generation model.

4. Experimental Results

Experiments were conducted on ImageNet-256 ( $256 \times 256$ ) with a token budget of 64 tokens.

Reconstruction Quality: SelfBootTok achieves a reconstruction FID (rFID) of 0.66, outperforming other 1D tokenizers (e.g., SoftVQ at 0.88, TiTok-B at 1.70) and performing comparably to methods using 256 tokens (e.g., SD-VAE).
Generation Performance: The method achieves a state-of-the-art generation FID (gFID) of 1.56 among 1D tokenizers. This is comparable to top-performing 2D tokenizers (VA-VAE: 1.35) and autoregressive models (MAR-H: 1.55), despite using significantly fewer tokens.
Efficiency: The parallel training strategy reduces total computational cost from 4.9B parameters/60K GFLOPs to 2.95B/36.6K GFLOPs and shortens training time from 28 days to 13 days.
Ablation Studies: Results confirm that the inclusion of both 1D and 2D local branches and the use of I-JEPA for alignment yield the best performance. The self-bootstrapping design significantly improves reconstruction across all metrics (rFID, PSNR, SSIM).

5. Significance and Claims

The paper claims that SelfBootTok establishes a new paradigm for image tokenization that effectively balances compression and generation. By shifting the learning of visual details to the tokenizer via self-bootstrapping, the method enables:

Superior Efficiency: Generators become far more efficient, requiring only global tokens to produce high-quality images.
Scalability: The ability to scale local representation learning using more data or parameters without retraining the generator.
State-of-the-Art Performance: Achieving top-tier generation quality (gFID 1.56) with only 64 tokens, demonstrating that compact latent representations can retain sufficient detail for high-fidelity generation when global and local information are properly decomposed.

The authors emphasize that this approach validates the potential of self-bootstrapping in image tokenization, offering a solution to the redundancy and scalability issues plaguing current 1D tokenization methods.