💻 computer science

ScaleErasure: Inference-Time Minimal Intervention for Precise Concept Erasure in Next-Scale Autoregressive Image Generation

This paper introduces ScaleErasure, an inference-time method that achieves precise concept erasure in next-scale autoregressive image generation by performing minimal, multi-dimensional intervention on logits to disentangle unsafe semantics from highly compressed early-scale representations.

Original authors: Cong Wang, Haiyu Wu, Zhiwei Jiang, Zifeng Cheng, Fei Shen, Yafeng Yin, Qing Gu

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cong Wang, Haiyu Wu, Zhiwei Jiang, Zifeng Cheng, Fei Shen, Yafeng Yin, Qing Gu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very talented artist who can draw anything you describe. However, this artist sometimes takes your instructions too literally and draws things that are inappropriate, copyrighted, or just plain unsafe. You want to stop them from drawing those specific bad things without turning off their creativity for everything else.

This paper introduces a new tool called ScaleErasure to solve this problem, specifically for a new type of "next-generation" image generator.

Here is the breakdown using simple analogies:

1. The Problem: The "Blurry Sketch" Dilemma

Most image generators work like a sculptor chipping away at a block of stone, or a painter adding details layer by layer. But this new type of generator (called Next-Scale Autoregressive) works differently.

Imagine the artist starts by drawing a tiny, 1x1 pixel dot, then a 2x2 grid, then a 4x4 grid, and so on, getting bigger and more detailed with each step.

The Issue: In the early, tiny steps (the "low scales"), the artist has to cram a huge amount of information into just a few pixels. It's like trying to describe a whole beach scene, including a specific person's swimsuit, using only four pixels.
The Entanglement: Because the pixels are so small, the "unsafe" part of the image (like nudity) gets mixed up with the "safe" part (like the sky or sand). They are tangled together in the same tiny pixel. If you try to erase the nudity, you might accidentally erase the sky too, ruining the whole picture.

2. The Solution: The "Precision Scalpel" (ScaleErasure)

Instead of trying to retrain the artist (which is slow and expensive) or editing their brain (which is risky), ScaleErasure acts as a smart editor who steps in while the artist is drawing.

It uses a "minimal intervention" strategy. It doesn't stop the artist; it just whispers a correction to the specific parts of the drawing that need it.

Here is how it works in three steps:

Step A: Zooming in on the Right Time (Scale Selection)

The editor knows that the artist is most confused during the middle stages of drawing—when the image is getting bigger but isn't fully detailed yet.

The Strategy: The editor ignores the very first tiny dots (too important to mess with) and the very last high-definition details (too late to change the main idea). They only step in during the "sweet spot" of the middle scales. This saves a lot of time and computing power.

Step B: Finding the Right Pixels (Token Selection)

The editor needs to know which pixels are the problem.

The Strategy: The editor asks the artist, "If I asked you to draw a 'nude person' instead of 'a person in a swimsuit,' which pixels would change?"
By comparing the drawing of the unsafe concept against the safe concept, the editor creates a map. They only touch the pixels that are actually responsible for the unsafe content, leaving the background (like the beach or sky) completely alone.

Step C: The Fine-Tuning (Bit-Channel Selection)

This is the most clever part. Remember how the pixels are tiny? Even inside one single pixel, there are multiple "layers" of information (like different color channels).

The Strategy: Sometimes, one pixel contains both "skin" (unsafe) and "sand" (safe) mixed together. The editor is smart enough to look inside that pixel. They only erase the specific "layer" that holds the skin information, while keeping the "layer" that holds the sand information intact. It's like removing the red paint from a mixed-color dot without removing the yellow paint.

3. The Result: A Clean Drawing, No Mess

The paper tested this method on two main problems:

Nudity: Stopping the generator from drawing inappropriate body parts.
Copyright: Stopping the generator from drawing specific characters like Pikachu or SpongeBob.

The Findings:

Better Safety: ScaleErasure was much better at removing the bad stuff than other methods.
Better Quality: Because it only touched the specific "bad" parts and left the rest alone, the final images still looked high-quality and kept the background scenery intact. Other methods often made the images look distorted or weird.
Efficiency: By only working during the middle stages of the drawing process, it didn't slow things down too much.

Summary Analogy

Imagine you are writing a story, but you accidentally type a swear word in the middle of a beautiful paragraph.

Old methods might try to rewrite the whole story from scratch (Fine-tuning) or delete the whole paragraph (Blind erasure).
ScaleErasure is like a spell-checker that knows exactly which letter is the swear word. It swaps just that one letter for a nice word, leaving the rest of the sentence, the paragraph, and the story perfectly intact.

The paper concludes that this "minimal intervention" approach is the best way to keep these powerful new image generators safe without breaking their ability to create beautiful art.

Technical Summary: ScaleErasure

Problem Statement

The paper addresses the challenge of concept erasure in next-scale autoregressive (AR) image generation, a paradigm distinct from the dominant diffusion models. While concept erasure aims to prevent models from generating unsafe content (e.g., nudity, copyrighted characters) while preserving general generative capability, existing methods are primarily designed for diffusion models and do not directly translate to next-scale AR.

The core difficulty in the next-scale AR paradigm lies in severe semantic entanglement. Unlike diffusion models that generate at fixed resolutions, next-scale AR models generate images from coarse to fine, starting at extremely low resolutions (e.g., 1×1 or 2×2 tokens). At these early scales, semantic information is highly compressed into a limited number of tokens. Consequently, a single token often encodes both unsafe semantics (e.g., "nudity") and unrelated safe semantics (e.g., "background" or "sky"). This entanglement makes it difficult to precisely select and erase unsafe concepts without degrading the overall image structure or removing unrelated content.

Methodology: ScaleErasure

The authors propose ScaleErasure, an inference-time concept erasure method that performs minimal intervention by selectively guiding predicted logits. The method operates without modifying model parameters, instead modulating the generation process at inference time.

Core Mechanism

ScaleErasure identifies logits most relevant to an unsafe concept and guides them toward a corresponding safe concept. This is achieved through two additional forward passes conditioned on the unsafe concept and a safe concept. The final guided logits ( $\ell^\star_k$ ) are a combination of the standard Classifier-Free Guidance (CFG) logits and the new Safe Guidance (SG) logits, determined by a binary mask ( $M_k$ ):
$\ell^\star_k = M_k \cdot \ell^{SG}_k + (1 - M_k) \cdot \ell^{CFG}_k$

To ensure precise and minimal intervention, the selection of logits and the application of guidance are conducted across three orthogonal dimensions:

Scale-Level Selection:
- Rationale: Early scales (1×1, 2×2) capture global structure and are too sensitive to intervention; late scales primarily refine high-frequency details where unsafe content is already fixed.
- Strategy: Guidance is restricted to a bounded intermediate range of scales (e.g., scales 3 to 8). This also enables an early-stop strategy for the additional forward passes, significantly reducing computational overhead compared to guiding all scales.
Token-Level Selection:
- Rationale: To avoid disrupting unrelated regions (e.g., the sky or background) within a token map.
- Strategy: The method leverages the model's internal cross-attention mechanism. It computes similarity scores between visual queries and concept keys (unsafe vs. unrelated concepts like "background"). Tokens with high relevance to the unsafe concept (exceeding a threshold $\tau_{token}$ ) are selected for intervention, while others are preserved.
Bit-Channel-Level Selection:
- Rationale: Next-scale AR models (specifically those using Binary Spherical Quantization) represent tokens via multiple independent bit channels. A single token may contain both unsafe and safe bits.
- Strategy: The method compares logits vectors from the prompt-conditioned pass and the unsafe-conditioned pass. Bit channels with low absolute differences (below threshold $\tau_{bit}$ ) are deemed irrelevant to the unsafe concept and are excluded from guidance. This allows for fine-grained disentanglement of semantics within a single token.

Safe Guidance Formulation

For selected logits, the method substitutes the prompt-conditioned logits with a safe-guided version. The guidance formula incorporates a penalization term to suppress residual unsafe activations:
$\ell^{SG}_k = \ell^{uc}_k + \beta_{CFG}(\ell^{sa}_k - \ell^{us}_k)$
where $\ell^{sa}_k$ and $\ell^{us}_k$ are logits conditioned on safe and unsafe concepts, respectively.

Key Contributions

ScaleErasure Framework: A novel inference-time method for concept erasure specifically tailored to the next-scale AR paradigm, addressing the unique challenge of semantic entanglement at low resolutions.
Selective Logits-Guidance Strategy: A three-dimensional selection mechanism (scales, tokens, and bit channels) that enables precise erasure of unsafe concepts while minimizing collateral damage to general generative capability.
Efficiency Optimization: The introduction of an early-stop strategy for the additional forward passes, which substantially reduces computational cost and inference latency while maintaining safety performance.

Experimental Results

The method was evaluated on the Infinity-2B model across two domains: nudity erasure (using the I2P benchmark) and copyright erasure (Pikachu and SpongeBob).

Nudity Erasure: ScaleErasure achieved the lowest total nude count on I2P (182 vs. 296 for the next best baseline, ESD-u) while maintaining superior general generation capability (lowest FID of 2.91 and highest CLIP Score of 30.68 on MS-COCO).
Copyright Erasure: The method achieved the best average $H_a$ (a metric balancing suppression and preservation) across both characters, demonstrating consistent effectiveness.
Qualitative Performance: Visual comparisons showed that ScaleErasure successfully removed unsafe content while preserving global structure and background details, whereas baselines often introduced artifacts, distortions, or failed to fully erase the concept.
Efficiency: By employing the early-stop strategy, ScaleErasure incurred only a ~9% latency increase over the base model, compared to a ~35% increase if guidance were applied across all scales.

Significance and Claims

The paper claims that ScaleErasure represents a significant step forward in securing next-scale autoregressive models, a paradigm that has been largely unexplored regarding safety mechanisms. The authors emphasize that their approach achieves precise concept erasure under conditions of severe semantic entanglement, a problem where previous methods struggle.

The significance of the work lies in its ability to suppress unsafe content (such as nudity or copyrighted material) without permanently altering model parameters or significantly degrading the model's general generative fidelity. By operating at inference time with minimal intervention, the method offers a flexible, reversible, and deployable solution for safety in emerging AR-based image generation systems. The authors note that while the method improves safety, it operates within the constraints of the specific paradigm and does not claim to solve all safety challenges in generative AI.