Technical Summary: BitResEdit

Problem Statement

Text-guided image editing with Visual Autoregressive (VAR) generators faces a fundamental challenge: existing editing methods, largely designed for diffusion models, do not transfer directly to VAR architectures. Diffusion editors rely on inverting continuous trajectories or manipulating attention maps, whereas VAR models (such as Infinity) generate images by sampling discrete, multi-scale codes through a causal, coarse-to-fine process.

Current VAR editing approaches (e.g., AREdit, VAREdit) operate primarily on token streams, feature maps, or flat next-token logits. These methods fail to leverage two native structural properties of bitwise-residual VAR generators:

1. Per-bit Bernoulli Prediction: The prediction head outputs independent log-odds for each bit rather than a categorical distribution over a token vocabulary.
2. Additive Multi-scale Residuals: The final image latent is assembled via a fixed linear sum of upsampled residual codes from multiple scales.

Existing methods miss the opportunity to guide the sampling process at the bit level and to localize edits algebraically within the continuous residual code field. Consequently, they struggle to balance target alignment (changing the object) with background preservation (keeping the rest of the image intact) without fine-tuning or auxiliary models.

Methodology

The authors propose BitResEdit, a training-free editor for bitwise-residual VAR generators (specifically Infinity). The method decouples the editing process into two distinct stages that operate on the generator's native structures: BitEdit (decision-time) and ResEdit (combination-time).

1. BitEdit: Bernoulli Trust-Region Source-Negative Guidance

BitEdit operates before sampling, modifying the per-bit log-odds to guide which bits are sampled.

• Source-Negative Contrast: The method computes a contrast vector $\Delta^k$ between the source prompt ($S$) and the target prompt ($T$) at every bit position, scale, and spatial location. This contrast is anchored to the target prompt's log-odds rather than the unconditional branch to avoid guiding shared features.
• Tilting: The post-Classifier-Free Guidance (CFG) log-odds are tilted along this source-target contrast direction, scaled by a guidance strength $\eta^k$ and a temperature $\tau$.
• Bernoulli-KL Trust Region: To prevent the tilted log-odds from drifting too far from the model's natural distribution (which could cause artifacts), the update is projected onto a per-bit Bernoulli-KL trust region around the clean CFG anchor. This is solved via bisection to find the maximum mixing weight that satisfies the KL-divergence bound.
• Annealing: The guidance strength $\eta^k$ is linearly annealed from the coarsest scale (global layout) to the finest scale (texture), ensuring edits focus on semantic structure first.

2. ResEdit: Gated Code-Space Residual Editing

ResEdit operates after sampling, determining where and how the sampled changes are written into the image's continuous latent space.

• Residual Formation: The sampled bits are converted to continuous codes and upsampled to the final VAE resolution. A residual is computed between the edited code and the cached source code.
• Mask-Gated Injection: A localization mask (provided by ground truth or an external pipeline) is processed (dilated, resized, blurred) to create a soft gate $G_{final}$.
• Algebraic Composition: The final edited latent is constructed by adding the gated residual to the source latent: $\tilde{c}_{out} = c_{src} + \alpha \cdot G_{final} \odot \Delta \tilde{c}$.
• Exact Preservation: Because the gate is zero outside the edit region, the residual vanishes exactly, preserving the background latent features algebraically without relying on attention maps or token recombination.

3. Coupling

The two components are coupled through the cumulative edited latent, which serves as the visual prefix for the next scale in the autoregressive generation. BitResEdit treats the edited prefix as a continuous code-space composition rather than a discrete token sequence, feeding it back into the generator's downsampling operator.

Key Contributions

1. Identification of Native Editability: The paper identifies two algebraic properties of bitwise-residual VAR generators that enable training-free editing: per-bit Bernoulli factorization (allowing closed-form KL bounds) and additive multi-scale code summation (allowing gated algebraic composition).
2. BitResEdit Framework: The instantiation of these properties into a two-stage editor:
   • BitEdit: A source-negative guidance mechanism that operates in log-odds space with a Bernoulli-KL trust region.
   • ResEdit: A mask-gated residual injection mechanism that operates in code space.
3. Empirical Performance: On the PIE-Bench dataset using the Infinity-2B backbone, BitResEdit achieves the strongest text alignment among same-backbone VAR editors while maintaining competitive background preservation.

Results

The method was evaluated on PIE-Bench (700 source-target pairs) against diffusion-based editors (e.g., P2P, MasaCtrl), flow-matching editors (e.g., FlowChef, VAGS), and other VAR editors (AREdit, VAREdit, EditInfinity).

• Text Alignment: BitResEdit achieves a CLIP score of 24.54 on the edited region, outperforming the strongest prior Infinity-2B editor (EditInfinity) by +1.07 points. It also leads in whole-image alignment among same-backbone methods.
• Background Preservation: While EditInfinity leads in preservation metrics (SSIM 92.12 vs. 91.71), BitResEdit remains competitive, staying within 0.42 SSIM points of the best preservation method while significantly outperforming it in alignment.
• Ablation Studies:
  • BitEdit is primarily responsible for text alignment improvements.
  • ResEdit is primarily responsible for background preservation.
  • The KL trust region in BitEdit acts as a safety rail but rarely binds at default settings, suggesting the source-negative tilt is naturally close to the target distribution.
  • Annealing the guidance strength is critical; constant guidance reduces alignment significantly.
• Latency: BitResEdit is training-free and runs in 10.41 seconds per image on an A100 GPU. This is comparable to other training-free VAR methods (AREdit: 11.84s) and significantly faster than per-image fine-tuning methods like EditInfinity (~467s).

Significance and Claims

The paper claims that BitResEdit demonstrates a new paradigm for editing autoregressive models: "Edit the bits, diff the codes."

• Native Geometry: The authors argue that effective editing in VAR models should exploit the generator's native geometry (bitwise log-odds and additive residuals) rather than treating the model as a black box or adapting diffusion-specific controls.
• Training-Free Efficiency: By leveraging the generator's existing arithmetic and probability structures, BitResEdit achieves strong results without fine-tuning, auxiliary generative models, or per-image optimization.
• Complementary Roles: The work highlights that decision-time bit guidance and combination-time code composition are complementary; one drives semantic alignment, while the other enforces spatial fidelity.
• Limitations: The authors acknowledge that the method relies on the quality of the localization mask. While robust to automatic segmentation masks, it degrades significantly with coarse bounding boxes. Additionally, object deletion remains a challenge because the residual editor is preservation-biased and struggles to synthesize background content for under-specified (subtractive) prompts.

In summary, BitResEdit provides a principled, training-free approach to text-guided editing for bitwise-residual VAR models, achieving state-of-the-art alignment for its backbone class while preserving the background through algebraic code composition.