🤖 AI

AnchorEdit: Maintaining Temporal Consistency in Multi-turn Image Editing via Causal Memory

AnchorEdit introduces the first autoregressive diffusion framework for high-resolution, long-term multi-turn image editing that utilizes a three-stage training curriculum and a causal memory mechanism to effectively eliminate identity drift and error accumulation across extended interaction sequences.

Original authors: Hang Xu, Xiaoxiao Ma, Guohui Zhang, Yu Hu, Siming Fu, Jie Huang, Lin Song, Haoyang Huang, Nan Duan, Feng Zhao

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hang Xu, Xiaoxiao Ma, Guohui Zhang, Yu Hu, Siming Fu, Jie Huang, Lin Song, Haoyang Huang, Nan Duan, Feng Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are working with a very talented but slightly forgetful digital artist. You ask them to edit a photo: "Make the car red." They do it perfectly. Then you say, "Now make it a convertible." They do that too. But by the third or fourth request, the car starts to look weird. Maybe it's no longer the same car, or the background has turned into a different city. This is the problem of "identity drift" in current AI image editors—they lose track of who or what they are editing as the conversation goes on.

The paper introduces AnchorEdit, a new system designed to fix this. Think of it as giving that digital artist a super-memory and a strict set of rules so they can handle long, complex editing sessions without losing their mind (or the subject's face).

Here is how it works, broken down into simple concepts:

1. The Problem: The "Video" vs. "Conversation" Mismatch

Current AI editors that are good at keeping things consistent often use video models. Video is great because it shows how things move and stay the same over time. However, most video models look at the "whole picture" at once (like reading a book from start to finish to understand a sentence).

But editing an image is like a conversation. You give an instruction, the AI does it, and then you give the next instruction based only on what just happened. You can't peek at the future instructions.

The Flaw: Existing video models try to "peek" at the future to stay consistent, which breaks the logic of a real conversation.
The Fix: AnchorEdit uses an Autoregressive (AR) approach. This means it treats editing like a chain of dominoes. It only looks at the past (the original photo and previous edits) to decide what to do next, just like a human would.

2. The Solution: A Three-Step Training Camp

To teach this AI to be a master editor, the researchers put it through a three-stage training curriculum:

Stage 1: The "Don't Change Anything" Drill (Identity Preserving)
Imagine a teacher telling the student, "Here is a photo. I want you to 'edit' it, but actually, I want you to keep it exactly the same."
The AI learns to recognize the "soul" of the object (the identity) so it doesn't accidentally morph a dog into a cat when asked to change its collar. They also use a special math trick (called Expanded RoPE) to teach the AI that an "edit" is a big jump in time, not just a tiny movement like a video frame.
Stage 2: The "Self-Correction" Drill (Causal Forcing)
This is the most important part. In normal training, the AI is always shown the perfect previous steps. But in real life, if you make a small mistake in step 1, step 2 has to work with that mistake.
The researchers use a technique called Self-Rollout. They let the AI make its own mistakes during training, then ask it to fix the next step based on its imperfect previous work. It's like a musician practicing a song, making a mistake, and then learning how to recover and keep the rhythm going without stopping. This stops errors from piling up like a snowball rolling down a hill.
Stage 3: The "Speed Run" (Distillation)
The AI is now smart but slow. To make it fast enough for real use, they compress it. They teach a smaller, faster version of the AI to mimic the big, slow one. The result is a model that can do high-quality edits in just 4 steps instead of dozens.

3. The Secret Weapon: The "Anchor" and the "Sliding Window"

When the AI is actually editing a photo during a long session (say, 10 turns in a row), it needs a way to remember the original subject without getting overwhelmed by all the history.

The Anchor: The AI keeps the very first image (the original photo) locked in its memory forever. No matter how many edits happen, it always has a reference point to say, "Wait, this person's face should still look like this."
The Sliding Window: It also keeps a "recent history" of the last few edits to understand the immediate context.
The Magic Trick: Even as old edits are forgotten to save space, the AI uses a special numbering system (Strided RoPE) to ensure the "distance" between the original photo and the current edit stays mathematically consistent. It's like keeping a ruler in your pocket that always measures the distance from the start, even if you throw away the middle pieces of paper.

4. The Results

The researchers built a new test (a benchmark) specifically to stress-test these long editing chains. They found that:

Old methods (like ChronoEdit or VINCIE) start to fail after a few turns. The subject's face changes, or the background gets messy.
AnchorEdit stays consistent even after 10+ turns. It can change a car from red to blue, to rusted steel, to a submarine, and back to a car, and the "driver" inside remains the same person throughout the whole journey.

Summary

AnchorEdit is like giving an AI editor a permanent anchor to the original image and a training regimen that teaches it how to recover from its own mistakes. It moves away from "looking at the whole video" to "having a real-time conversation," ensuring that after 10 requests, the image still looks like the same person or object you started with.

Technical Summary: AnchorEdit

Problem Statement

Multi-turn image editing is critical for iterative design workflows, yet current models struggle with identity drift and error accumulation over successive modification steps. While recent research has leveraged video priors to improve temporal consistency, existing video-based editing methods typically rely on bidirectional attention mechanisms. This architecture is fundamentally misaligned with the causal, sequential nature of interactive editing, where each step must depend strictly on the source image and previous states without access to future instructions. This misalignment leads to degraded instruction following, visual inconsistencies, and an inability to extrapolate to longer editing chains. Furthermore, fixed-length contexts in current models prevent truly online interaction.

Methodology

The authors propose AnchorEdit, the first autoregressive (AR) diffusion-based framework designed specifically for high-resolution (1024p), long-term multi-turn image editing. The approach bridges the gap between video priors and causal inference through a systematic three-stage training curriculum and a specialized inference memory mechanism.

1. Three-Stage Training Curriculum

Stage 1: Identity-Preserving Single-Turn Pretraining
- Adapts a video generation backbone (Wan2.1-T2V-14B) to instruction-guided editing.
- Expanded RoPE: Increases the relative RoPE distance between the source and edited states (from $\Delta=1$ to $\Delta=S$ ) to prevent semantic collapse during discrete transitions, weakening the model's bias toward smooth motion.
- Null-Instruction Training: Trains the model to reconstruct the source image given a null instruction, enforcing identity mapping and preserving regions unrelated to the edit.
Stage 2: Causal Multi-Turn Autoregressive Training
- Converts generation to a causal formulation using causal attention masks.
- Self-Rollout Strategy: Randomly replaces ground-truth historical frames with the model's own previous predictions during training to mitigate exposure bias and bridge the train-inference gap.
- Synthetic Degradation Injection: Applies non-Gaussian degradations (e.g., JPEG compression, color shifts, motion blur) to historical frames to simulate inference-time drift.
- Temporally-Progressive Objective: Scales task difficulty, noise intensity, and loss weights ( $\lambda_i$ ) progressively across turns, forcing the model to learn robust error-correction capabilities for later steps in the sequence.
Stage 3: Distribution Matching Distillation
- Compresses the sampling process into an efficient 4-step generator.
- Utilizes Causal Consistency Distillation to learn a consistent flow map from the teacher model, followed by Distribution Matching Distillation (DMD) combined with flow-matching loss to align the student's output with the high-fidelity data distribution.

2. Long-Chain Streaming Inference Pipeline

Memory Mechanism: Implements a selective KV cache where the first frame (source image) acts as a persistent attention sink (never evicted) to anchor global subject identity. A sliding window maintains local temporal coherence for recent edits.
Strided RoPE Indexing: To enable extrapolation beyond the training sequence length, the model assigns RoPE indices based on the frame's role rather than sequential order. The sink frame is fixed at index 0, while local frames use a fixed stride $s$ . This preserves the "semantic leap" distance learned in Stage 1, preventing identity collapse even when historical frames are dropped.

Key Contributions

First High-Resolution AR Diffusion Framework: AnchorEdit is the first autoregressive diffusion model specifically designed for high-resolution, long-term multi-turn image editing.
Robust Forcing and Memory Mechanisms: The paper introduces self-rollout forcing and historical memory mechanisms (persistent sink + sliding window) to the multi-turn editing paradigm. These strategies effectively mitigate exposure bias and cumulative errors, ensuring strict identity consistency over 10+ editing rounds.
Comprehensive Benchmark: The authors introduce a new high-resolution multi-turn editing benchmark (1024p) with diverse sequence depths, designed to stress-test model stability. This addresses the lack of standardized evaluation tools for long-horizon editing.

Results

Extensive experiments demonstrate that AnchorEdit achieves state-of-the-art results:

Quantitative Performance: On the custom multi-turn benchmark, AnchorEdit achieves a 52.9% Success Rate (SR), significantly outperforming baselines like JoyImage-Edit (29.8%) and VINCIE (34.1%). It also maintains high Consistency (6.2) and Semantic Following (5.3) scores.
Long-Horizon Stability: The model maintains exceptional subject fidelity and instruction adherence even over 10+ interaction rounds, whereas existing methods suffer from identity drift and visual degradation after only a few turns.
Efficiency: The distilled 4-step inference delivers these results with significantly higher efficiency than full-diffusion baselines.
Ablation Studies: Confirm that Expanded RoPE, Self-Rollout, and Progressive Degradation are critical components for preventing identity collapse and handling error accumulation.

Significance

The paper claims that AnchorEdit establishes a reliable and scalable foundation for long-horizon interactive visual creation. By repurposing autoregressive video diffusion models and aligning them with the causal structure of editing, the framework demonstrates that it is possible to maintain exceptional subject fidelity and semantic consistency across extended editing trajectories. This work addresses the critical bottleneck of error accumulation in iterative design, moving beyond single-turn capabilities toward practical, application-level solutions for complex content creation.