💻 computer science

EditCaption: Human-Aligned Instruction Synthesis for Image Editing via Supervised Fine-Tuning and Direct Preference Optimization

The paper proposes EditCaption, a two-stage post-training pipeline combining supervised fine-tuning and direct preference optimization to address systematic failure modes in vision-language models, thereby generating high-quality, human-aligned instruction triplets that significantly improve image editing performance compared to state-of-the-art baselines.

Original authors: Xiangyuan Wang, Honghao Cai, Yunhao Bai, Tianze Zhou, Haohua Chen, Yao Hu, Xu Tang, Yibo Chen, Wei Zhu

Published 2026-04-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiangyuan Wang, Honghao Cai, Yunhao Bai, Tianze Zhou, Haohua Chen, Yao Hu, Xu Tang, Yibo Chen, Wei Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot artist how to paint. You don't just show the robot a "before" picture and an "after" picture; you have to give it a recipe (an instruction) on exactly how to turn the first picture into the second.

For example, if the "before" picture shows a dog in a park, and the "after" picture shows the same dog wearing a hat on the left side, the recipe needs to say: "Put a hat on the dog's left ear."

The problem is that when we ask big, smart AI computers (called Vision-Language Models) to write these recipes automatically, they often get confused. They might say, "Put a hat on the dog's right ear," or they might forget to mention the hat is red, or they might get the angle of the photo wrong. If the robot artist follows a bad recipe, the painting turns out wrong.

This paper, titled EditCaption, is about fixing the robot's ability to write these recipes so they are perfect.

The Three Big Mistakes

The authors found that when AI tries to describe the difference between two photos, it makes three specific types of "hallucinations" (mistakes):

The "Left/Right" Mix-up: The AI gets its directions backward. It thinks "left" is "right."
The "Camera Angle" Confusion: The AI doesn't understand if the camera moved closer or if the person turned around. It gets the perspective wrong.
The "Vague Description" Problem: The AI says "change the color" instead of "change the shirt from blue to red." It misses the tiny, important details.

In fact, the authors tested this and found that nearly half (47%) of the recipes written by the best AI models were so broken that a human couldn't use them to train a robot artist.

The Solution: A Two-Step Training Camp

To fix this, the authors created a new system called EditCaption. Think of it as a two-step training camp for the AI, designed to turn it from a confused student into a master editor.

Step 1: The "Homework" Phase (Supervised Fine-Tuning)

First, they gathered 100,000 pairs of photos. They asked a smart AI to write the initial recipes, but then they used a "quality checker" (a tool called EditScore) to filter out the bad ones.

Then, the most important part: Humans stepped in. Real people looked at the remaining 100,000 recipes and fixed them. They corrected the left/right confusion, clarified the camera angles, and added the missing details (like "red hat" instead of just "hat"). The AI studied these 100,000 perfect recipes like a student studying a textbook. This is called Supervised Fine-Tuning (SFT).

Step 2: The "Debate" Phase (Direct Preference Optimization)

Even after studying the textbook, the AI still made some subtle mistakes. So, the authors set up a "debate."

They showed the AI a photo pair and asked it to write a recipe. The AI wrote a "good but flawed" recipe (e.g., "Put a hat on the right"). Then, a human wrote the perfect recipe (e.g., "Put a hat on the left").

The AI was then shown both versions and told: "The human version is the winner. You need to learn why the human version is better and stop making the mistake you just made." This process, called Direct Preference Optimization (DPO), teaches the AI to avoid its specific bad habits by comparing its own mistakes against human perfection.

The Results: Beating the Giants

The authors tested their new AI (based on a model called Qwen3-VL) against other famous AI models, including some very expensive, closed-source giants like Gemini and GPT-4.

The Score: Their new AI scored 4.712 out of 5 on a difficult test, beating the previous leaders.
The Human Test: Before their training, the AI's instructions were useless 47% of the time. After their two-step training, the "useless" rate dropped to just 23%, and the "correct" rate jumped to 66%.

The Big Picture

Why does this matter?
Building AI that can edit photos is hard because you need millions of "Before/After" examples with perfect instructions. Hiring humans to write millions of instructions is too expensive and slow.

EditCaption provides a scalable way to generate these instructions automatically. It's like building a machine that can write perfect cooking recipes for a robot chef. Once the robot chef learns from these perfect recipes, it can create amazing new images for us, following our commands accurately without getting confused about left, right, or colors.

In short: They found the AI was bad at giving directions, built a system to teach it the right way using human corrections, and proved that this new teacher makes the AI smarter than almost any other model out there.

1. Problem Statement

The paper addresses a critical bottleneck in scaling instruction-guided image editing: the lack of high-quality training triplets (source image, target image, and precise editing instruction). While Vision-Language Models (VLMs) are often used to automate the synthesis of these instructions, the authors identify three systematic failure modes in current VLM outputs when applied to image-pair reasoning:

Orientation Inconsistency: Models frequently confuse directional relationships (e.g., describing a leftward shift as rightward).
Viewpoint Ambiguity: Models fail to accurately characterize perspective shifts or camera motion changes between source and target images.
Lack of Fine-Grained Detail: Descriptions are often generic, omitting critical attribute-level changes (color, texture, specific object modifications).

Human evaluation reveals that over 47% of instructions generated by strong baseline VLMs contain critical errors (P0) that render them unusable for training downstream editing models. The root cause is that VLMs are trained to maximize token likelihood for fluent language, making them insensitive to the precise relational and spatial structures encoded in image pairs.

2. Methodology: EditCaption Pipeline

The authors propose EditCaption, a scalable, two-stage post-training pipeline designed to align VLMs with human expectations for spatial and attribute accuracy. The pipeline utilizes Qwen3-VL models.

Stage 1: Supervised Fine-Tuning (SFT)

Data Construction: A dataset of 100K image pairs is constructed.
- Initial Generation: GLM models generate initial captions.
- Quality Filtering: An open-source reward model, EditScore, filters out misaligned or ambiguous instructions.
- Human Refinement: Human annotators correct the remaining samples to ensure semantic accuracy, spatial clarity (resolving left/right/relative positions), and fine-grained attribute descriptions.
Objective: Standard cross-entropy loss to instill the model's basic capacity for structured, accurate instruction generation.

Stage 2: Direct Preference Optimization (DPO)

Data Construction: A dataset of 10K preference pairs is created to target the specific failure modes identified in the analysis.
- Rejected Samples ( $y_r$ ): Outputs from the SFT-trained model that exhibit systematic errors (orientation, viewpoint, or detail issues).
- Chosen Samples ( $y_w$ ): Human-corrected instructions that explicitly resolve these errors.
Objective: DPO is applied to align the model with human judgment, directly increasing the likelihood of preferred instructions over failure outputs without requiring an explicit reward model. This stage targets residual systematic errors that SFT alone cannot fix.

3. Key Contributions

Systematic Failure Analysis: The paper provides quantitative evidence (via human evaluation on 400 pairs) identifying orientation inconsistency, viewpoint ambiguity, and lack of fine-grained detail as the dominant error categories in VLM-based instruction synthesis.
Scalable Two-Stage Pipeline: Introduction of a novel data construction and training framework combining 100K human-refined SFT data with 10K human-annotated preference pairs for DPO.
State-of-the-Art Performance: Demonstration that the SFT+DPO combination significantly outperforms existing open-source models and approaches or surpasses closed-source giants (like Gemini-3-Pro and GPT-4.1) on multiple benchmarks.

4. Experimental Results

The method was evaluated on three benchmarks: an in-house Eval-400 set, ByteMorph-Bench (focusing on non-rigid motion and camera transforms), and HQ-Edit (high-resolution, detailed instructions).

Objective Evaluation (Weighted Score):
- The Qwen3-VL-235B (SFT+DPO) model achieved a score of 4.712 on Eval-400, surpassing Gemini-3-Pro (4.706), GPT-4.1 (4.220), and Kimi-K2.5 (4.111).
- On the challenging ByteMorph-Bench, the 235B model scored 4.588, significantly exceeding Gemini-3-Pro (4.522) and GPT-4.1 (3.412).
- The smaller 32B model (SFT+DPO) also showed strong performance (4.386 on Eval-400), outperforming larger base models like Qwen3.5-397B.
Human Evaluation:
- Critical Error Reduction: The rate of critical errors (P0) dropped from 47.75% (base model) to 23% after the two-stage training.
- Correctness Improvement: The instruction correctness rate improved from 41.75% to 66%, matching the performance of Gemini-3-Pro.
Ablation Study:
- DPO alone performed poorly, confirming that a strong generative prior (established by SFT) is necessary before preference optimization can be effective.
- SFT + DPO consistently yielded the best results, proving the complementary nature of the two stages: SFT builds core capability, while DPO corrects residual systematic errors.

5. Significance

This work provides a practical and effective solution for scalable data construction in the field of image editing. By moving beyond simple automated captioning to a human-aligned, error-targeted training pipeline, EditCaption enables the creation of high-quality instruction-image datasets that were previously prohibitively expensive to build manually. The results suggest that with targeted alignment (SFT+DPO), open-source models can rival or exceed proprietary closed-source models in complex spatial reasoning tasks, offering a viable path for training next-generation image editing foundation models.