🤖 AI

Understanding vs. Generation: Navigating Optimization Dilemma in Multimodal Models

This paper addresses the trade-off between generation and understanding in multimodal models by proposing the Reason-Reflect-Refine (R3) framework, which re-frames generation as a multi-step "generate-understand-regenerate" process to simultaneously enhance both capabilities.

Original authors: Sen Ye, Mengde Xu, Shuyang Gu, Di He, Liwei Wang, Han Hu

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sen Ye, Mengde Xu, Shuyang Gu, Di He, Liwei Wang, Han Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot artist how to paint. For a long time, researchers faced a frustrating problem: the better the robot got at painting (generating), the worse it got at understanding what it was painting (comprehending), and vice versa.

It was like a seesaw. If you pushed the robot to become a master painter, it forgot how to count the objects in its own painting. If you trained it to be a sharp critic who could spot errors, it lost its creative spark and couldn't paint well anymore.

This paper, presented at ICLR 2026, introduces a clever new way to train these robots called R3 (Reason-Reflect-Refine). Here is how it works, explained through simple analogies.

The Old Way: The "One-Shot" Artist

Previously, the robot was told: "Here is a description of a cat. Go paint it."
The robot would try to paint it in one giant leap.

The Problem: If the robot made a mistake (like drawing a dog instead of a cat), it wouldn't know. It just handed over the bad painting and moved on. It was like a student taking a test without checking their work. They might get the answer wrong, but they never learned why because they weren't forced to think about the question while answering.

The New Way: The "Painter-Critic" Loop (R3)

The authors realized that to get good at both painting and understanding, the robot needs to do both at the same time. They broke the process down into three steps, like a master painter working in a studio:

1. Reason (The Blueprint)

Instead of just splashing paint on the canvas, the robot first stops and thinks.

Analogy: Imagine an architect drawing a detailed blueprint before building a house. The robot reads your request ("A cat on a red chair") and writes down a plan: "I need a fluffy cat, sitting on a red wooden chair, with sunlight coming from the left."
Why it helps: This forces the robot to understand the request deeply before it even starts painting.

2. Reflect (The Critic)

The robot paints a first draft based on the blueprint. Then, it steps back and critiques its own work.

Analogy: The robot looks at the painting and asks, "Wait, the prompt said 'red chair,' but I painted a blue one. And is that a cat or a dog?"
The Magic: This is the key. By forcing the robot to judge the image, it is actively using its "understanding" brain. It's not just painting; it's analyzing.

3. Refine (The Fix)

If the robot finds a mistake, it doesn't give up. It writes a note to itself: "Change the chair to red and fix the cat's ears," and then repaints that specific part.

Analogy: It's like editing a photo. You don't take a new picture from scratch; you just fix the parts that are wrong.
The robot keeps doing this loop (Paint → Critique → Fix) until the picture is perfect.

Why This Solves the Problem

The paper argues that the old "seesaw" problem happened because the robot was trying to learn two separate skills at once without connecting them.

With R3, the robot learns that you cannot be a good painter unless you are a good critic.

To fix the painting (Generation), it must first understand what is wrong (Understanding).
Because it is constantly practicing its "critic" skills while it paints, it gets better at understanding.
Because it is constantly using its "understanding" to fix the painting, it gets better at painting.

The Results

The researchers tested this on a robot named BAGEL.

Before R3: The robot was okay at painting but bad at counting objects, or good at counting but bad at painting.
After R3: The robot became a super-artist. It painted better images and simultaneously got much better at understanding complex details (like counting how many cats were in the picture).

The Bottom Line

Think of it like learning to drive.

Old Method: You just drive the car. If you crash, you try again. You never really learn the rules of the road.
R3 Method: You drive, then you stop and ask, "Did I follow the speed limit? Did I check my blind spot?" Then you drive again, correcting your mistakes.

By making the robot think, check, and fix its own work, the paper shows we can finally build AI that is both a creative genius and a sharp thinker, without having to choose between the two.

1. Problem Statement

The paper addresses a fundamental trade-off in current multimodal large models (MLLMs): enhancing generative capabilities often degrades understanding capabilities, and vice versa.

The Dilemma: Models fine-tuned for high-fidelity image synthesis (e.g., diffusion-based) often fail at precise visual understanding tasks (counting, spatial reasoning). Conversely, models optimized for Visual Question Answering (VQA) or dense captioning exhibit weak creative generation.
Root Cause: The authors argue that this conflict arises because traditional approaches treat generation and understanding as independent, competing objectives. They compete for the same model capacity, leading to a zero-sum game where optimizing one objective monopolizes resources at the expense of the other.
Limitation of Existing Solutions: Previous attempts to resolve this via specialized tokenizers or disentangled architectures have only achieved partial success. Naive co-training on mixed data yields negligible gains because the optimization trajectories for generation and understanding are not inherently aligned.

2. Methodology: The Reason-Reflect-Refine (R3) Framework

The authors propose the Reason-Reflect-Refine (R3) framework, which re-conceptualizes image generation not as a single-shot mapping, but as a multi-step, iterative process that explicitly integrates understanding into the generative loop.

Core Workflow

The framework operates on a unified multimodal model (built upon BAGEL) and decomposes generation into three alternating stages:

Reason: The model analyzes the user prompt ( $c$ ) to generate a detailed reasoning plan ( $t_1$ ) and an initial image ( $I_1$ ). This acts as a "blueprint" phase.
Reflect: The model evaluates the generated image ( $I_1$ $I_{1}$ ) against the original prompt ( $c$ $c$ ). This is a critical understanding task where the model assesses alignment.
- If aligned: The model outputs a termination signal ("No further edit needed").
- If misaligned: The model generates a specific textual editing instruction ( $e_{i+1}$ ) identifying discrepancies.
Refine: The model executes the editing instruction to modify the image ( $I_i \to I_{i+1}$ ).
This loop repeats until the model self-determines the output is satisfactory.

Training Strategy: Tree-RL with GRPO

To train this complex, multi-turn trajectory, the authors employ a Tree-based Reinforcement Learning (RL) strategy using Group Relative Policy Optimization (GRPO).

Tree Structure: Instead of training end-to-end on long, noisy trajectories, the process is split into a "Reason" stage and a "Reflect-Refine" stage. The output of the Reason stage serves as the initial condition for the Reflect-Refine stage.
Stage-Wise Rewards:
- Reason Stage: Rewards are based on the quality of the initial image ( $V(I_1, c)$ ) and the correctness of the reasoning format.
- Reflect-Refine Stage: A specific metric $C_j$ is used. It rewards measurable improvement (if the new image score $V_j$ is higher than the previous $\hat{V}$ ) or correct termination (if the image was already perfect and the model correctly stops).
Optimization: The text generation (Reasoning/Reflection) uses standard GRPO, while the image generation (Diffusion/Refinement) uses FlowGRPO (adapting GRPO for flow-matching/diffusion models).
Efficiency: An "immediate rollout" strategy and importance sampling are used to accelerate convergence without compromising performance.

3. Key Contributions

Systematic Analysis: Identified that the conflict between generation and understanding stems from separate optimization objectives competing for model capacity.
R3 Framework: Proposed a novel paradigm where understanding is an active component of generation. By forcing the model to "reflect" on its own output, the framework creates a synergistic relationship where improving generation inherently requires and strengthens understanding.
Tree-RL Strategy: Developed a training methodology that stabilizes long-horizon generation tasks by providing clear, stage-wise supervision, overcoming the instability of full-trajectory RL.
Co-Evolution: Demonstrated that integrating reflection into the generation pipeline allows the model to improve generation performance without degrading (and often while improving) its understanding capabilities.

4. Experimental Results

The framework was evaluated on GenEval, GenEval++, and TIIF benchmarks, using BAGEL as the baseline.

Generation Performance:
- On GenEval++, R3 achieved an overall score of 0.689, significantly outperforming the baseline BAGEL (0.371) and the SOTA Echo-4o (0.679).
- Notable improvements were seen in complex tasks like Multi-Count (0.800 vs. 0.625 for Echo-4o).
Understanding Performance (The "Co-Evolution" Effect):
- Crucially, the model's understanding abilities improved alongside generation.
- VQA Accuracy: Improved from 79.3% (BAGEL) to 84.6% (R3) on the counting subset.
- Image-Text Alignment (ITA): Improved from 60.6% to 73.4%.
- Ablation: A version with only the "Reason" stage showed minor gains, proving that the Reflect-Refine loop is the key driver for unlocking understanding capabilities.
Generalization: The framework showed strong transferability to the TIIF benchmark (general domain), achieving an overall score of 82.02 compared to BAGEL's 70.97.
Efficiency: The model learns to self-terminate. On GenEval++, 45% of prompts required zero refinement turns, and 90% required 2 or fewer, demonstrating adaptive computational efficiency.

5. Significance and Impact

Resolving the Trade-off: R3 provides a concrete path to breaking the "generation vs. understanding" trade-off. It proves that these capabilities do not need to be mutually exclusive if the training objective forces them to collaborate.
New Paradigm for Unified Models: The paper suggests that future unified multimodal models should not just be trained on mixed data but should be architected to treat understanding as a sub-task of generation.
Scalability: The Tree-RL approach offers a scalable method for training complex, multi-step generative agents, applicable beyond text-to-image (e.g., demonstrated in the appendix on Maze Navigation).
Future Direction: The work points toward "next-generation" unified models where the model actively critiques and refines its own outputs, mimicking human creative processes (planning, drafting, editing) to achieve higher fidelity and semantic alignment.