RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Reward

RetouchIQ is a framework that leverages MLLM agents guided by a novel generalist reward model to perform instruction-based image retouching, overcoming the challenge of subjective evaluation in reinforcement learning to achieve superior semantic consistency and perceptual quality compared to existing systems.

The Gist

Imagine you have a beautiful photograph, but it feels a little "flat." You want to tell your computer, "Make this look like a dreamy, golden sunset," or "Give this a moody, cinematic blue vibe."

In the past, asking a computer to do this was like trying to teach a toddler to drive a Formula 1 car. You could give the instruction, but the computer either didn't understand the nuance, or it tried to follow a rigid rulebook (like "increase brightness by 10%") that often ruined the photo.

RETOUCHIQ is a new system that changes the game. Think of it as hiring a super-smart, artistic digital assistant who doesn't just guess, but actually thinks before they touch the sliders.

Here is how it works, broken down into simple concepts:

1. The Problem: The "One Right Answer" Trap

Imagine you ask a human editor, "Make this photo feel warmer."

• Editor A might add a little orange.
• Editor B might boost the reds and lower the shadows.
• Editor C might add a golden glow.

All three are correct! They are all "warm."

Old computer systems tried to learn by comparing their work to a single "perfect" example. If the computer tried to be like Editor A but the training data said Editor B was the "right" one, the computer got confused and failed. It was like trying to learn to paint by only being allowed to copy one specific shade of blue, even though the sky can be blue in a thousand different ways.

2. The Solution: The "Art Critic" (The Generalist Reward Model)

RETOUCHIQ introduces a special component called the Generalist Reward Model. Think of this as a tough but fair Art Critic sitting next to the editor.

Instead of checking if the photo matches a specific "correct" file, the Critic looks at the photo and the user's request and asks:

• "Does this actually feel warm?"
• "Are the colors balanced?"
• "Does it look like a sunset or just a mess?"

The Critic doesn't just give a score (like "7 out of 10"). It writes a little report explaining why it gave that score. "The warmth is good, but the shadows are too heavy."

3. The Learning Loop: Trial, Error, and Coaching

Here is the magic sauce of how RETOUCHIQ learns:

1. The Editor (Policy Model): The AI tries to edit the photo based on your instruction. It makes a guess.
2. The Critic (Reward Model): The Critic looks at the result. It generates a custom checklist of what makes a "good" edit for this specific photo and gives a score.
3. The Coaching: If the score is low, the Critic says, "Hey, you made it too bright, and the colors look muddy." The Editor learns from this feedback and tries again.

This happens thousands of times. The Editor gets better and better at understanding not just the words, but the feeling you want.

4. The Secret Sauce: "Policy-Guided Training"

There was a tricky problem. The "Art Critic" was being trained on photos that were edited by a robot that just randomly messed with sliders (like turning the brightness up and down randomly). But the "Editor" AI was learning to make complex, artistic edits.

It was like training a race car driver by having them practice on a dirt track with potholes, but then sending them onto a real racetrack. They didn't match!

The researchers fixed this with a method called Policy-Guided Reward Training (PGRT).

• Instead of training the Critic on random, messy edits, they trained it on the actual edits the Editor AI was making.
• This way, the Critic learns to judge the specific, complex style of the Editor. They learn to speak the same language.

Why This Matters

Before RETOUCHIQ, if you wanted to edit a photo professionally, you needed to know what "Exposure," "Contrast," and "Temperature" meant. You had to be the expert.

With RETOUCHIQ:

• You are the Director: You just say, "Make it feel like a cozy winter morning."
• The AI is the Cinematographer: It figures out the technical settings to make that happen.
• The Result: A photo that looks professional, matches your mood, and keeps the original details intact (unlike other AI tools that sometimes turn people into aliens or change the background).

In short: RETOUCHIQ is like having a professional photo editor in your pocket who listens to your vague ideas, thinks deeply about how to achieve them, and learns from a smart critic to get it right every time.

Technical Summary

1. Problem Statement

The paper addresses the challenge of enabling Multimodal Large Language Models (MLLMs) to perform professional, instruction-based image retouching using external tools (e.g., Adobe Lightroom). While MLLMs have shown potential in reasoning and tool use, applying them to image editing faces two critical hurdles:

• Subjectivity of Aesthetics: Unlike objective tasks (e.g., math or coding), image retouching is inherently subjective. Multiple distinct edits can satisfy a single user instruction (e.g., "make it moody"). Consequently, traditional verifiable rewards (which compare the output to a single ground-truth reference image using pixel-level metrics like L1/L2) are unreliable and often penalize valid, creative edits.
• Distribution Shift in Training: Existing methods often generate "weak" training data by randomly perturbing parameters. However, these synthetic weak edits often differ significantly in distribution from the complex, combined edits generated by the policy model during Reinforcement Learning (RL), leading to sub-optimal reward signals.

2. Methodology: RETOUCHIQ Framework

RETOUCHIQ is an MLLM agent framework designed to translate natural language instructions into executable, parameterized editing steps within professional software. The system operates through a two-stage training pipeline:

A. Policy Model Training

The policy model is trained to output two components:

1. Reasoning Trace: A textual explanation of the aesthetic intent and the strategy to achieve it.
2. Editing Configs: A structured sequence of tool parameters (e.g., Exposure: +0.9, Temperature: 5695).

Training Stages:

1. Supervised Fine-Tuning (SFT): The model is trained on a curated dataset of 190k instruction-reasoning-edit triples derived from real human user trajectories. This teaches the model to map linguistic goals to specific tool parameters.
2. Reinforcement Learning (RL): The model explores diverse reasoning paths and editing strategies to maximize a reward signal, moving beyond simple imitation of the training data.

B. The Generalist Reward Model (GRM)

To overcome the limitations of fixed reference-based rewards, the authors introduce a Generalist Reward Model, an RL-fine-tuned MLLM that evaluates edits dynamically.

• Mechanism: Instead of comparing pixels, the GRM first generates a set of context-specific metrics (e.g., "The sky should have a teal-blue tone," "Shadows should be softened") based on the user instruction and the image content. It then scores the edited image against these generated metrics.
• Data Construction: The GRM is trained on pairwise comparisons of "Strong" (user-edited) vs. "Weak" (sub-optimal) images.

C. Policy-Guided Reward Training (PGRT)

A key innovation is the PGRT scheme, which addresses the distribution shift between synthetic weak edits and real policy outputs.

• Problem: Standard RL training uses "perturbed" images (randomly adjusted parameters) as negative examples. These often look different from the complex errors a policy model actually makes.
• Solution: PGRT aligns the reward model's training distribution with the policy model's distribution. During the RL phase, the "Weak" images used to train the reward model are generated by the policy model itself rather than by random perturbation.
• Process: The policy and reward models are optimized in an alternating training strategy. The policy generates edits; the reward model (trained on the policy's own outputs) provides feedback; the policy updates based on this feedback.

3. Key Contributions

1. RETOUCHIQ Framework: The first framework to address the subjective nature of image retouching using an MLLM agent guided by a Generalist Reward Model, enabling flexible, context-aware supervision.
2. Generalist Reward Model (GRM): A novel reward mechanism that generates dynamic evaluation metrics for each case rather than relying on static, rule-based pixel comparisons.
3. Policy-Guided Reward Training (PGRT): A new RL paradigm that eliminates the distribution gap between training data and policy outputs by using the policy's own generated "weak" edits to refine the reward model, resulting in more stable and accurate learning.
4. RETOUCHEVAL Benchmark: The introduction of a new benchmark consisting of 300 instruction-image pairs covering quality enhancement, style transformation, and local retouching, filling a gap in the evaluation of professional tool-based editing.

4. Experimental Results

The authors evaluated RETOUCHIQ against state-of-the-art baselines, including general-purpose MLLMs (GPT-5, Gemini-2.5), MLLM agents (JarvisArt, MonetGPT), and diffusion-based models (Flux-Pro).

• Quantitative Performance: RETOUCHIQ significantly outperformed all baselines on the RETOUCHEVAL and MIT-Adobe5K benchmarks.
  • It achieved the highest scores in Perceptual Quality (PQ) and Semantic Consistency (SC).
  • It demonstrated superior performance in Style Transformation and Local Retouching, where diffusion models often fail to preserve image identity and MLLM agents struggle with complex parameter reasoning.
• Ablation Studies:
  • RL vs. SFT: The RL stage with the Generalist Reward consistently improved performance over the SFT-only baseline.
  • PGRT Effectiveness: Models trained with PGRT showed higher accuracy in distinguishing good vs. bad edits and produced higher-quality policy outputs compared to those trained with standard perturbed data.
• Qualitative Results: Visual comparisons showed RETOUCHIQ produces edits that are more aesthetically pleasing, better aligned with user intent (e.g., "warm vintage feel," "cinematic glow"), and less prone to artifacts or identity distortion compared to diffusion models.

5. Significance

• Bridging Subjectivity and Automation: RETOUCHIQ demonstrates that AI can handle the inherently subjective domain of artistic photo editing by moving away from rigid ground-truth comparisons toward dynamic, reasoning-based evaluation.
• Explainable Tool Use: Unlike "black-box" diffusion models that generate pixels directly, RETOUCHIQ provides interpretable reasoning and executable parameters, making it a transparent assistant for professional workflows.
• New Paradigm for RL in Creative Tasks: The introduction of PGRT offers a blueprint for applying Reinforcement Learning to other creative domains where "correctness" is ambiguous and training data distributions are difficult to simulate.
• Professional Integration: By interfacing directly with tools like Adobe Lightroom, the system offers a practical path toward AI-assisted professional photography, rather than just generating new images from scratch.