One Model, Two Minds: Task-Conditioned Reasoning for Unified Image Quality and Aesthetic Assessment

This paper introduces TATAR, a unified framework that addresses the reasoning and optimization mismatches between Image Quality and Aesthetic Assessment by employing task-conditioned post-training with distinct reasoning styles and asymmetric reward strategies, thereby outperforming existing unified and specialized models across multiple benchmarks.

The Gist

The Big Idea: One Brain, Two Different Hats

Imagine you have a super-smart robot assistant. You want this robot to do two very different jobs:

1. The Inspector: Look at a photo and tell you if it's broken, blurry, or noisy (Image Quality Assessment).
2. The Art Critic: Look at a photo and tell you if it's beautiful, emotional, or well-composed (Image Aesthetic Assessment).

The Problem:
Previously, researchers tried to teach this robot to do both jobs using the exact same "brain settings." They told it to think the same way and gave it the same "grade" for both tasks.

• Why this failed: It's like asking a mechanic to fix a car engine and then immediately asking them to write a poem about the sunset.
  • Fixing the engine (Quality) needs quick, factual, technical answers. "The tire is flat. The oil is low." Long, flowery thinking just wastes time and confuses the diagnosis.
  • Writing the poem (Aesthetics) needs slow, deep, emotional thinking. "The light hits the leaves like gold, evoking a sense of peace." A quick, one-word answer misses the point.

The paper argues that forcing the robot to use the same "thinking style" for both jobs makes it bad at both.

The Solution: TATAR (The "Two Minds" System)

The authors created a new system called TATAR (Task-Aware Thinking with Asymmetric Rewards). Think of it as giving the robot a "magic hat" that changes its personality depending on the job.

Here is how TATAR works in three simple steps:

1. The "Fast vs. Slow" Thinking Switch (Reasoning Construction)

Before the robot starts learning, the researchers teach it two different ways to talk:

• For the Inspector (Quality): They teach it to be Fast. It learns to spot "bugs" (blur, noise, compression) quickly and give a short, punchy report. Analogy: Like a security guard checking a bag for prohibited items—quick, specific, no fluff.
• For the Art Critic (Aesthetics): They teach it to be Slow. It learns to take a deep breath, look at the colors, the story, and the mood, and write a long, thoughtful paragraph. Analogy: Like a sommelier tasting wine—swirling, sniffing, and describing the notes before giving a rating.

2. The Two-Stage Training (SFT + GRPO)

The robot doesn't learn everything at once. It goes through two school grades:

• Grade 1 (SFT - Supervised Fine-Tuning): The robot practices the "Fast" and "Slow" styles. It learns how to format its answers so it doesn't get confused. It learns to wear the "Inspector Hat" for one task and the "Critic Hat" for the other.
• Grade 2 (GRPO - Reinforcement Learning): Now that the robot knows how to think, it learns how to score accurately. This is where the magic happens.

3. The "Asymmetric Rewards" (The Grading System)

This is the most important part. In the old days, the robot got the same type of grade for both jobs. TATAR changes the grading rules:

• For the Inspector: The grade is based on Precision. Did you get the exact number right? (e.g., Is the blur score 4.2 or 4.3?). The reward is a smooth curve that punishes small errors gently but heavily punishes big mistakes.
• For the Art Critic: The grade is based on Ranking. Since beauty is subjective, getting the "exact" number is hard. Instead, the robot is rewarded for getting the order right. If it says Image A is more beautiful than Image B, and humans agree, it gets a reward. It's like judging a beauty pageant: you don't need to know the exact score of every contestant, you just need to know who is #1, #2, and #3.

Why This Matters (The Results)

The researchers tested TATAR on 8 different datasets (like a giant test bank of photos).

• The Result: TATAR beat all previous "unified" models (the ones that tried to do both jobs with one brain).
• The Surprise: It didn't just beat the other unified models; it was almost as good as the specialized models that only do one job.
  • It became a master Inspector and a master Art Critic simultaneously.
  • It stopped the "mode collapse" where the robot would get lazy and give short answers for the Art Critic job (which used to happen when it was forced to use the same thinking style for everything).

Summary Analogy

Imagine a restaurant.

• Old Way: You hire one chef and tell them to cook a perfect steak (technical) and a perfect soufflé (artistic) using the exact same recipe and oven settings. The steak comes out dry, and the soufflé falls flat.
• TATAR Way: You hire one chef, but you give them two different cookbooks and two different tasting menus.
  • When making the steak, they follow the "Technical Cookbook" (precise timing, temperature) and get graded on exact doneness.
  • When making the soufflé, they follow the "Artistic Cookbook" (fluffiness, presentation) and get graded on how much the judges love it compared to others.

By letting the same chef switch between these two distinct mindsets, TATAR creates a system that is both technically precise and artistically sensitive.

Technical Summary

1. Problem Statement

The paper addresses the challenge of unifying Image Quality Assessment (IQA) and Image Aesthetic Assessment (IAA) within a single Multimodal Large Language Model (MLLM). While existing unified approaches share a backbone and training strategy, the authors argue they suffer from two fundamental mismatches:

• Reasoning Mismatch:
  • IQA relies on low-level, objective perceptual cues (e.g., blur, noise, compression). It benefits from concise, fast reasoning focused on detecting distortions.
  • IAA relies on high-level semantics, composition, and lighting. It requires deliberative, slow reasoning to integrate multiple aesthetic factors into a holistic judgment.
  • Current Issue: Applying a uniform Chain-of-Thought (CoT) strategy to both tasks leads to suboptimal performance (e.g., unnecessary deliberation for IQA or insufficient depth for IAA).
• Optimization Mismatch:
  • IQA is an absolute scoring task where point-wise regression against a Mean Opinion Score (MOS) is natural.
  • IAA is subjective and preference-based. Treating it as a strict point-wise regression target leads to unstable optimization and poor alignment with human relative judgments.
  • Current Issue: Using a single shared reward function (e.g., MSE or a generic reinforcement reward) fails to capture the distinct supervision geometries of the two tasks.

2. Methodology: TATAR Framework

The authors propose TATAR (Task-Aware Thinking with Asymmetric Rewards), a unified framework that shares a visual-language backbone but conditions the post-training process on the specific nature of each task.

A. Fast–Slow Reasoning Construction (Data Synthesis)

To teach the model distinct response modes, the authors construct the QACoT-score dataset:

• IQA (Fast Mode): Uses score-conditioned reverse inference. A teacher model generates short, distortion-focused rationales based on image-score pairs (e.g., from KonIQ-10k).
• IAA (Slow Mode): Uses structured aesthetic summarization. A teacher model synthesizes long, integrative narratives from rich attribute-level annotations (e.g., from ArtiMuse), covering composition, emotion, and originality.
• Filtering: A unified judge model filters synthetic data to ensure format compliance, score validity, and reasoning quality.

B. Two-Stage Task-Conditioned Learning

The training pipeline consists of two distinct stages:

1. Stage 1: Format-Oriented Supervised Fine-Tuning (SFT):
   • The model is fine-tuned on the mixed QACoT dataset.
   • Goal: Establish behavioral priors. The model learns to adopt "fast" (concise) reasoning for IQA and "slow" (deliberative) reasoning for IAA, preventing mode collapse during reinforcement learning.
2. Stage 2: Task-Conditioned Reinforcement Fine-Tuning (RFT):
   • Uses Group Relative Policy Optimization (GRPO).
   • Goal: Refine scoring accuracy. The model samples multiple completions, and the policy is updated based on relative advantages within the group, regularized against the Stage 1 reference model.

C. Asymmetric Reward Design

Instead of a single reward, TATAR applies task-specific rewards while sharing the optimizer:

• Shared Format Reward: Ensures valid output structure (tags and numeric scores) for both tasks.
• IQA Reward (Gaussian Score Shaping): Uses a smooth Gaussian function based on the absolute error between the predicted score and the ground-truth MOS. This provides dense feedback for absolute fidelity.
• IAA Reward (Thurstone-Style Ranking): Since IAA is preference-based, the reward is derived from a pairwise ranking loss. It treats the predicted scores as a distribution and optimizes the model to preserve the relative order of samples (Thurstone model), weighted by the ambiguity of the score difference. This stabilizes training on subjective data.

3. Key Contributions

1. Diagnosis of Mismatches: The paper empirically demonstrates that unified IQA-IAA models fail due to a reasoning mismatch (uniform CoT hurts IQA) and an optimization mismatch (shared regression hurts IAA).
2. TATAR Framework: A novel unified architecture that decouples the shared backbone from task-specific post-training, utilizing Fast–Slow reasoning construction and Asymmetric Rewards.
3. State-of-the-Art Performance: Extensive experiments show TATAR outperforms prior unified baselines and specialized models across 8 benchmarks.

4. Experimental Results

The model was evaluated on 8 benchmarks (4 for IQA, 4 for IAA) under in-domain and cross-domain settings.

• Performance: TATAR achieves the best average performance on both tasks.
  • IAA: 51.0/52.5 (SRCC/PLCC) average, outperforming the previous unified baseline (UniPercept) by +6.2/+7.4 points.
  • IQA: 76.7/76.5 (SRCC/PLCC) average, outperforming UniPercept by +4.9/+4.0 points.
• Comparison:
  • Vs. Proprietary Models: TATAR significantly outperforms GPT-4o and other commercial MLLMs, which often suffer from score-range misalignment in zero-shot settings.
  • Vs. Specialized Models: TATAR matches the in-domain performance of specialized IQA models (e.g., DeQA) while simultaneously handling IAA, a capability specialized models lack.
• Ablation Studies:
  • Two-Stage Training: SFT is critical; RL-only training causes IQA completion lengths to collapse and IAA format rewards to remain low.
  • Reward Design: Using the task-conditioned asymmetric rewards (Gaussian for IQA, Ranking for IAA) yields the highest gains compared to using a single reward type for both.

5. Significance and Conclusion

The paper establishes task-conditioned post-training as a principled paradigm for unified perceptual scoring. It demonstrates that "one size fits all" objectives are insufficient for tasks with fundamentally different decision regimes. By explicitly aligning reasoning modes (Fast vs. Slow) and optimization signals (Absolute vs. Relative) to the specific task, TATAR achieves superior stability and accuracy.

Key Takeaway: Effective unified perception requires sharing the representation (the visual backbone) but adapting the reasoning and optimization to the specific nature of the task. This approach bridges the gap between objective quality assessment and subjective aesthetic judgment within a single MLLM.