💻 computer science

Understanding Pure Textual Reasoning for Blind Image Quality Assessment

This paper investigates the role of textual reasoning in Blind Image Quality Assessment by comparing Chain-of-Thought, Self-Consistency, and Autoencoder paradigms, revealing that while existing models rely heavily on visual data, the Self-Consistency approach significantly narrows the performance gap between image- and text-only predictions.

Original authors: Yuan Li, Shin'ya Nishida

Published 2026-03-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuan Li, Shin'ya Nishida

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a food critic trying to rate a dish. You have two ways to do it:

The Visual Approach: You look at the plate, smell the aroma, and take a bite. You give it a score based on what your senses tell you.
The Descriptive Approach: You write a long, detailed paragraph about the dish ("The sauce is rich," "The texture is crunchy"), and then, based only on reading that paragraph, you give it a score.

For a long time, AI models that rate image quality (called Blind Image Quality Assessment) have been like the Visual Approach. They look at a photo and give it a number (e.g., "7 out of 10"). But recently, researchers started using "Multimodal Large Language Models" (super-smart AI that can see and talk). These new models try to do both: they look at the photo, write a description, and then give a score.

The big question this paper asks is: Is the description actually helping the AI decide the score, or is the AI just ignoring the words and guessing the score based on the picture anyway?

Here is a simple breakdown of what the researchers found and the three "training methods" they tested to fix this.

The Problem: The "Ghost Writer"

The researchers discovered that in many existing AI models, the text description is like a ghost writer. The AI writes a nice paragraph about the photo ("This image is clear and vibrant!"), but when it comes time to give the actual score, it ignores the paragraph completely and just looks at the image again. The text is just a fancy explanation added after the fact, not a real part of the thinking process.

To fix this, the team tried three different ways to train the AI to make the text actually matter.

The Three Training Methods

1. Chain-of-Thought (The "Step-by-Step" Student)

The Analogy: Imagine a student who is told: "First, describe the picture. Then, read your description and guess the score."
What Happened: The AI wrote the description, but when it tried to guess the score from the text alone, it failed miserably. It was like a student who wrote a great essay but couldn't answer a question based on their own essay.
The Result: This method didn't really help. The AI still relied too much on the picture and didn't learn to "think" through the words.

2. Self-Consistency (The "Echo Chamber" Coach)

The Analogy: Imagine a coach who says: "Look at the picture and write a description and a score. Then, pretend you are a different person who only sees your description. Can you guess the score again? If your two guesses match, you get a reward."
What Happened: This was the big winner. By forcing the AI to check if its description could stand on its own, it learned to write descriptions that actually contained the "secret sauce" needed to determine the score.
The Result: The gap between "looking at the picture" and "reading the text" almost disappeared. The AI learned that if it writes "blurry," the score must be low, and it learned to rely on those words.

3. Autoencoder (The "Reverse Engineer")

The Analogy: Imagine a detective who is given the final verdict (the score) first, and then asked to write a report explaining why that verdict was reached. Later, the detective has to read their own report and guess the verdict again.
What Happened: This forced the AI to put the "score" into the "words" during the writing phase. It learned to encode the quality directly into the language.
The Result: This was helpful, but not quite as good as the "Echo Chamber" method. However, it taught the AI to use more natural words like "blurry" or "focus" rather than just generic praise.

The "Magic Words" Test

The researchers did one final test to see if the AI was cheating. They took the descriptions and removed all the obvious "score words" like "good," "bad," "average," or "poor."

Old AI: If you took away "good," the AI couldn't guess the score anymore. It was just memorizing the word "good."
New AI (Self-Consistency): Even without the words "good" or "bad," the AI could still guess the score correctly! This proved it had learned to understand why an image was good (e.g., "the lighting is balanced," "the edges are sharp") rather than just memorizing a keyword.

The Big Takeaway

This paper shows that for AI to truly "understand" image quality, it can't just look at the picture and talk about it separately. It needs to be trained so that the words it writes are the actual reason it gives the score.

The Self-Consistency method is the key: it forces the AI to be honest. It makes the AI realize that if it can't explain the score using only words, it doesn't actually understand the image yet. This makes the AI more reliable, interpretable, and smarter.

1. Problem Statement

Blind Image Quality Assessment (BIQA) traditionally relies on mapping visual features directly to quality scores, often lacking interpretability. Recent approaches leverage Multimodal Large Language Models (MLLMs) to generate textual explanations (captions/reasoning) alongside scores. However, a critical gap exists in understanding how much these generated texts actually contribute to the quality prediction.

Previous works (e.g., Q-Align) have shown that models can achieve state-of-the-art performance by bypassing textual reasoning entirely, suggesting that current MLLM-based BIQA systems might be "cheating" by relying on visual features while generating superficial text explanations. The core questions addressed are:

To what extent can text alone represent image quality information?
How do different training paradigms influence the "Image-Text Gap" (the performance difference between image-conditioned and text-only predictions)?
Do models genuinely engage in textual reasoning, or do they merely produce superficial explanations?

2. Methodology

The authors propose an information-flow perspective to analyze and improve the relationship between images, text, and scores. They introduce and systematically compare three distinct training paradigms using a shared MLLM backbone (Qwen-VL-2.5-7B-Instruct) and Group-Relative Policy Optimization (GRPO) for reinforcement learning.

A. Three Training Paradigms

Chain-of-Thought (CoT):
- Flow: Image $\rightarrow$ Text (Reasoning) $\rightarrow$ Score.
- Mechanism: The model generates $N$ reasoning traces from the image. Each trace then independently produces $M$ score predictions.
- Goal: To test if the generated text can fully encapsulate visual quality information to support score prediction.
Self-Consistency:
- Flow:
  - Stage 1: Image $\rightarrow$ (Text, Score).
  - Stage 2: Text $\rightarrow$ Score (using only the generated text from Stage 1).
- Mechanism: The model is trained to ensure the score predicted from the text alone is consistent with the score predicted from the image. It uses a self-consistency reward to align the text-based reasoning with the visual input.
- Goal: To bridge the gap between image-based and text-based predictions, forcing the model to learn robust textual representations of quality.
Autoencoder-like:
- Flow:
  - Stage 1: (Image + Ground Truth Score) $\rightarrow$ Text (Reasoning).
  - Stage 2: Text $\rightarrow$ Score.
- Mechanism: The model is explicitly given the ground-truth score during the first pass to generate a justification (text). In the second pass, it must predict the score solely from this text.
- Goal: To force the model to encode quality-relevant semantics directly into the reasoning text, effectively treating the text as a compressed representation of the score.

B. Evaluation Metrics

Performance: Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank Order Correlation Coefficient (SRCC) across six datasets (KonIQ, SPAQ, LIVE-W, KADID, AGIQA, CSIQ).
Image-Text Gap: The difference in performance between image-conditioned and text-only inference.
Ablation: Removing score-related words (e.g., "good," "poor," "blurry") from the text to test if the model relies on superficial cues or deeper reasoning.
Attention Analysis: Examining token-level attention weights to see which words drive the prediction.

3. Key Results

A. Performance Comparison

Chain-of-Thought: Introduced little to no improvement in BIQA performance. The text-only performance remained low, indicating that standard CoT does not force the model to encode quality information into the text effectively.
Self-Consistency: Significantly reduced the Image-Text gap.
- It achieved text-only performance comparable to deep learning-based BIQA frameworks.
- The PLCC/SRCC difference between image- and text-conditioned predictions narrowed to 0.02/0.03.
- It performed best when the loss was weighted to prioritize text-to-score mapping ( $\alpha=1, \beta=0$ ).
Autoencoder-like: Less effective at closing the gap compared to Self-Consistency but revealed a different optimization direction. It performed well when explicitly learning the reasoning-to-score relationship ( $\alpha=0, \beta=1$ ).

B. Attention and Token Analysis

Baseline/CoT: Focused on generic visual terms like "focus" and "clear," similar to standard image captioning.
Self-Consistency: Shifted attention toward explicit score-related words (e.g., "good," "moderate," "average").
Autoencoder-like: Highlighted natural quality cues (e.g., "blurry," "focus," "composition").
Robustness Test: When score-related words (e.g., "good," "poor") were removed from the text, the performance drop was negligible. This proves that the models (especially Self-Consistency) have learned implicit quality cues and possess genuine reasoning abilities rather than relying on superficial keyword shortcuts.

4. Key Contributions

Systematic Evaluation of Paradigms: The paper provides the first structured baseline comparing CoT, Self-Consistency, and Autoencoder-like approaches specifically for learning textual representations of image quality.
Bridging the Image-Text Gap: Demonstrated that the Self-Consistency paradigm is the most effective method for narrowing the performance gap between image-based and text-only BIQA, achieving near-parity in some metrics.
Insight into Reasoning Mechanisms: Through token-level analysis, the authors revealed that different training strategies fundamentally alter what the model pays attention to (shifting from visual descriptors to quality descriptors).
General Framework: Proposed a scalable, self-supervised framework (using GRPO) that can be applied to other downstream tasks lacking intermediate reasoning annotations, enabling the induction of interpretable textual explanations.

5. Significance

This work challenges the assumption that MLLMs in BIQA are merely generating superficial explanations. By proving that Self-Consistency training can force a model to rely on textual reasoning for accurate score prediction, the paper:

Validates the potential of interpretable BIQA systems where the text is not just an output but a functional component of the prediction.
Offers a pathway to develop more reliable vision-language models that can generalize to scenarios where visual input might be restricted or where explicit reasoning is required.
Provides a methodological blueprint for improving "black-box" vision tasks by enforcing consistency between modalities, moving beyond simple score regression to true multimodal understanding.