💻 computer science

HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluation

This paper introduces HanMoVLM, a specialized vision-language model enhanced with expert-validated Chain-of-Thought reasoning and a novel auction-based dataset (HanMo-Bench) to achieve professional-grade evaluation of Chinese paintings and serve as a high-quality verifier for improving artistic image generation.

Original authors: Hongji Yang, Yucheng Zhou, Wencheng Han, Songlian Li, Xiaotong Zhao, Jianbing Shen

Published 2026-03-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hongji Yang, Yucheng Zhou, Wencheng Han, Songlian Li, Xiaotong Zhao, Jianbing Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot that can look at a picture and tell you, "That's a mountain," or "That's a dog." This robot is like a generalist art critic who has seen millions of photos from the internet. It's great at identifying objects, but if you ask it to judge the soul of a traditional Chinese painting, it's completely lost. It might think a painting is "good" just because the colors are bright or the dog looks realistic, missing the subtle, spiritual depth that makes Chinese art special.

The paper you shared introduces HanMoVLM, a new AI designed to fix this. Think of it as upgrading that generalist robot into a Master Art Critic who has spent decades studying Chinese brushwork, history, and philosophy.

Here is a breakdown of how they did it, using simple analogies:

1. The Problem: The "Tourist" vs. The "Local"

The Tourist (Generic AI): When a regular AI looks at a Chinese landscape painting, it sees "trees, rocks, and water." It judges it like a photograph: "Is the dog realistic? Is the perspective correct?" But Chinese art isn't about perfect realism; it's about "spirit," "flow," and "mood." The tourist AI gets the facts right but misses the feeling.
The Local (HanMoVLM): HanMoVLM is trained to think like a local expert. It doesn't just see a rock; it sees the "spirit" of the rock and how the artist's brushstroke conveys a feeling of strength or tranquility.

2. The Solution: Teaching the AI to "Think Like a Master"

The researchers didn't just feed the AI more pictures. They taught it a specific step-by-step thinking process (called Chain-of-Thought), similar to how a human expert would analyze a painting:

Step 1: The Big Picture (Theme): First, identify what kind of painting it is. Is it a Landscape (mountains/water), Flower & Bird, or Figure (people)? You can't judge a fish by how well it climbs a tree; you need the right rules for the right genre.
Step 2: Zoom In (The "RoI"): The AI learns to spot the "Regions of Interest." Instead of looking at the whole canvas, it zooms in on the specific pine tree or the tiny figure of a traveler to see the details of the brushwork.
Step 3: The Three-Layer Taste Test: This is the secret sauce. The AI evaluates the painting on three levels, like tasting a complex dish:
1. Brush & Ink (The Ingredients): Is the technique skillful? Are the lines strong and the ink layered?
2. Spirit Resonance (The Aroma): Does the painting feel "alive"? Does it have a rhythm or energy that flows through the image?
3. Artistic Conception (The Aftertaste): This is the most important part. Does the painting create a poetic mood? Does it make you feel something deep? In Chinese art, this "vibe" is worth more than just having perfect details.

3. The Training: The "Strict Coach"

To make sure the AI actually learned this, the researchers built a special gym (called HanMo-Bench).

The Weights: They gathered real, famous paintings from auctions (the "gold standard") and also AI-generated paintings (some good, some bad).
The Coach (Reward Function): They created a "coach" that watches the AI's thinking process. If the AI says, "This is a 5-star painting because the dog looks real," the coach gives it a low score and says, "Wrong! You missed the spirit." If the AI says, "The brushwork is weak, but the mood is peaceful," the coach gives it a high score.
The Result: Through this strict training, the AI learned to stop guessing and start reasoning like a human expert.

4. The Superpower: The "Art Director"

Once HanMoVLM became a great critic, the researchers used it as a filter for creating new art.

Imagine you ask an AI to "paint a mountain scene." It might generate 10 different versions.
A regular AI might just pick the first one.
HanMoVLM looks at all 10, acts as the "Art Director," and picks the one that actually feels like a masterpiece, discarding the ones that look fake or soulless. This is called Test-time Scaling—using a smart judge to pick the best output without needing to retrain the painter.

Summary

In short, HanMoVLM is an AI that stopped being a "tourist" who just recognizes objects and became a "scholar" who understands art. By teaching it to think step-by-step and rewarding it for understanding the spirit of Chinese painting, they created a tool that can:

Grade art with the same accuracy as human experts.
Filter AI-generated art to ensure it's truly beautiful, not just technically correct.

It bridges the gap between "computers seeing pixels" and "humans feeling art."

1. Problem Statement

Current Large Vision-Language Models (VLMs) excel at general visual recognition and logical reasoning but suffer from "artistic blindness" when evaluating specific, culturally rich domains like Chinese traditional painting. The paper identifies three primary gaps preventing generic VLMs from acting as professional art critics:

Artistic Misalignment: Generic models lack the domain-specific knowledge to understand the unique aesthetics of non-Western art (e.g., Chinese painting relies on "spirit resonance" and "artistic conception" rather than realism or linear perspective).
Understanding-Evaluation Gap: While models can identify elements (e.g., "mountains," "trees"), they fail to translate this recognition into a professional, expert-level evaluation or scoring.
Low-Quality Data: There is a scarcity of high-quality, expert-annotated datasets that link visual features to professional valuations and reasoning chains.

2. Methodology

The authors propose HanMoVLM, a specialized VLM framework designed to mimic the reasoning process of Chinese art experts. The methodology consists of four core components:

A. HanMo-Bench (Dataset Construction)

A new benchmark dataset containing 13,162 training and 600 test images, comprising:

Authentic Masterpieces: High-resolution images from public auction records with professional valuations.
Synthetic Artwork: AI-generated images (using models like GPT-Image-1.5 and Nano-Banana) to test generative capabilities.
Unified Valuation: A 0–5 scoring scale where authentic masterpieces are normalized based on auction tiers, and synthetic works are rated by VLMs and verified by human experts.
Expert Chain-of-Thought (CoT): Multi-turn dialogues constructed by experts covering content identification, Region of Interest (RoI) localization, theme classification, and three-tier evaluation.

B. Expert-Level Chain-of-Thought (CoT)

The model is guided through a structured reasoning process that mimics human experts:

Content & Theme Identification: Classifying the painting into Landscape (Shanshui), Flower & Bird, or Figure categories.
RoI Localization: Identifying specific regions of interest (e.g., specific brushstrokes, composition centers) rather than just global features.
Theme-Specific Evaluation: Applying criteria specific to the identified genre (e.g., checking "bone strength" in Figure paintings vs. "scattered perspective" in Landscapes).
Three-Tier Evaluation: Assessing the work based on the traditional Chinese hierarchy:
- Brush & Ink (Bimo): Quality of lines, ink layers, and structure.
- Spirit Resonance (Qiyun): Vitality, rhythm, and flow of the artwork.
- Artistic Conception (Yijing): The poetic space, emotional depth, and cultural meaning (weighted highest).

C. Training Pipeline (SFT + RFT)

Supervised Fine-Tuning (SFT): The base model (Qwen3-VL) is fine-tuned using the expert-generated CoT and answers to learn the structured reasoning format.
Reinforcement Fine-Tuning (RFT): The model is further optimized using Group Relative Policy Optimization (GRPO). A novel multi-component reward function is designed to supervise the reasoning chain:
- Accuracy Reward ( $R_{acc}$ ): Ensures the final score matches the ground truth.
- BERT Reward ( $R_{BERT}$ ): Measures semantic similarity between the model's reasoning text and expert references.
- IoU Reward ( $R_{mIoU}$ ): Evaluates the precision of predicted Regions of Interest (RoIs) against ground-truth bounding boxes.

D. Test-Time Scaling for Generation

HanMoVLM serves as an external verifier for Text-to-Image (T2I) models. Instead of retraining generative models, the system employs a Best-of-N strategy:

A T2I model generates $N$ candidate images from a single prompt.
HanMoVLM evaluates all candidates using its expert reasoning.
The image with the highest expert score is selected as the final output.

3. Key Contributions

HanMo-Bench: The first rigorous benchmark for Chinese painting evaluation featuring authentic auction-grade works, AI-generated art, and expert-validated CoT data.
HanMoVLM Framework: A VLM architecture that integrates global-to-local visual analysis with domain-specific CoT, enabling structured expert reasoning.
Specialized Reward Mechanism: A novel reward function that provides granular supervision across theme identification, RoI localization, and rating rationality, significantly reducing hallucinations in reasoning.
Test-Time Scaling Application: Demonstrating that a high-quality verifier can drastically improve the artistic quality of T2I generation without modifying the generative model itself.

4. Experimental Results

Evaluation Performance: HanMoVLM significantly outperforms state-of-the-art models (including Gemini-3, GPT-5.1, and InternVL).
- On the 8B parameter version, HanMoVLM achieved 46.67% accuracy in score prediction, compared to ~21.5% for the base model with standard CoT.
- It achieved a 0.8912 RMSE and 0.5598 BERTScore, indicating high alignment with expert reasoning.
Human Study: In a study with 43 Chinese art experts, HanMoVLM's rankings showed high consistency with human preferences (Kendall's $\tau$ = 0.758, Spearman's $\rho$ = 0.845), far surpassing aesthetic scores and PickScore which failed to correlate with human rankings in this domain.
Generation Quality: When used as a verifier for T2I models, HanMoVLM successfully filtered out AI artifacts and culturally misaligned outputs, selecting images that were rated significantly higher by human experts.
Ablation Studies: Removing any component of the reward function (Accuracy, BERT, or IoU) resulted in a measurable drop in the corresponding metric, confirming the necessity of the multi-faceted supervision.

5. Significance

This work bridges the critical gap between general AI vision and specialized cultural expertise. It demonstrates that:

Domain Adaptation: VLMs can be transformed into domain experts through structured CoT and reinforcement learning, even for abstract cultural concepts like "Spirit Resonance."
Quality Control: High-quality, expert-aligned verifiers are a more efficient path to improving generative art than retraining massive diffusion models.
Cultural Preservation: The framework provides a scalable tool for evaluating, preserving, and generating traditional Chinese art, addressing the "Western-centric bias" prevalent in current AI models.

In conclusion, HanMoVLM establishes a new paradigm for professional artistic evaluation, proving that with the right data, reasoning structures, and reward mechanisms, AI can achieve a level of cultural understanding previously thought to require years of human training.