Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping

This paper introduces Venus, a two-stage framework built upon the new AesGuide dataset, which empowers multimodal large language models to provide actionable aesthetic guidance and achieve state-of-the-art performance in aesthetic cropping by addressing the gap between ordinary users and professional photographers.

The Gist

Imagine you have a brand-new smartphone camera. You take a photo of a beautiful sunset, but it looks a bit flat. You ask your friend, "How's this photo?"

• Old AI (like GPT-4o or AesExpert): "Wow, great job! The colors are vibrant, and the composition is lovely! Keep it up!" (They are being too nice to be helpful).
• The Problem: They don't tell you what is wrong or how to fix it. They are like a cheerleader who never gives you a playbook.

Enter "Venus."

Think of Venus not just as a camera app, but as a wise, experienced photography mentor who is always in your pocket. This new system, created by researchers at Peking University, is designed to bridge the gap between taking a "good enough" snapshot and capturing a "masterpiece."

Here is how Venus works, broken down into simple steps:

1. The Missing Piece: "Aesthetic Guidance"

Before Venus, AI could describe a photo or give it a score, but it couldn't act like a pro photographer.

• The Analogy: Imagine a cooking app that tells you, "This soup tastes good!" but refuses to tell you, "You forgot the salt, and the heat is too high."
• The Venus Solution: Venus is the first system that can say, "Hey, the sky is too bright and washing out the building. Try lowering your camera angle to frame the columns better, or wait for the clouds to move." It gives actionable advice while you are taking the picture.

2. The Secret Sauce: The "AesGuide" Cookbook

To teach Venus how to be a mentor, the researchers couldn't just use random internet photos. They needed a special training manual.

• The Analogy: They created a massive library called AesGuide. Imagine a library with 10,000 photos, but instead of just captions, each photo has a detailed critique written by a team of 20 professional photographers.
• The Process: They didn't just ask the AI to guess. They used a "Refinement Factory":
  1. An AI drafts a critique.
  2. Real human experts (the "Head Chefs") review it, fix the mistakes, and ensure the advice is actually useful.
  3. This creates a high-quality dataset where the AI learns to spot flaws (like a messy background) and suggest fixes (like "blur the background").

3. The Two-Stage Training: "Learn to Critique, Then Learn to Crop"

Venus learns in two distinct phases, like a student first learning theory and then practicing on the field.

Stage 1: The Critic (Aesthetic Guidance)

• What happens: Venus is trained to look at a photo and answer three questions: "How good is this?" (Score), "What's wrong?" (Analysis), and "How do I fix it?" (Guidance).
• The Metaphor: It's like a film director watching a rough cut of a movie and saying, "The lighting is too harsh on the actor's face, and the camera is too shaky. Let's fix the light and steady the shot."
• The Result: The AI stops being a "yes-man" and starts being a constructive critic.

Stage 2: The Editor (Aesthetic Cropping)

• What happens: Now that Venus knows why a photo is bad, it learns to fix it by cropping (cutting out the bad parts).
• The Twist: Most AI just guesses where to cut. Venus uses Chain-of-Thought (CoT).
• The Metaphor: Instead of just handing you a cropped photo, Venus explains its logic like a detective: "I am cutting off the top of the building because it looks cluttered. I am moving the person to the center because the empty space on the left makes them look lonely. Here is the new frame, and here is exactly why it looks better."
• Why this matters: It's not just a black box cutting the image; it's an interactive conversation. You can say, "I don't like those boats," and Venus will re-crop to focus on the mountains, explaining why that new composition works.

4. Why This is a Big Deal

• For Regular People: You don't need to be a photography expert to take amazing photos. You just need to talk to Venus, get advice, and let it help you frame the shot.
• For the Tech World: It moves AI from being a "passive judge" (giving a score) to an "active partner" (helping you create).
• The Proof: In tests, Venus didn't just give better advice than other AIs; it also cropped photos better than specialized tools, all while explaining its reasoning in plain English.

Summary

Venus is like giving every smartphone user a personal photography coach. It doesn't just say "Good job!"; it says, "Here's what's missing, here's how to fix it, and here is the perfect crop to make your photo shine." It turns the art of photography from a guessing game into a guided, interactive experience.

Technical Summary

1. Problem Definition

The paper addresses a significant gap in computational aesthetics: the inability of current Multimodal Large Language Models (MLLMs) to provide Aesthetic Guidance (AG).

• The Gap: While smartphones have democratized photography, most users lack the professional expertise to identify aesthetic flaws (e.g., poor composition, lighting issues) or receive actionable advice during capture.
• Limitations of Current MLLMs: Existing general MLLMs (e.g., GPT-4o) and aesthetic-oriented models (e.g., AesExpert) tend to offer overly positive, "flattering" feedback. They fail to critically identify issues or provide specific, actionable shooting guidance (e.g., "adjust the angle," "use a wider aperture").
• Consequence: Without AG capability, these models also struggle with Aesthetic Cropping (reframing photos post-capture). They cannot effectively identify distracting regions or optimize compositional balance, and existing specialized cropping models lack interpretability and interactivity.

2. Methodology

The authors propose Venus, a two-stage framework designed to bridge the gap between subjective aesthetic understanding and objective, actionable adjustments.

Stage 1: Aesthetic Guidance Capability Building

• Dataset (AesGuide): The authors introduce AesGuide, the first large-scale dataset and benchmark specifically for AG.
  • Scale: 10,748 real-world photos.
  • Annotations: Each image includes an aesthetic score, detailed analysis, and actionable guidance (Issue Identification + Shooting Guidance).
  • Construction: A Aesthetic Guidance Generation Framework (AGGF) was used, combining MLLM refinement (using GPT-4o and Qwen2.5-VL) with a two-round review by 20 professional photography experts to ensure high-quality, noise-free labels.
• Training Strategy: The authors employ Supervised Instruction Fine-Tuning (SFT) on open-source MLLMs.
  • Progressive Reasoning: Instead of direct answers, the model is trained to answer progressively complex questions:
    1. Aesthetic Analysis: Forming an overall impression.
    2. Issue Identification: Critically analyzing visual strengths and weaknesses.
    3. Shooting Guidance: Proposing actionable refinements.
  • This aligns the model's reasoning process with human aesthetic cognition.

Stage 2: Aesthetic Cropping Power Activation

• Goal: To unlock the cropping capabilities of the AG-trained MLLM.
• Mechanism: The framework uses Chain-of-Thought (CoT) based Aesthetic Rationales (AR).
  • AR-Generation: For both "good" and "bad" crop candidates, GPT-4o generates explanations justifying why a specific crop improves or worsens the composition.
  • AR-Validation: A stronger MLLM (Qwen2.5-VL-72B) verifies that the generated rationales align with visual content and logic.
• Training: The model is fine-tuned to jointly predict the cropping box and its aesthetic rationale. This grounds spatial decisions in explicit aesthetic reasoning, moving beyond simple coordinate regression.

3. Key Contributions

1. Definition of Aesthetic Guidance (AG): Formally defines AG as a domain bridging subjective aesthetic understanding with objective, actionable shooting adjustments.
2. AesGuide Benchmark: Introduces the first dataset and benchmark for AG, featuring 10,748 annotated photos with scores, analyses, and guidance, evaluated via three metrics (Completeness, Preciseness, Relevance) and expert assessment.
3. Venus Framework: Proposes a two-stage pipeline that first empowers MLLMs with AG capabilities and then activates aesthetic cropping power via CoT-based rationales.
4. State-of-the-Art Performance: Demonstrates that equipping MLLMs with AG capability is essential for high-quality aesthetic cropping, achieving SOTA results in both tasks.

4. Experimental Results

The authors evaluated Venus on five open-source MLLMs (including Qwen-VL, InternVL, MiniCPM-V, and LLaVA variants) and compared them against proprietary models (GPT-4o, Gemini) and specialized cropping models.

• Aesthetic Guidance (AesGuide Benchmark):
  • All Venus variants significantly outperformed baseline models across Completeness, Preciseness, and Relevance.
  • Venus-L-13B achieved the top rank (Mean score: 1.49 vs. 0.98 for GPT-4o), proving that progressive training effectively eliminates "flattery" and improves critical analysis.
• Aesthetic Cropping (FLMS Benchmark):
  • Venus-Q achieved an IoU of 87.01% and a Recall (R) of 92.0%, surpassing the previous SOTA specialized model (ASM-Net: 84.86%) and significantly outperforming GPT-4o (71.61%).
  • Interpretability & Interactivity: Unlike specialized models (which are "black boxes"), Venus provides natural language explanations for its crops and supports interactive refinement (e.g., "I don't like the boats, focus on the sky").
• Ablation Studies:
  • Removing the progressive analysis in Stage 1 reduced AG performance.
  • Removing the AG pretraining (Stage 1) before Stage 2 caused a significant drop in cropping performance, confirming that AG capability is a prerequisite for effective cropping.
  • Removing CoT rationales (AR-Generation/Validation) degraded cropping accuracy, highlighting the importance of reasoning traces.

5. Significance

• Paradigm Shift: Moves computational aesthetics from passive assessment (scoring/describing) to active, actionable improvement (guiding capture and refining composition).
• Human-AI Alignment: By training on expert-annotated data and using progressive reasoning, Venus aligns MLLM behavior with professional photographic standards, addressing the "overly positive feedback" problem.
• Practical Application: The framework enables an interactive, explainable photography assistant that can guide users during capture (AG) and refine images after capture (Cropping), making professional-level aesthetic refinement accessible to ordinary smartphone users.
• Open Science: The authors release the AesGuide dataset, the Venus framework, and code, fostering further research in computational aesthetics.

In summary, Venus successfully demonstrates that equipping MLLMs with the ability to critically analyze and guide photography (AG) is the key to unlocking superior, interpretable, and interactive aesthetic cropping capabilities.