PortraitCraft: A Benchmark for Portrait Composition Understanding and Generation

1. Problem Statement

Portrait composition is a critical factor in visual aesthetics, influencing subject emphasis, cropping, body pose, facial expression, and the interaction between the subject and the background. However, existing research and datasets face several limitations:

• Coarse Evaluation: Most current datasets (e.g., AVA, AADB) focus on generic image aesthetics or provide only a single global aesthetic score, failing to capture the multi-faceted nature of portrait composition.
• Lack of Structure: Existing resources do not support structured, fine-grained analysis of specific composition attributes (e.g., lighting, spatial organization, visual stability).
• Disconnected Tasks: Research on portrait composition understanding (analysis) and generation (synthesis) is typically conducted in isolation. There is no unified benchmark that links structured composition analysis with controllable generation based on explicit composition constraints.
• Data Scarcity: There is a lack of large-scale, expert-annotated datasets specifically designed for portrait photography that include multi-level supervision (scores, attributes, explanations, and generation prompts).

2. Methodology

The authors introduce PortraitCraft, a unified benchmark designed to bridge the gap between composition understanding and generation. The methodology consists of three core components:

A. Dataset Construction

• Scale & Source: The dataset comprises approximately 50,000 curated real portrait images sourced from Unsplash.
• Curation Process: A multi-stage filtering pipeline was used, involving keyword retrieval, automatic model screening, and rigorous manual inspection by experts to remove low-quality, AI-generated, or privacy-sensitive images.
• Annotation Framework:
  • Expert Involvement: Professional designers and photographers with 3–10+ years of experience performed the core annotations.
  • Model-Assisted Generation: Advanced Multimodal Large Language Models (MLLMs) were used to generate initial explanation texts, Visual Question Answering (VQA) pairs, and structured descriptions, which were then reviewed and revised by experts.
  • Multi-Level Supervision: The dataset includes:
    1. Global Composition Scores: Overall aesthetic ratings.
    2. 13 Composition Attributes: Fine-grained labels covering color/style, technical/lighting quality, spatial organization, subject presentation, and visual stability.
    3. Attribute-Level Explanations: Textual rationales for the attribute scores.
    4. VQA Pairs: Image-grounded multiple-choice questions with high-distractor options.
    5. Structured Descriptions: Composition-oriented textual prompts for generation tasks.

B. Benchmark Tasks

The benchmark defines two complementary tracks within a unified framework:

1. Track 1: Portrait Composition Understanding
   • Goal: Evaluate a model's ability to analyze composition quality structurally.
   • Inputs: A single portrait image.
   • Outputs:
     • Prediction of an overall composition score.
     • Classification of 13 specific attributes into three levels (Good, Medium, Poor).
     • Answering a multiple-choice visual question based on the image.
2. Track 2: Portrait Composition Generation
   • Goal: Evaluate a model's ability to generate portraits that strictly adhere to explicit composition constraints.
   • Inputs: A structured textual description detailing composition requirements (e.g., subject placement, negative space, depth layering).
   • Outputs: A generated portrait image.
   • Evaluation: Focuses on composition alignment (how well the generated image matches the structural constraints in the text) rather than just visual realism or pixel-level similarity.

C. Evaluation Protocols

• Understanding Metrics: Spearman's Rank Correlation Coefficient (SRCC) and Pearson Linear Correlation Coefficient (PLCC) for score prediction; Accuracy for attribute classification and VQA.
• Generation Metrics: An internal model extracts structured composition properties from generated images to compare them against the target textual specifications, calculating an alignment score.

3. Key Contributions

1. Unified Benchmark: The first framework to jointly evaluate portrait composition understanding and generation, connecting analysis with controllable synthesis.
2. Large-Scale Curated Dataset: A dataset of ~50,000 real portraits with rich, multi-level supervision (global scores, 13 attributes, explanations, VQA, and structured prompts), constructed via expert-human-in-the-loop processes.
3. Structured Representation: Introduction of a standardized 13-attribute taxonomy for portrait composition, moving beyond coarse global scoring to fine-grained, interpretable assessment.
4. Standardized Evaluation: Defined protocols for both tracks, including specific metrics for composition alignment in generation, addressing the lack of standardized testing in this domain.
5. Baseline Results: Provided reference implementations using Qwen3-VL-4B, demonstrating the feasibility of the tasks and establishing a performance baseline for future research.

4. Experimental Results

The authors conducted experiments using Qwen3-VL-4B as a baseline model:

• Track 1 (Understanding):
  • Fine-tuning the model on PortraitCraft significantly improved performance over zero-shot settings across all metrics.
  • SRCC/PLCC: Improved from ~0.67/0.64 (zero-shot) to ~0.78/0.75 (fine-tuned), indicating better alignment with expert scoring.
  • VQA Accuracy: Increased from 78.25% to 87.65%, showing enhanced image-grounded reasoning.
  • Attribute Accuracy: Improved from 42.90% to 45.92%. The smaller gain here highlights that fine-grained judgment across 13 attributes remains a challenging task.
• Track 2 (Generation):
  • Qualitative results demonstrated that fine-tuned models can generate visually plausible portraits that adhere to specific structural constraints (e.g., subject placement, spatial organization).
  • The results confirm that the dataset effectively supports composition-aware generation, though precise control over fine-grained structures remains an open research challenge.

5. Significance and Impact

• Advancing Aesthetic AI: Shifts the paradigm from generic aesthetic scoring to structured, interpretable, and fine-grained portrait analysis.
• Controllable Generation: Provides a pathway for generating images that are not just realistic but compositionally correct according to explicit human-defined rules, which is crucial for professional photography and design applications.
• Research Foundation: Offers a comprehensive resource for studying the relationship between visual understanding and generative control, enabling future work on interpretable aesthetic assessment and structured image synthesis.
• Community Resource: The dataset and code are publicly available, fostering reproducible research in portrait composition.

In summary, PortraitCraft addresses a critical gap in computer vision by providing a structured, expert-annotated benchmark that treats portrait composition as a multi-faceted problem, enabling both deep analysis and precise, constraint-based generation.