Personalizing Text-to-Image Generation to Individual Taste

1. Problem Statement

Current Text-to-Image (T2I) diffusion models (e.g., Flux 2, Stable Diffusion) excel at generating high-fidelity images but suffer from a fundamental limitation: they optimize for population-level consensus rather than individual user preferences.

• Subjectivity Gap: Aesthetic judgment is highly subjective. Two users typing the same prompt often desire fundamentally different visual outcomes (e.g., lighting, composition, style), yet current systems treat all users identically.
• Limitations of Existing Reward Models: Current reward models (e.g., ImageReward, HPS) are trained on aggregated, "average" human preferences. They fail to capture individual nuances and often rely on outdated or uncurated datasets, leading to "reward hacking" where models produce generic, oversaturated, or stylized "AI-looking" images that degrade perceptual quality for specific users.
• Data Scarcity: Existing datasets for aesthetic assessment either focus on real-world photography (ignoring AI artifacts) or AI-generated images with sparse user ratings (1–4 ratings per image), making it impossible to train robust models for zero-shot personalization.

2. Methodology

The authors propose PAM∃LA (Personalized Aesthetic Model & Large-scale Appraisals), a comprehensive framework consisting of a novel dataset and a predictive model.

A. The PAM∃LA Dataset

• Scale: Contains 70,000 ratings across 5,077 diverse images.
• Density: Each image is rated by 15 unique users, providing a dense distribution of subjective preferences.
• Content: Images are generated using state-of-the-art models (Flux 2 and Nano Banana) to minimize visual artifacts. The dataset covers two primary domains:
  • Artistic: Classical compositions (landscapes, portraits, still lifes).
  • Photorealistic: Commercial and cinematic subjects (architecture, food, fashion).
• Annotation: Users rate images on a 5-point scale (Bad to Excellent). The dataset includes user demographics (age, gender, nationality, education, art experience) and image metadata.
• Splits: The data is partitioned into:
  • Seen Users: Testing on new prompt-image pairs for users present in training.
  • Unseen Users: Few-shot testing where the model predicts ratings for new users based on a small context set ($k=15$) of their ratings.

B. The PAM∃LA Predictor (Model Architecture)

The predictor is a user-conditioned multimodal transformer designed to estimate a personalized aesthetic score.

• Input Embeddings:
  • Visual & Text: Extracted via a frozen SigLIP2 encoder from the image and its prompt.
  • Metadata: User demographics and image attributes are serialized into text and encoded using a frozen Llama-Embed-Nemotron-8B encoder.
  • User Identity: A learnable embedding table captures idiosyncratic preferences for users seen during training.
• Fusion: All features are projected to a shared dimension and concatenated into a token sequence with a learnable [CLS] token.
• Processing: A shallow fusion transformer encoder processes the sequence.
• Output: The [CLS] representation passes through a linear regression head to predict the aesthetic score.
• Training Strategy:
  • Joint Training: The model is trained jointly on PAM∃LA, LAPIS (artworks), and PARA (photographs) to ensure robustness across visual domains.
  • Few-Shot Personalization (Unseen Users): For new users, the model constructs a "preference profile vector" by computing a rating-weighted average of their image embeddings. It then retrieves the $K$ nearest training users based on this profile and interpolates their learned user embeddings to generate a prediction.

C. Image Steering via Prompt Optimization

The framework utilizes reward-driven iterative prompt optimization:

1. An LLM generates prompt variations.
2. Images are rendered using the T2I model.
3. The PAM∃LA predictor scores the images.
4. The highest-scoring prompt is selected for the next iteration.
   This process steers the generation toward the specific user's taste without fine-tuning the diffusion model itself.

3. Key Contributions

1. PAM∃LA Dataset: The first large-scale benchmark for personalized T2I evaluation, featuring dense multi-rater coverage (15 ratings/image) on AI-generated content, bridging the gap between computational aesthetics and generative AI.
2. Personalized Reward Predictor: A novel model that outperforms state-of-the-art population-level reward models in predicting individual user preferences, achieving strong zero-shot generalization to unseen users via few-shot learning.
3. Effective Steering: Demonstration that simple prompt optimization guided by personalized rewards can significantly improve image quality for specific users, avoiding the "generic AI look" associated with global reward models.
4. Demographic Insights: Analysis showing that demographic factors (e.g., age) significantly influence aesthetic preferences, and that global models inadvertently overfit to the biases of dominant annotator groups.

4. Results

Quantitative Performance

• Prediction Accuracy: On the held-out test set, PAM∃LA outperforms all baselines (LAION, ImageReward, HPSv3, Q-Align) in both User-Level and Population-Level metrics.
  • User SROCC: 0.4514 (vs. 0.4019 for HPSv3).
  • User Pairwise Accuracy: 0.6631 (vs. 0.6427 for HPSv3).
  • Population PLCC: 0.6116 (maintaining SOTA performance on global averages).
• Diverging Preferences: The model correctly predicts preferences in "hard" cases where users disagree (61.44% accuracy for seen users), a task where consensus models fail (random chance).

Qualitative & User Study Findings

• User Preference: In a pairwise study (15,300 ratings), images optimized for a user's specific taste using PAM∃LA were significantly preferred over unoptimized images and images optimized with generic models (HPSv3, Q-Align).
• Quality Preservation: Unlike global models that often degrade realism or over-saturate images during optimization, PAM∃LA steers images by adjusting compositional elements (lighting, angle, viewpoint) while maintaining photorealism.
• Consistency: Optimization runs are reproducible; the model consistently converges to the same stylistic elements for a specific user across different LLM sampling runs.
• Demographic Bias: The study revealed that global models treat high saturation as a universal quality proxy, whereas PAM∃LA shows that younger users prefer higher saturation while older users may prefer different aesthetics, highlighting hidden biases in consensus models.

5. Significance and Conclusion

The paper establishes that personalization is critical for the next generation of generative AI. By moving away from "average" human appeal toward individual taste alignment, PAM∃LA demonstrates that:

• Subjectivity is learnable: Individual aesthetic preferences can be modeled effectively using large-scale, dense datasets and multimodal transformers.
• Consensus is not optimal: Optimizing for the "crowd" often degrades the experience for the individual, leading to generic outputs.
• Data Quality Matters: High-quality, diverse, and dense annotations are essential to capture the nuance of human judgment.

The release of the PAM∃LA dataset and model provides a standardized benchmark for the community to advance personalized T2I alignment, moving the field from generating "good" images to generating images that are "good for you."