POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation

1. Problem Statement

Visual text generation models (e.g., diffusion models) face a fundamental trade-off between text accuracy (rendering correct characters) and overall image coherence/aesthetics (following complex prompts and maintaining visual quality).

• The Conflict: Optimizing for text accuracy often degrades aesthetic quality and instruction-following capabilities, and vice versa.
• Limitations of Current RL Approaches: Recent Reinforcement Learning (RL) methods like Group Relative Policy Optimization (GRPO) attempt to solve this via multi-reward optimization. However, they typically use a weighted-sum approach to aggregate rewards (e.g., OCR score + CLIP score + Aesthetic score).
  • Instability: This scalarization creates ambiguous optimization targets when rewards conflict (e.g., a sample with high OCR but low aesthetics), leading to unstable training and suboptimal convergence.
  • Data Inefficiency: Existing methods often rely on static, large datasets. Using too much data is computationally expensive, while small datasets lack diversity. Furthermore, there is no systematic strategy for selecting which prompts to train on to maximize learning efficiency.

2. Methodology: POCA Framework

The authors propose POCA (Pareto-Optimal Curriculum Alignment), a synergistic framework combining two core components to address the multi-objective optimization and data selection challenges.

A. Bi-Directional Pareto-Optimal Selection

Instead of collapsing multiple rewards into a single scalar value, POCA treats the problem as a true multi-objective optimization task.

• Pareto-Optimal Set Identification: Within a group of sampled outputs for a given prompt, the algorithm identifies the Pareto set.
  • Non-Dominated Set (Positive): Samples where no other sample is better in all reward dimensions. These represent the best trade-offs.
  • Fully Dominated Set (Negative): Samples that are worse than others in all dimensions.
• Bi-Directional Sorting: The algorithm performs sorting on both the original rewards (to find the best) and the negated rewards (to find the worst).
• Policy Update: Only samples in the Pareto set (both positive and negative) are used to calculate the advantage function in GRPO. Samples outside this set are assigned zero advantage. This eliminates inconsistent signals, stabilizing the training process by focusing only on clear trade-off examples.

B. Adaptive Curriculum Planning

POCA introduces a dynamic "easy-to-hard" learning path to manage the training sequence efficiently.

• Difficulty Measurer: The system uses the OCR reward model as a proxy for difficulty. Empirical analysis shows OCR rewards have higher variance across the dataset compared to CLIP or aesthetic scores, making them a better discriminator for difficulty.
  • It calculates the mean OCR reward for each prompt and normalizes it using the Empirical Cumulative Distribution Function (ECDF) to determine a percentile rank (difficulty score).
• Training Scheduler: The dataset is dynamically partitioned into three bins: Easy, Medium, and Hard based on the ECDF scores.
  • The training process is divided into stages (e.g., 3 stages).
  • The mixture ratio of prompts from these bins changes over time (e.g., starting with mostly Easy/Medium and gradually increasing the proportion of Hard prompts).
  • This ensures the model masters simpler trade-offs before tackling complex, conflicting objectives, mimicking human learning progression.

3. Key Contributions

1. Stabilization of Multi-Reward RL: The authors demonstrate that standard weighted-sum GRPO is unstable for conflicting rewards. They introduce a bi-directional Pareto sorting mechanism that filters out ambiguous signals, retaining only valid positive and negative trade-off samples for optimization.
2. Adaptive Curriculum Strategy: They design a model-driven curriculum that automatically assesses prompt difficulty and schedules data exposure from easy to hard. This improves convergence speed and final performance without requiring massive static datasets.
3. Synergistic Framework (POCA): By combining Pareto-optimal selection with adaptive curriculum planning, the framework achieves superior balance between text accuracy, text-image alignment, and aesthetic quality.

4. Experimental Results

The method was evaluated on the AnyText-benchmark and a custom POCA-20k dataset (20k images with generated prompts).

• Quantitative Performance:
  • POCA outperforms the base model (AnyText) and the current SOTA (AnyText2) across all metrics: Sentence Accuracy (Sen.ACC), Normalized Edit Distance (NED), CLIP Score (alignment), and HPS Score (aesthetics).
  • Crucially, POCA significantly outperforms the GRPO baseline (which uses weighted-sum aggregation), proving that Pareto selection resolves the instability of scalarization.
• Qualitative Analysis:
  • Visual comparisons show POCA generates images with seamless text integration, harmonious colors, and higher fidelity to complex instructions (e.g., specific lighting, textures, and layout constraints) compared to baselines.
  • Human preference studies (22 participants) confirmed POCA wins significantly in text accuracy, prompt following, and aesthetics.
• Ablation Studies:
  • Weighted-sum vs. Pareto: Pareto-guided training achieved higher OCR and CLIP rewards, whereas the weighted-sum baseline converged to a suboptimal state.
  • Curriculum: The adaptive multi-stage design yielded faster convergence and higher accuracy than single-stage training or naive data mixing.
  • Data Efficiency: POCA trained on a 1k subset achieved performance comparable to methods trained on 10k subsets, highlighting high data efficiency.

5. Significance

• Theoretical Advancement: POCA provides a robust solution to the "multi-objective dilemma" in diffusion model alignment, moving beyond simple scalarization to true Pareto-optimal learning.
• Practical Efficiency: The adaptive curriculum strategy offers a blueprint for training high-performance models with reduced computational costs by intelligently selecting data rather than brute-forcing large datasets.
• Generalizability: The framework is model-agnostic, demonstrated by successful application to both AnyText and Glyph-SDXL-v2 architectures.
• Impact on Visual Text Generation: By effectively balancing the conflicting goals of legibility and aesthetics, POCA enables the generation of high-quality, design-ready images with accurate text, a critical requirement for commercial applications like posters, logos, and advertisements.