Technical Summary: Personalized Image Generation via Human-in-the-loop Bayesian Optimization

1. Problem Statement

Despite rapid advancements in diffusion models (e.g., FLUX, Stable Diffusion 3), a fundamental gap remains between a user's mental image ($x^*$) and the best image ($x_p^*$) achievable through language prompts alone. This gap arises because the image space is higher-dimensional than language, user vocabulary is limited, and joint language-image models (like CLIP) possess inherent imperfections.

While users can often identify when a generated image is "closer" to their mental target than a previous one, they struggle to articulate the specific linguistic changes required to bridge this gap. Existing methods relying on language prompting, or those requiring extensive offline training with large datasets (e.g., RLHF, DiffusionDPO), are either insufficient for hyper-personalization or computationally expensive. The paper frames this as a human-in-the-loop black-box optimization problem: how to iteratively refine an image using only preferential feedback from a user, without access to the user's internal satisfaction function or the ground truth target $x^*$.

2. Methodology: MultiBO

The authors propose MultiBO (Multi-Choice Preferential Bayesian Optimization), a training-free, interactive framework that optimizes image generation by modifying the internal representations of a diffusion model based on user preferences.

Core Components

The methodology addresses two primary challenges: the high dimensionality of the image space and the difficulty of obtaining precise numerical ratings from users.

A. Multi-Choice Preferential Bayesian Optimization (Multi-choicePBO)

Instead of standard pairwise comparisons (A vs. B), MultiBO presents the user with a set of $K$ images and asks them to select the $N$ best options ($N$-out-of-$K$).

• MultiwiseGPR: To model the user's latent satisfaction function $f$ from this complex feedback, the authors introduce a MultiwiseGaussian Process Regression (MultiwiseGPR) likelihood model. This extends the standard Binomial-Probit/Logit models used in pairwise preference optimization to a Multinomial-Logit framework. It maps the probability of a user selecting a specific subset of images to the latent function values, allowing the system to infer $f$ more efficiently with fewer queries.
• Dynamic Balanced Subspace (DBS) Acquisition Function: Standard Expected Improvement (EI) is computationally intractable when optimizing over $K$ choices jointly. The authors propose DBS, a lightweight acquisition function that:
  1. Computes a single Expected Improvement sample ($x_{EI}$).
  2. Constructs $K$ "anchor" points (bridge vectors) between the current best estimate ($\hat{x}^*$) and $x_{EI}$.
  3. Samples perturbations from a dynamic subspace $\Omega \subseteq Y$. This subspace is determined by the spectral gap of the gradient covariance matrix of the GPR posterior, ensuring that the $K$ choices presented to the user span the most visually significant dimensions of the latent space while maintaining diversity.

B. Self-Attention Warping

Direct optimization in the high-dimensional latent space ($D \approx 16k$) is infeasible. MultiBO instead optimizes within the self-attention mechanism of the diffusion model (specifically the Query, Key, and Value matrices, $Q, K, V$).

• Transformation Space: The optimization is constrained to a space of valid warping transformations (a composition of Affine and Thin-Plate Spline transforms) applied to the attention features.
• Rationale: Self-attention controls spatial attributes, texture, and shape. By warping these features, MultiBO can induce global and local semantic changes without the dimensionality cost of pixel-space optimization. This avoids the need for cross-attention (which is tied to the text prompt) and focuses on refining the visual output.

Workflow

1. Initialization: Generate an initial image $x_p^*$ from a prompt.
2. Iteration (up to $B$ rounds):
   • The DBS module proposes $K$ sets of warping parameters.
   • These parameters are applied to the self-attention features of the diffusion model to generate $K$ candidate images.
   • The user selects the $N$ best images.
   • The MultiwiseGPR model updates its posterior belief of the satisfaction function $f$.
   • The process repeats until the budget $B$ is exhausted, outputting the final optimized image $\hat{x}^*$.

3. Key Contributions

1. Multi-Choice Preference Framework: The paper demonstrates that $N$-out-of-$K$ preference feedback provides significantly richer information than pairwise comparisons, reducing the user query burden. It mathematically formalizes this via the MultiwiseGPR likelihood and a corresponding acquisition function.
2. Attention-Space Optimization: Unlike methods that optimize noise vectors or pixel space, MultiBO optimizes self-attention features ($Q, K, V$) using a constrained warping transformation space. This balances the need for expressive editing with computational feasibility.
3. Training-Free Personalization: The approach requires no offline training on large datasets or fine-tuning of the diffusion model. It operates entirely at inference time, making it model-agnostic and adaptable to individual user preferences.
4. Dynamic Subspace Selection: The DBS acquisition function adaptively identifies the most influential dimensions for optimization, preventing the search from collapsing into redundant variations.

4. Experimental Results

The authors evaluated MultiBO against five categories of baselines: training-based preference alignment (DiffusionDPO, IterComp), training-free inference methods (DEMON, DNO, DAS), and metric-guided variants of MultiBO.

• Target Alignment: MultiBO significantly outperformed all baselines in CLIP-I2I and LPIPS metrics (measuring similarity to the target $x^*$), achieving results comparable to an "Oracle" L2-guided upper bound. Notably, it outperformed variants where the human was replaced by automated reward models (e.g., MultiBOCLIP-I2I), validating that human feedback captures nuances missed by proxy metrics.
• Preference Metrics: MultiBO achieved competitive or superior scores on standard reward metrics (ImageReward, Aesthetic, HPSv2) compared to training-based methods, despite using no training data.
• Robustness: Unlike some baselines (e.g., DNO, DAS) that suffered from "reward hacking" (generating poor images that scored high on specific metrics), MultiBO maintained visual integrity.
• Human Evaluation: In a study with 30 volunteers, MultiBO achieved a 70.82% win rate in selecting the image closest to the target, significantly outperforming all baselines (the next best was 15.60%).
• Ablation Studies:
  • Multiwise vs. Pairwise: Multi-choice feedback ($K=4$) converged faster and with higher quality than pairwise ($K=2$).
  • Choice Set Size: $K=4$ offered the optimal trade-off between information gain and user cognitive load; $K=10$ led to diminishing returns.
  • Layer Selection: Editing middle-to-later self-attention layers (decoder layers) yielded the best spatial control without destroying semantic structure.
  • Timesteps: Editing during the early-to-mid denoising steps provided the most steering freedom.

5. Significance and Claims

The paper claims that MultiBO effectively narrows the "last-mile" gap between a user's imagination and generative model output. Its significance lies in:

• Efficiency: Achieving high-fidelity personalization with only ~50 user queries, avoiding the computational cost of training large reward models.
• Human-Centric Design: By leveraging the human ability to judge relative quality (preference) rather than absolute quality (rating), the system bypasses the limitations of language prompting and imperfect reward models.
• Practicality: The method is training-free and model-agnostic, offering a viable path for hyper-personalized image generation without requiring massive datasets or specialized hardware for training.

The authors acknowledge limitations, noting that optimization can stall in scenarios where user preferences are ambiguous (equally bad/good choices) and that the constrained warping space may limit the scope of possible edits compared to full latent space optimization. Future work is proposed to incorporate informed priors from pre-trained reward models and explore high-dimensional optimization strategies better suited to latent diffusion geometry.