🤖 AI

Self-Improving Diffusion Classifiers with Minority Preference Optimization

This paper introduces MiPO, a method that enhances the zero-shot classification performance of diffusion models in minority regions by fine-tuning pretrained models with minority preference optimization to improve their perception of underrepresented data distributions without requiring additional image data or external reward models.

Original authors: Hyunsoo Kim, Jungmyung Wi, Soobin Um, Donghyun Kim, Suhyun Kim

Published 2026-07-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hyunsoo Kim, Jungmyung Wi, Soobin Um, Donghyun Kim, Suhyun Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart artist who has spent years painting millions of pictures. This artist is so good that if you ask them to paint a "dog" or a "car," they can do it perfectly. In the world of AI, this artist is called a Diffusion Model.

Recently, scientists discovered that this artist is also surprisingly good at playing a guessing game: "What is in this picture?" This is called a Diffusion Classifier. However, there's a catch. The artist is a bit of a snob. They are incredibly confident when guessing common things (like a standard dog or a car) because they've seen them a million times. But if you show them something rare or unusual (like a specific type of weird duck or a rare sink), they often get it wrong. They struggle with the "minority" items because they haven't practiced them enough.

The Problem: The Artist's Blind Spot

The paper explains that this happens because the artist was trained mostly on common, popular images. They are great at the "majority" but terrible at the "minority." Previous attempts to fix this were like giving the artist a cheat sheet only while they were painting. It helped them make a few rare pictures, but it didn't actually teach the artist how to understand rare things better. Once the cheat sheet was gone, they were back to being confused.

The Solution: MiPO (Minority Preference Optimization)

The authors, Hyunsoo Kim and his team, created a new method called MiPO. Think of MiPO as a self-improving coach that helps the artist learn to appreciate and understand the rare stuff without needing to see new real-life photos.

Here is how MiPO works, using a simple analogy:

The Practice Session (Self-Generation): Instead of showing the artist new photos (which they don't have), MiPO asks the artist to imagine and draw pictures based on random descriptions (captions).
The "Rareness" Score: After the artist draws a picture, MiPO checks it. If the picture looks like something the artist usually ignores (a "minority" region), MiPO gives them a high score (a reward). If it looks like a boring, common picture, they get a low score.
The Gentle Nudge (LoRA & RL): MiPO uses a special, lightweight tool (called LoRA) to tweak the artist's brain. It's like adding a small pair of glasses that helps the artist focus better on the rare details. It uses a technique called Group Relative Policy Optimization (GRPO), which is like comparing a group of drawings. If one drawing is "rarer" than the others in the group, the artist gets extra credit for that specific style.
The Safety Net (KL Regularization): There's a risk that in trying to be "rare," the artist might start drawing nonsense or forget how to draw normal things. To prevent this, MiPO adds a "safety belt" (KL Regularization). This ensures the artist stays true to their original skills while just expanding their knowledge of the rare stuff.

The Results

The paper tested this new coach on five different standard tests (like CIFAR10 and ImageNet). Here is what happened:

Better at the Hard Stuff: The artist became much better at recognizing the rare, low-density items they used to fail at.
Still Good at the Easy Stuff: Because of the "safety belt," the artist didn't forget how to draw or recognize common things.
No Extra Photos Needed: The whole process happened using only text descriptions. They didn't need to feed the model any new images.
Fast and Flexible: The "glasses" (LoRA) are small and easy to put on or take off. They work with different types of artists (different diffusion models) and don't slow down the process much.

The Bottom Line

The paper claims that by teaching the AI to "practice" generating and recognizing rare items through this self-improving loop, the AI becomes a much better all-around observer. It fixes the bias where the AI only understands the popular stuff, making it more robust and accurate for everything from common objects to the weird and wonderful ones.

In short: MiPO teaches the AI to stop ignoring the underdogs, making it a smarter, fairer classifier without needing any new training data.

Technical Summary: Self-Improving Diffusion Classifiers with Minority Preference Optimization (MiPO)

1. Problem Statement

Diffusion classifiers (DCs) have demonstrated robust zero-shot classification performance, particularly regarding out-of-distribution (OOD) samples and compositional understanding. However, their effectiveness is heavily dependent on the pretraining data distribution. Current DCs perform well in "majority" (high-density) regions of the data manifold but exhibit significantly degraded accuracy in "minority" (low-density) regions.

This performance gap arises because the standard noise-matching loss function used to train diffusion models is inherently biased toward frequent, high-density samples. Consequently, the generative coverage of the model is sparse in underrepresented regions, leading to poor perception capabilities for minority concepts. While prior works have attempted to address minority sampling through inference-time guidance or auxiliary classifiers, these methods often incur high computational costs, require external models, or fail to update the underlying representation of the diffusion model itself, limiting their ability to improve classification performance.

2. Methodology: MiPO

The authors propose MiPO (Minority Preference Optimization), a self-improving framework that fine-tunes a pretrained text-to-image diffusion model to enhance its representation of minority regions without requiring additional image data or external reward models.

Core Components

Self-Improving Mechanism: MiPO utilizes only arbitrary caption data (e.g., from HPSv2) to generate candidate samples. It does not rely on external image datasets or foundation models for reward calculation.
Minority Preference Reward: To quantify how well a generated sample covers a minority region, the method employs a DDIM-based approximation of Tweedie's formula.
- For a generated image $x_0$ , Gaussian noise is added to create a perturbed latent $x_t$ .
- The model reconstructs $\hat{x}_0$ via DDIM inversion.
- The Minority Score is defined as the reconstruction discrepancy: $M(x_0) = \|\hat{x}_0 - x_0\|_2$ .
- Higher reconstruction errors indicate regions where the denoiser exhibits greater uncertainty (i.e., minority regions), serving as a positive reward signal.
Reinforcement Learning with GRPO: The model is fine-tuned using Group Relative Policy Optimization (GRPO).
- Multiple trajectories are sampled from the same prompt and initial noise.
- Group-normalized advantages are computed based on the minority rewards.
- The policy is updated to maximize the likelihood of generating high-reward (minority) samples.
LoRA Integration: To ensure efficiency and stability, updates are applied via a Low-Rank Adaptation (LoRA) adapter rather than full model fine-tuning. This allows for plug-and-play compatibility with various backbones (e.g., Stable Diffusion 1.5 and 2.0).
KL Regularization: To prevent the policy from diverging too far from the pretrained distribution (which could degrade visual fidelity or majority performance), a Kullback–Leibler (KL) divergence term is added to the objective function:
$\mathcal{J}(\theta) = \alpha \mathcal{J}_{Minority}(\theta) + \beta D_{KL}(\pi_\theta \| \pi_{pre})$
This balances minority exploration with the preservation of the original generative prior.
Selective Policy Optimization: The method updates only the early 40% of diffusion timesteps. Empirical analysis suggests that early steps are critical for forming coarse semantic and structural information, and restricting updates to this phase reduces computational cost while maintaining visual fidelity.

3. Key Contributions

Novel Connection: The paper establishes a direct relationship between minority sampling in generation and the perception capability of diffusion classifiers, demonstrating that enhancing generative coverage of low-density regions improves zero-shot classification.
MiPO Framework: A self-improving approach that fine-tunes diffusion models using minority preference rewards derived solely from caption data, eliminating the need for external image data, external reward models (e.g., CLIPScore, ImageReward), or per-prompt optimization.
Efficient Optimization: The integration of GRPO with LoRA and selective timestep updates enables stable, prompt-adaptive minority sampling with negligible inference overhead and a small parameter footprint.
Comprehensive Evaluation: The method is evaluated across five standard datasets, demonstrating improvements in both zero-shot classification accuracy and minority image generation quality.

4. Experimental Results

The authors evaluated MiPO on SD 1.5 and SD 2.0 backbones across multiple datasets:

Zero-Shot Classification:
- CIFAR10: Improved from 85.38% to 87.89% (SD 1.5).
- CIFAR10-C (Robustness): Improved from 57.72% to 59.76% (SD 1.5) and 64.61% to 65.33% (SD 2.0).
- ImageNet-Tiny: Improved from 46.50% to 47.30% (SD 1.5) and 51.45% to 55.25% (SD 2.0).
- Caltech & SUN09: Consistent gains observed across both backbones.
- Note: The paper acknowledges that performance on complex, multi-object datasets like LabelME and VOC2007 was less stable or showed slight degradation, attributing this to domain gaps and annotation inconsistencies.
Minority Generation:
- MiPO achieved comparable or superior performance to the state-of-the-art MinorityPrompt baseline in minority-sensitive metrics (Minority Score, JEPA-SCORE) while maintaining high perceptual quality (CLIPScore, HPSv3).
- Crucially, MiPO operates in a fully prompt-adaptive manner (1s per image) without the heavy per-prompt optimization required by MinorityPrompt (63s per image).
Human Preference:
- MiPO achieved the highest ImageReward score (0.228) compared to the baseline (0.184) and MinorityPrompt (-0.026), indicating a better balance between minority-oriented generation and human-preferred visual quality.

5. Significance and Claims

The paper claims that diffusion classifier perception is inherently biased toward majority regions and that this bias can be alleviated through minority preference optimization. By demonstrating that enhancing the generative coverage of underrepresented regions directly translates to improved recognition capabilities, the authors argue that the "perception" of a diffusion model is inextricably linked to its "generation" capabilities.

The significance of MiPO lies in its ability to improve diffusion classifiers without additional image data or external models, relying instead on a self-improving loop driven by minority rewards. The authors position this as a step toward more robust and equitable generative models, showing that stable, prompt-adaptive minority sampling is achievable through efficient policy optimization (GRPO + LoRA) and that this approach yields tangible gains in zero-shot classification tasks.

The paper remains modest regarding its limitations, noting that the method relies on caption-based self-improvement, which may lead to unstable improvements on datasets with significant domain gaps or ambiguous annotations. Furthermore, it acknowledges an inherent trade-off between maximizing minority distinctiveness and preserving the high-level aesthetic alignment of the pretrained prior.