💬 NLP

From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models

This paper introduces **VideoBiasEval**, a diagnostic framework that reveals how alignment tuning—while improving video quality—unintentionally amplifies and stabilizes social biases by propagating stereotypes from human preference datasets through reward models into video diffusion models.

Original authors: Zefan Cai, Haoyi Qiu, Haozhe Zhao, Ke Wan, Jiachen Li, Jiuxiang Gu, Wen Xiao, Nanyun Peng, Junjie Hu

Published 2026-02-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Zefan Cai, Haoyi Qiu, Haozhe Zhao, Ke Wan, Jiachen Li, Jiuxiang Gu, Wen Xiao, Nanyun Peng, Junjie Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to cook. At first, the robot just watches random videos of people in kitchens. It’s a bit messy, and sometimes it shows a person chopping an onion, but other times it shows a person dancing. It’s not "bad," but it’s not very consistent.

To make the robot better, you decide to give it a "Preference Guide." You show it thousands of videos and say, "People liked these videos more because they look professional and clean." This process is called Alignment Tuning.

This paper, "From Preferences to Prejudice," discovers a hidden problem: When we teach the robot to be "better" based on what humans "prefer," we accidentally teach it to be biased.

Here is the breakdown of their discovery using three simple analogies:

1. The "Echo Chamber" Effect (How Bias Travels)

Think of bias like a whisper in a crowded room.

The Whisper (Human Data): When humans rate images, they often unconsciously prefer certain looks (for example, favoring certain ethnicities or genders).
The Megaphone (Reward Models): When we train a "Reward Model" (the robot's guide) on those human ratings, the model doesn't just repeat the whisper; it turns it into a shout. It learns that "High Quality = This specific type of person."
The Broadcast (Video Models): Finally, when the video generator tries to please the "Megaphone," it stops being diverse entirely. It starts producing the same "preferred" types of people over and over again.

The paper shows that alignment doesn't just fix the video quality; it "locks in" the stereotypes.

2. The "Polished Stereotype" (The Danger of Smoothness)

This is the most chilling finding in the paper. Usually, when an AI makes a mistake, it looks like a glitch—a weirdly shaped hand or a flickering face. You can see it's an error.

However, the researchers found that Alignment Tuning acts like a professional makeup artist for stereotypes.

Before alignment, a model might be biased, but it’s "messy" and inconsistent. After alignment, the model becomes much better at making smooth, high-quality, beautiful videos. But because it is now so "good" at following the biased guide, it produces perfectly polished, high-definition stereotypes. It’s no longer a glitch; it’s a smooth, convincing, and "professional-looking" prejudice. This makes the bias much harder to spot and more dangerous because it looks "correct."

3. The "Volume Knob" (Can we fix it?)

The researchers asked: "If we can turn the bias up, can we turn it down?"

They treated bias like a Volume Knob. They created a "Man-Preferred" guide and a "Woman-Preferred" guide.

If they gave the robot the "Man-Preferred" guide, the volume of male representations went up.
If they gave it the "Woman-Preferred" guide, they could actually "counter-steer" the robot and force it to show more women.

The takeaway: They proved that bias in AI isn't a permanent broken part of the machine; it's a setting. We can use this "knob" to intentionally design AI that is more diverse and fair, rather than just letting it follow the accidental prejudices of the internet.

Summary in one sentence:

When we teach AI to follow human "preferences" to make videos look better, we are accidentally teaching it to turn messy human prejudices into smooth, high-definition stereotypes—but we also learn that we can use those same tools to steer the AI toward fairness.

Technical Summary: From Preferences to Prejudice

Title: From Preferences to Prejudice: The Role of Alignment Tuning in Shaping Social Bias in Video Diffusion Models
Published in: Transactions on Machine Learning Research (02/2026)

1. Problem Statement

As video diffusion models advance, a standard post-training technique is alignment tuning (e.g., via RLHF or reward-weighted fine-tuning) to make model outputs better match human preferences. However, these "preferences" are often collected from subjective human judgments that lack demographic diversity. The authors identify a critical risk: these subjective biases in human preference datasets can be inherited by reward models and subsequently amplified and stabilized in video generation models. This creates a pipeline where social stereotypes (gender and ethnicity) are not just present, but are made more visually coherent and temporally persistent, making them harder to detect and more difficult to mitigate.

2. Methodology

The researchers propose a holistic diagnostic framework and an end-to-end analysis pipeline:

VideoBiasEval Framework: A new diagnostic framework designed to disentangle semantic content from social identity.
- Event-Based Prompting: Uses a deterministic template ⟨actor, verb, context⟩. By varying the actor (gender/ethnicity) while keeping the verb and context constant, they isolate how models represent specific identities across 42 socially-associated verbs.
- Multi-Granular Metrics:
  - Ethnicity-Aware Gender Bias (PBSG): Measures the skew between male and female representations within specific ethnic groups.
  - Ethnicity Bias (RDSe & SDI): Uses Representation Deviation Score to measure deviation from uniform distribution and Simpson’s Diversity Index to measure overall ethnic diversity.
  - Temporal Attribute Stability (TAS): A novel metric quantifying how consistently an identity is maintained across video frames (detecting "identity flickering").
- VLM-based Evaluators: Employs an ensemble of Vision-Language Models (Qwen2-VL, InternVL2.5) to automatically annotate social attributes in generated video frames.
End-to-End Pipeline Analysis: The study traces bias through three stages:
1. Human Preference Datasets (HPDv2, Pick-a-Pic).
2. Image Reward Models (HPSv2.0, HPSv2.1, PickScore).
3. Aligned Video Diffusion Models (fine-tuned versions of VideoCrafter-V2).
Controllable Preference Modeling: To test mitigation, they constructed "counter-biased" reward models (Man-Preferred vs. Woman-Preferred) by manipulating the gender distribution in training datasets to see if they could steer video models toward equilibrium.

3. Key Contributions

VideoBiasEval: The first comprehensive framework for evaluating social bias in video generation using event-centric prompting and multi-granular temporal metrics.
End-to-End Bias Tracing: The first study to connect the dots from human preference data $\rightarrow$ reward models $\rightarrow$ aligned video models, proving the propagation of bias.
Discovery of "Bias Stabilization": Identifying that alignment tuning improves temporal coherence but also makes biased portrayals more "stable" and visually authoritative.
Mitigation Strategy: Demonstrating that controllable reward modeling can serve as a tool for intentional debiasing.

4. Key Results

Bias Propagation: Human preference datasets show a strong preference for White representations and non-neutral gender distributions. Reward models (like HPSv2.0) inherit and significantly amplify these biases.
Alignment Amplification: Alignment tuning often worsens gender disparity. For example, aligning with HPSv2.0 increased the male-preference bias (PBSG) in video models.
The Stability Paradox: While alignment improves the Temporal Attribute Stability (TAS) (making videos look smoother), it also "locks in" stereotypes. Biased representations become more temporally consistent, transforming flickering artifacts into stable, systematic distortions.
Diversity Trade-offs: While some alignment methods (like T2V-Turbo) reduce the dominance of White representations, they often do so at the cost of overall ethnic diversity (lower SDI).
Controllability: The researchers successfully demonstrated that using a "woman-preferred" reward model could effectively counteract the "male default" bias found in base video models, steering them toward more equitable gender representation.

5. Significance

This paper shifts the conversation from "how to make videos look better" to "how to make videos represent society fairly." It highlights that aesthetic alignment is not socially neutral. By showing that alignment can act as a "bias stabilizer," the authors warn that high-quality, coherent video generation might actually make harmful stereotypes more insidious and harder to identify. The work provides a technical roadmap for developers to implement bias-aware evaluation and mitigation strategies throughout the entire generative AI lifecycle.