💻 computer science

When Physical Preferences Meet Semantic Constraints: Physical and Semantic Direct Preference Optimization for Text-to-Video Generation

This paper introduces Physical and Semantic Direct Preference Optimization (PSDPO), a novel method that resolves the trade-off between physical plausibility and semantic consistency in text-to-video generation by modulating preference pair contributions based on signal agreement, thereby improving physical realism without sacrificing semantic fidelity.

Original authors: Siwei Meng, Yawei Luo, Shu Zhang, Ping Liu

Published 2026-07-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Siwei Meng, Yawei Luo, Shu Zhang, Ping Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to paint pictures based on your spoken descriptions. You tell it, "Draw a cat chasing a laser pointer," and it spits out a masterpiece. But then you ask for something slightly more complex, like "A cat jumping off a table." The robot might draw a cat that defies gravity, floating in mid-air like a ghost, or one that melts into the floor. This is the current state of Text-to-Video generation. These AI models are incredible at making things look real and matching the words you type, but they often struggle with the invisible rules of our universe: physics. They don't always understand that objects have weight, that liquids splash, or that things can't pass through each other.

To fix this, researchers have been using a technique called Direct Preference Optimization (DPO). Think of this like a strict art teacher. You show the teacher two videos: one where a ball bounces realistically, and one where it floats. The teacher says, "I like the bouncing one better." The AI learns from this feedback to make more bouncing balls. However, there's a catch. Sometimes, the "better" video for physics is actually a disaster for the story. The bouncing ball video might be physically perfect, but it might be a red ball when you asked for a blue one, or it might be a dog instead of a cat. The AI, trying to please the teacher's physics lesson, starts ignoring your original request entirely. It becomes a master of physics but a terrible storyteller.

This is the puzzle tackled by a new paper from researchers at the University of Nevada, Reno, and Zhejiang University. They noticed that when AI tries to learn physics from these "this is better than that" comparisons, it often gets confused and starts hallucinating, dropping the semantic meaning (the story) to focus solely on the movement. They call this the "physical-semantic conflict."

The researchers propose a clever solution called PSDPO (Physical and Semantic Direct Preference Optimization). Instead of blindly listening to every piece of feedback, PSDPO acts like a smart filter. It checks the feedback before letting the AI learn from it. If the AI is shown a video that is physically perfect but semantically wrong (like the red ball instead of the blue one), PSDPO says, "Hold on, this lesson is too confusing; let's not learn from this one right now." It weighs the lessons, giving full credit to videos that are both physically correct and match the story, while gently downgrading the ones that only get the physics right but mess up the story.

To make this work, they built a special "judge" system called the Physical Commonsense Evaluation (PCE). This system doesn't just look at the video; it breaks the scene down into steps, checking if the physics makes sense (like does the water flow down?) and if the story matches (is it actually a cup of water?). They found that this relative comparison (Video A vs. Video B) is much more reliable than trying to grade a single video on its own.

The results are promising. When they tested their new method, the AI generated videos that were up to 2 times better at following the laws of physics compared to the standard methods, all while keeping the story exactly as the user asked. They also discovered that the order in which the AI learns matters. If you teach it the hard, confusing lessons too early, it gets lost. But if you start with the clear, easy lessons (where physics and story agree) and only introduce the tricky, conflicting ones later, the AI learns much faster and stays on track.

In short, the paper suggests that by being a bit more selective about which lessons the AI learns from—and by teaching in the right order—we can create video generators that are not only physically realistic but also faithful to the stories we tell them. It's a step toward AI that understands not just how things move, but what things are, ensuring the magic of the story isn't lost in the pursuit of realism.

Technical Summary: Physical and Semantic Direct Preference Optimization for Text-to-Video Generation

1. Problem Statement

Text-to-video (T2V) generation models have achieved significant visual realism but frequently violate basic physical principles. While preference-based alignment methods, particularly Direct Preference Optimization (DPO), have emerged as a promising approach to improve physical plausibility through pairwise comparisons, they introduce a critical failure mode: semantic drift.

The core issue arises from a structural asymmetry in supervision signals:

Physical Plausibility: Typically assessed via relative comparison (determining which of two videos obeys physical laws better). This process does not inherently verify if either video faithfully depicts the prompt's specific objects or scene.
Semantic Consistency: Naturally treated as a per-sample evaluation (checking if a video matches the text).

Standard DPO assumes that the physically preferred sample is also semantically valid. However, empirical analysis reveals that in over 50% of physical preference pairs, the video deemed "physically better" actually violates the prompt's semantic constraints (e.g., depicting a whole lemon instead of a lemon slice, even if the motion is physically accurate). When standard DPO optimizes on these "conflict pairs," it drives the model toward physically plausible but semantically inconsistent solutions, leading to unstable convergence and degraded text-video alignment.

2. Methodology: PSDPO

The authors propose Physical and Semantic Direct Preference Optimization (PSDPO), a framework that treats physical alignment as a constrained optimization problem. PSDPO operates entirely within the standard DPO framework without requiring auxiliary models, additional loss terms, or architectural changes.

2.1 Constrained Formulation

The method formulates the objective as minimizing the DPO loss subject to a semantic feasibility constraint: the physically preferred video ( $y_w$ ) must also maintain semantic alignment scores ( $S_{sem}$ ) greater than or equal to the rejected video ( $y_l$ ).
$\min_{\theta} L_{DPO}(\theta) \quad \text{s.t.} \quad S_{sem}(y_w, x) \geq S_{sem}(y_l, x)$

2.2 Asymmetric Modulation

Instead of hard-filtering out violating pairs (which would discard valuable physical supervision signals), PSDPO introduces a smooth, strictly positive modulation function, $\mathcal{W}(\Delta S)$ , where $\Delta S = S_{sem}(y_w, x) - S_{sem}(y_l, x)$ .

Match Pairs ( $\Delta S \geq 0$ ): The weight is 1.0, recovering standard DPO behavior.
Conflict Pairs ( $\Delta S < 0$ ): The weight is attenuated using an asymmetric exponential function: $\mathcal{W}(\Delta S) = \exp(\Delta S / \tau)$ .

This mechanism proportionally suppresses the gradient contribution of pairs where physical preference conflicts with semantic consistency, bounded by a temperature parameter $\tau$ .

2.3 Gradient Analysis and Drift Control

A gradient-level analysis demonstrates that unconstrained DPO introduces a systematic directional bias (drift) from conflict pairs. PSDPO bounds this drift to a controllable residual. To further mitigate cumulative drift over training, the authors derive a staged optimization protocol:

Stage 1: Train exclusively on "Match Pairs" ( $D_{match}$ ) to establish a stable preference representation with zero drift accumulation.
Stage 2: Introduce "Conflict Pairs" ( $D_{conf}$ ) with modulated weights to extract residual physical signals in a controlled manner.

2.4 Physical Commonsense Evaluation (PCE) System

To generate the necessary preference data, the authors developed a PCE system based on a hierarchical reasoning pipeline (using Qwen3-VL). It combines:

Physical Phenomena Parser: Identifies core physical events from text.
Auxiliary Geometric Metrics: Uses tools like Depth Anything and RAFT to estimate flow magnitude and depth instability.
Hierarchical Reasoning: An MLLM judge that verifies physical laws and metrics without hallucinating, achieving 75.54% agreement with human judgments on relative physical plausibility.

3. Key Contributions

Identification of Structural Failure: The paper identifies that physical-semantic conflict is a systematic tendency in preference-based T2V alignment, not merely a data artifact, caused by the reliance on relative physical comparison without semantic fidelity checks.
PSDPO Framework: A novel constrained preference optimization method that integrates semantic feasibility into the learning objective via a modulation mechanism, requiring no auxiliary models.
Theoretical Guarantees: Gradient-level analysis proving that PSDPO bounds semantic drift from conflicting pairs and a derived staged protocol that provably reduces cumulative drift.
PCE System: A robust evaluation system for constructing physical preference datasets that outperforms absolute scoring and standalone MLLMs in agreement with human judgments.

4. Experimental Results

Experiments were conducted on VBench, VideoPhy-2, and PhyGenBench using the Wan-1.3B model.

Physical Plausibility: PSDPO improves physical plausibility by up to 2× over the baseline on VideoPhy-2 (e.g., improving the "Hard" subset score from 0.039 to 0.078).
Semantic Consistency: Unlike Vanilla DPO, which degrades semantic fidelity, PSDPO maintains or improves semantic scores on VBench (Total score: 84.13 vs. 82.79 for Vanilla DPO).
Comparison with SOTA: PSDPO outperforms existing preference-based methods (VideoDPO, PhysHPO, RDPO) and specialized physics-aware methods (PhysGen, PhyMAGIC) across multiple benchmarks, achieving a better balance between physical realism and semantic alignment.
Ablation Studies:
- Modulation Scale ( $\tau$ ): $\tau=0.02$ provides the optimal balance, significantly recovering semantic adherence while preserving physical learning.
- Staged Training: A switching step ( $T_s$ ) of 1000 steps (in a 3k total step schedule) yields the best results, balancing early stability with later physical signal extraction.

5. Significance and Claims

The paper claims that PSDPO demonstrates that principled formulation can yield both theoretical soundness and practical simplicity. By addressing the structural asymmetry between physical and semantic supervision, the method achieves a more reliable balance than existing approaches.

The authors emphasize that their approach:

Operates entirely within the standard DPO framework.
Requires no auxiliary models, additional loss terms, or changes to the base architecture.
Does not rely on external physical simulators or Chain-of-Thought reasoning during inference (unlike some competing methods).
Successfully resolves the tension between improving physical plausibility and maintaining semantic consistency, a critical step toward generating physically grounded video content from text.