Technical Summary: Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

Problem Statement

On-Policy Distillation (OPD) adapts diffusion models by training a student to match a teacher's local velocity predictions along trajectories generated by the student itself. While modern diffusion systems universally employ Classifier-Free Guidance (CFG) to improve generation quality, the interaction between CFG and OPD remains poorly understood.

Standard CFG composes a guided velocity prediction ($v_{cfg}$) from two separate branches: a positive branch ($v_+$) conditioned on the target prompt and a negative branch ($v_-$) conditioned on a null or negative prompt. The guided velocity is defined as $v_{cfg} = \gamma v_+ + (1-\gamma)v_-$, where $\gamma$ is the guidance scale.

Existing OPD methods typically extend velocity matching to this composed prediction, minimizing the loss $\|v_{cfg}^T - v_{cfg}^S\|_2^2$. The authors identify a critical flaw in this approach: branch-level under-identification. Because the loss constrains only the linear combination of the two branches, the positive and negative branch errors ($e_+$ and $e_-$) can compensate for one another. Specifically, the condition $\gamma e_+ + (1-\gamma)e_- = 0$ admits infinitely many non-zero error pairs.

This ambiguity leads to a failure mode termed Negative Branch Asymmetry (NBA). In scenarios where the teacher possesses "privileged information" in its negative branch (e.g., reference images or dense control signals) that the student lacks, the naive composed objective induces antagonistic dynamics: the training process reduces the positive branch error while simultaneously increasing the negative branch error. While this compensation may yield low loss at the training guidance scale, it causes the student to drift significantly when evaluated at different inference guidance scales, as the learned error compensation no longer holds.

Methodology

To resolve the branch ambiguity and prevent error compensation, the authors propose Positive–Direction Matching (PDM), a branch-aware OPD objective.

1. Positive–Direction Matching (PDM)

Instead of matching the composed guided velocity, PDM separately constrains:

1. The Positive Prediction: Matching the positive branch velocity ($v_+$) directly.
2. The CFG Conditional Direction: Matching the difference vector $d = v_+ - v_-$.

The loss function is formulated as:
$$\mathcal{L}_{PDM} = \|e_+\|_2^2 + \lambda \|d_T - d_S\|_2^2 = \|e_+\|_2^2 + \lambda \|e_+ - e_-\|_2^2$$
where $\lambda > 0$ controls the strength of direction matching. Zero loss in PDM forces both $e_+$ and $e_-$ to zero, eliminating the freedom for errors to cancel each other out.

2. Independent Branch Matching (IBM)

As a baseline to isolate the benefits of branch-level supervision from the specific design of PDM, the authors introduce IBM. This method directly matches the positive and negative branches with weights derived from the expanded naive objective:
$$\mathcal{L}_{IBM} = \gamma^2 \|e_+\|_2^2 + (\gamma - 1)^2 \|e_-\|_2^2$$
Unlike PDM, IBM does not explicitly enforce the conditional direction but removes the cross-branch interaction term present in the naive composed loss.

3. Efficient Supervision for Video

For dense-to-sparse video control applications, the authors propose supervising only the first $K$ states of the student's denoising trajectory rather than the full trajectory. This reduces the computational cost of teacher evaluations while maintaining performance.

Key Contributions

1. Identification of Branch Ambiguity: The paper demonstrates that CFG-composed OPD is under-identified at the branch level. It establishes that this ambiguity is benign under shared negative conditioning (where both teacher and student share the same null condition) but becomes harmful under privileged negative conditioning, leading to NBA.
2. Negative Branch Asymmetry (NBA): The authors define NBA as the failure mode where naive matching reduces positive branch error at the expense of increasing negative branch error, causing excess sensitivity to inference guidance scales.
3. Positive–Direction Matching (PDM): A novel, branch-aware objective that separately supervises the positive prediction and the CFG conditional direction. This prevents cross-branch error compensation and restores identifiable supervision.
4. Empirical Validation: The study provides quantitative evidence across image and video domains, showing that branch-aware supervision improves robustness and knowledge transfer, particularly in settings where the teacher has access to information unavailable to the student.

Experimental Results

Image-Domain Diagnostics

The authors conducted two controlled experiments to diagnose branch-error dynamics:

• Text-Rendering Distillation (Shared Conditioning): Both teacher and student share the same negative condition. Here, naive matching and PDM both result in joint reduction of positive and negative branch errors. The student tracks the teacher's intrinsic CFG sensitivity, confirming that ambiguity is benign in this regime.
• Reference-Conditioned Distillation (Privileged Conditioning): The teacher receives a reference image in its negative branch, while the student does not.
  • Naive Matching: Exhibits antagonistic dynamics, reducing $e_+$ while increasing $e_-$.
  • PDM: Successfully reduces $e_+$ while preventing the growth of $e_-$.
  • Guidance Sensitivity: At inference, naive students trained at $\gamma=2$ degrade significantly when tested at $\gamma=1$ (showing style drift and distortion), whereas PDM students remain stable and closely follow the teacher's behavior across scales.

Dense-to-Sparse Video Control

The method was applied to the Wan-VACE benchmark, where a teacher observes dense per-frame control (pose, depth, scribble) and the student receives only sparse keyframes.

• Control Fidelity: At the training guidance scale ($\gamma=5$), PDM outperformed naive OPD, SFT, and off-policy distillation across all modalities (Pose, Depth, Scribble) in metrics such as MPJPE, PCK, and FVD.
• Guidance-Scale Generalization: When tested at inference scales different from training (e.g., $\gamma=1$ or $\gamma=3$), naive OPD performance degraded sharply. In contrast, PDM and IBM remained stable, demonstrating that branch-aware supervision mitigates the excess sensitivity induced by composed matching.
• Efficiency: Supervising only the first $K=8$ states of the trajectory provided the best trade-off between control fidelity and training cost, achieving results comparable to full-trajectory supervision with significantly reduced computation.

Significance and Claims

The paper claims that the standard practice of matching CFG-composed predictions in on-policy distillation is fundamentally flawed when the teacher and student have asymmetric access to information in the negative branch. The authors argue that Negative Branch Asymmetry is a distinct failure mode that manifests as excessive sensitivity to guidance scale changes.

The significance of this work lies in:

1. Correcting a Theoretical Blind Spot: It identifies that "matching the guided output" is insufficient when the underlying branches are not symmetrically constrained.
2. Robust Knowledge Transfer: By enforcing branch-level constraints (via PDM), the student learns a more faithful representation of the teacher's conditional logic, leading to robust performance across varying inference conditions.
3. Practical Applicability: The proposed method is directly applicable to complex generation tasks like video control, where teachers often possess privileged context (dense frames) that students must learn to approximate from sparse inputs.

The authors modestly note that while PDM empirically outperforms IBM, the theoretical reasons for PDM's specific advantage over other branch-aware formulations (like IBM) remain an open question for future work. The primary contribution is the identification of the failure mode and the demonstration that separating the supervision of the positive branch and the conditional direction resolves it.