🤖 machine learning

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging

Any-OPD introduces a novel framework for heterogeneous on-policy distillation between arbitrary latent flow-matching models by bridging them through a frozen, model-agnostic vision representation and continuous noise-level matching, enabling a 2.5B student to rival a 12B teacher's performance where traditional latent regression fails.

Original authors: Siming Fu, Zheming Fu, Ruizhe He, Hualiang Wang, Jie Huang, Xiaoxiao Ma, Mingchen Zhong, Weihu Huang, Xiaoxuan He, Haojun Xu

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Siming Fu, Zheming Fu, Ruizhe He, Hualiang Wang, Jie Huang, Xiaoxiao Ma, Mingchen Zhong, Weihu Huang, Xiaoxuan He, Haojun Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where artists don't just paint with brushes, but with math. This is the realm of AI image generation, a field where computers learn to create pictures from scratch by studying millions of examples. To do this, modern AI models use a clever trick called flow matching. Think of it like a river flowing from a chaotic, noisy waterfall (pure static) down to a calm, clear lake (a perfect image). The AI learns the exact path of that river so it can guide any drop of water from the noise to a beautiful picture.

However, there's a catch. The most powerful AI artists are like giant, slow-moving elephants: they are massive, require supercomputers to run, and take forever to generate a single image. We want the speed and efficiency of a nimble squirrel, but we also want the elephant's artistic genius. The solution is distillation: teaching a small, fast student model to copy the big, slow teacher. But here's the problem: usually, the teacher and student must speak the exact same language, use the same internal map, and follow the same clock. If they are from different "families" of AI, they might as well be speaking different dialects or living in different time zones. Until now, trying to teach a squirrel how to be an elephant when they use completely different blueprints was impossible.

This paper introduces Any-OPD, a new method that acts like a universal translator and a time-bending coach. The researchers found that you don't need the teacher and student to share the same internal maps or clocks to learn from each other. Instead of forcing them to compare their secret internal notes (which would be gibberish to the other), Any-OPD lets them compare the final pictures they create using a third-party "art critic" (a frozen vision model called DINOv2) that understands the meaning of the image, not just the pixels.

The team tested this by taking a tiny, 2.5-billion-parameter student model (SD3.5-Medium) and teaching it to mimic a massive, 12-billion-parameter teacher (FLUX.1-dev). These two models are completely different: they use different internal structures, different ways of handling noise, and different schedules. Standard methods tried to force a direct comparison between their internal data, which caused the training to collapse into a blurry mess. Any-OPD, however, sidestepped this by only comparing the final images in a shared "meaning space."

The results were striking. The tiny student, after being trained with Any-OPD, didn't just get a little better; it became a master artist. It improved its ability to create images that humans prefer (measured by a score called PickScore) from 0.846 to 0.884, and its human preference score (HPSv3) jumped from 9.12 to 10.97. In fact, this small student started to rival, and in some cases even exceed, the performance of its giant 12-billion-parameter teacher, all while running at a fraction of the size and cost.

The paper explicitly rules out the idea that you can simply compare the raw internal data (latents) or the pixel-by-pixel details of the images when the models are different. They showed that trying to do a direct "pixel regression" across different model families causes the training to fail completely. Instead, they proved that by anchoring the student first and then using a noise-level matching system to align their steps in time, you can successfully transfer knowledge between any two models, no matter how different they are. This suggests that in the future, we won't be stuck with just one giant AI; we can take the best teacher available, no matter who built it, and teach any small model we need to deploy to be just as good.

Technical Summary: Any-OPD

Problem Statement

Current on-policy distillation (OPD) for flow-matching and diffusion models relies on a homogeneity assumption: the teacher and student must share the same Variational Autoencoder (VAE) latent space, architecture, and noise schedule (timestep grid). This assumption restricts distillation to pairs within the same model family.

In practice, the most desirable distillation scenarios are heterogeneous: a user may wish to deploy a small, efficient model (e.g., SD3.5-Medium) while leveraging the superior quality of a large, state-of-the-art teacher from a different family (e.g., FLUX.1-dev). Standard OPD fails in this setting due to two fundamental mismatches:

No Shared Space: Teacher latents are invalid targets in the student's coordinate system. Furthermore, pixel-wise regression fails because teacher refinement is a stochastic re-synthesis that preserves semantics but redraws local details; regressing toward this stochastic mean causes training collapse or severe blurring.
No Shared Clock: Different models utilize distinct noise shift functions. Consequently, identical solver indices (step numbers) correspond to different noise levels, making index-aligned supervision pair incomparable states.

Existing methods cannot supervise a student's own trajectory using a teacher that operates in a foreign latent space and noise schedule.

Methodology: Any-OPD

The authors propose Any-OPD (Any Teacher, Any Student, On-Policy Distillation), the first framework enabling on-policy distillation between arbitrary pairs of latent flow-matching generators. The method decouples the teacher and student by referencing neither model's internal parameters, relying instead on a third, external representation and noise-level alignment.

1. Anchoring Phase (Offline)

To address the distribution-level gap, the student is first "anchored" to the teacher's manifold.

The teacher generates images from prompts, which are then re-encoded using the student's VAE.
This projects the teacher's output distribution into the student's latent coordinate system.
The student is trained via standard flow-matching loss on these re-encoded targets. This ensures the student learns to produce images of the "correct kind" before attempting on-policy refinement.

2. On-Policy Distillation Phase

The core distillation loop operates without shared latents or synchronized timesteps:

Teacher as a Black-Box Sampler: The teacher acts purely as a noise-and-denoise projection operator ( $\Pi^\sigma_T$ ). Given a student sample, the teacher re-noises it to a specific noise level $\sigma$ and regenerates a refined version.
Noise-Level Matching (Solving the Clock Gap): Instead of aligning by step index, the method aligns by noise level. After the teacher refines a sample at noise level $\sigma_T^k$ , the gradient is back-propagated only through the student's trajectory steps where the noise level $\sigma_S^j \ge \sigma_T^k$ . This ensures the student learns corrections relevant to the specific noise scale of the refinement.
Representation-Space Bridging (Solving the Space Gap): The student's refined output and the teacher's refined output are decoded into pixels using their respective VAEs. These images are then compared in a frozen, model-agnostic feature space (specifically, the CLS token of a frozen DINOv2).
- The loss is defined as $1 - \cos(f(\hat{x}_S), f(x_{ref}))$ , where $f$ is the frozen extractor.
- This global, coordinate-free comparison is invariant to the local stochastic re-synthesis that defeats pixel or latent regression.

Key Contributions

Formulation of Heterogeneous OPD: The paper identifies and formalizes the two primary obstructions (lack of shared space and shared clock) that prevent existing on-policy methods from working across model families.
The Any-OPD Framework: A novel architecture-agnostic distillation procedure that couples models solely through a frozen external representation space and aligns trajectories via noise-level matching rather than solver indices.
Empirical Validation: The first successful on-policy distillation across distinct model families (FLUX.1-dev $\to$ SD3.5-Medium), demonstrating that a 2.5B student can approach or exceed the performance of a 12B teacher on preference metrics.

Experimental Results

The authors instantiate Any-OPD by distilling FLUX.1-dev (12B parameters) into SD3.5-Medium (2.5B parameters).

Performance Gains: The distilled student significantly outperforms the baseline SD3.5-Medium and approaches or exceeds the 12B teacher on several metrics:
- PickScore: Increased from 0.846 (baseline) to 0.884 (surpassing the teacher's 0.878).
- HPSv3: Increased from 9.12 to 10.97 (approaching the teacher's 11.19).
- ImageReward: Increased from 0.922 to 1.116 (surpassing the teacher's 0.971).
Ablation Findings:
- Latent Regression: Direct latent regression across the VAE boundary causes immediate training collapse.
- Pixel/MSE Loss: Fails due to the stochastic nature of teacher refinement.
- DINOv2 CLS: The only stable and effective objective, confirming that global semantic alignment is required over local pixel/patch alignment.
- Anchoring: Essential for stability; starting OPD without the anchoring phase results in inferior performance.
- Generalization: Swapping the teacher to Z-Image (another distinct family) with identical hyperparameters yields consistent improvements, verifying the method's interchangeability.

Significance and Claims

The paper claims that Any-OPD removes the structural barriers preventing the use of the strongest available generative models as teachers for smaller, deployable models. By treating the teacher purely as a sampler and comparing outputs in a frozen, external feature space, the method allows for the transfer of high-quality generation capabilities across completely different architectures, VAEs, and noise schedules.

The authors emphasize that this decoupling makes the "strongest available generator, whoever builds it," a usable teacher for "whatever model one needs to deploy," effectively democratizing access to state-of-the-art generation quality without requiring architectural homogeneity.