Technical Summary: Condition-Dependent Source Flow Matching (CSFM)

Problem Statement

Flow Matching (FM) has emerged as a powerful alternative to diffusion models for text-to-image (T2I) generation, offering deterministic dynamics and flexibility in source distribution selection. However, despite the theoretical freedom to choose any source distribution, most existing FM systems default to a fixed standard Gaussian source ($N(0, I)$). This choice is largely inherited from diffusion models for simplicity rather than optimization.

The paper identifies a critical gap: while recent works have explored condition-dependent or learned sources, there is limited systematic understanding of:

1. Why learning a source distribution improves the FM objective in high-dimensional, strongly conditioned regimes.
2. How to implement source learning stably in modern T2I architectures without causing training collapse or instability.
3. When source learning yields the most significant performance gains, particularly regarding the geometry of the target representation space.

Existing methods that modify source-target coupling (e.g., Optimal Transport) struggle in high-dimensional conditional settings, while other learned source approaches lack guidance on training dynamics and regularization.

Methodology: Condition-Dependent Source Flow Matching (CSFM)

The authors propose CSFM, a framework that jointly learns a condition-dependent source distribution $p_\phi(X_0 | C)$ and the velocity field $v_\theta$ under the flow-matching objective.

1. Adaptive Source-Target Coupling

Instead of a fixed condition-agnostic coupling $\pi(X_0, X_1, C) = p_0(X_0)p_1(X_1, C)$, CSFM introduces a learnable coupling $\pi_\phi(X_0, X_1, C) = p_\phi(X_0 | C)p_1(X_1, C)$. A source generator $g_\phi$ maps the condition $C$ to the parameters of a conditional Gaussian source:
$$p_\phi(X_0 | C) = \mathcal{N}(\mu_\phi(C), \text{diag}(\sigma^2_\phi(C)))$$
This allows the source to adapt its location and support based on the conditioning signal, theoretically reducing the intrinsic variance term in the FM loss, which characterizes the ambiguity of velocity supervision.

2. Stabilization Mechanisms

The authors identify two primary obstacles to joint training: source collapse (variance shrinking to zero) and weak supervision signals in tightly integrated architectures. They propose two specific components to address these:

• Variance-Only Regularization ($\mathcal{L}_{VarReg}$): Standard KL regularization toward a standard normal prior constrains both the mean and variance, forcing the source mean $\mu_\phi(C)$ toward the origin and restricting its ability to align with target modes. CSFM employs a variance-only regularization that penalizes deviations of $\sigma^2_\phi(C)$ from unit variance while leaving $\mu_\phi(C)$ unconstrained. This prevents collapse while allowing the source to relocate toward condition-compatible target regions.
• Directional Source-Target Alignment ($\mathcal{L}_{align}$): In complex T2I settings, the flow model's ability to modulate the vector field with conditioning can weaken the learning signal for the source generator. To counteract this, the authors introduce a negative cosine similarity loss that encourages directional alignment between the learned source sample $X_0$ and the target $X_1$. This provides specific guidance without overly restricting the magnitude or stochastic support of the source.

The total training objective is:
$$\mathcal{L}_{total} = \mathcal{L}_{FM} + \lambda_{VarReg}\mathcal{L}_{VarReg} + \lambda_{align}\mathcal{L}_{align}$$

Key Contributions

1. Theoretical Insight (Why): The paper connects source learning to the reduction of the intrinsic variance term in the FM objective. By learning a condition-dependent source, the method reduces path intersections and velocity ambiguity, leading to cleaner supervision, lower gradient variance, and straighter transport paths.
2. Practical Framework (How): It proposes a stable training recipe for modern T2I systems involving variance-only regularization and directional alignment, enabling end-to-end training without constraining source mobility or causing collapse.
3. Conditional Effectiveness (When): The authors demonstrate that the benefits of source learning are contingent on the geometry of the target representation space. Gains are most pronounced in semantically structured latent spaces (e.g., RAE with DINOv2) where conditions induce compact clusters, whereas gains diminish in entangled spaces (e.g., SD-VAE) where conditions correspond to highly multimodal distributions.

Experimental Results

The authors evaluated CSFM across multiple benchmarks, architectures, and scales:

• Convergence Speed: On ImageNet-1K (256×256), CSFM achieved up to 3.01× faster FID convergence and 2.48× faster CLIP Score convergence compared to standard FM with a fixed Gaussian source.
• Performance Metrics: Using the UnifiedNextDiT backbone and RAE (DINOv2) representation, CSFM achieved an FID of 2.453 and a CLIP Score of 0.3420, outperforming the Gaussian baseline (FID 3.036) and other condition-dependent source/coupling methods (e.g., CrossFlow, C2OT, CPD).
• Flow Geometry: Few-step generation experiments showed that CSFM degrades more gracefully as sampling steps decrease (e.g., from 50 to 3 steps), indicating straighter flow fields and fewer path intersections.
• Robustness: The method demonstrated consistent improvements across different conditioning architectures (LightningDiT, MMDiT, UnifiedNextDiT) and text encoders (CLIP, Qwen3).
• Scaling: When scaled to a 1.3B parameter model on GenEval and DPG-Bench, CSFM maintained superior performance over the Gaussian baseline, achieving 0.80 on GenEval and 81.11 on DPG-Bench.

Significance and Claims

The paper claims that treating the source distribution as a learnable, condition-dependent design choice is a critical, yet underutilized, component of modern flow matching. By addressing the specific challenges of high-dimensional conditional generation—specifically through variance-only regularization and directional alignment—CSFM provides a principled way to improve training efficiency and generation quality.

The authors emphasize that the effectiveness of this approach is not universal but is tightly linked to the representational capacity of the target space. In structured latent spaces where conditioning signals create clear semantic clusters, learning the source yields substantial gains. The work suggests that future advancements in generative modeling should consider the joint optimization of source and target representations rather than relying on fixed priors inherited from diffusion models.