Preconditioned Score and Flow Matching
This paper identifies that the ill-conditioned covariance of intermediate distributions in flow matching and score-based diffusion causes optimization bias and stagnation, and proposes reversible preconditioning maps to reshape this geometry, thereby enabling continued progress along suppressed directions and yielding better-trained models.