Global Minimizers of Sigmoid Contrastive Loss
This paper theoretically characterizes the global minimizers of sigmoid contrastive loss as -Constellations, providing a rigorous explanation for the success of SigLIP models, the origin of the modality gap, and the necessary dimensionality for high-quality representations while proposing an improved reparameterization for training dynamics.