Quadratic polarity and polar Fenchel-Young divergences from the canonical Legendre polarity

This paper establishes a unified framework linking quadratic polarities to deformed Legendre transformations via linear algebra on homogeneous coordinates, defines polar divergences that generalize Fenchel-Young and Bregman divergences, and elucidates the reference duality in information geometry through total polar Fenchel-Young divergences.

The Gist

Imagine you are standing in a vast, multi-dimensional room. In this room, you have two ways of looking at the world: the "Shape" view (looking at objects like hills and valleys) and the "Shadow" view (looking at the walls and boundaries that define those shapes).

This paper is about a magical mirror that connects these two views. It explains how to translate a "shape" (a mathematical function) into its "shadow" (its dual form) and back again, using a specific type of reflection called Polarity.

Here is the breakdown of the paper's ideas in simple, everyday language:

1. The Magic Mirror: What is Polarity?

In geometry, Polarity is like a rule that swaps points for lines (or in higher dimensions, points for flat planes).

• The Analogy: Imagine you have a lighthouse (a point). The light it casts creates a shadow on the wall. In this paper, the "shadow" isn't just a dark spot; it's a specific flat wall (a hyperplane) that defines the boundary of the light.
• The Paper's Twist: The authors show that this "swapping" isn't just a random trick. It's a fundamental law of geometry that can be done using simple math (linear algebra) on a special grid of numbers.

2. The Legendre-Fenchel Transform: The "Dual" Personality

You might know that a function (like a hill) can be described by its height at every point. But in math, there's a famous way to describe that same hill by looking at the slopes of its sides instead of its height. This is called the Legendre-Fenchel Transform.

• The Analogy: Think of a hill.
  • View A (Primal): "At this spot, the hill is 100 meters high."
  • View B (Dual): "At this spot, the slope is 45 degrees."
• The paper explains that the Legendre transform is actually just a Polarity operation. If you take the "shape" of the hill and reflect it through this magical mirror, the resulting "shadow" is exactly the hill described by its slopes.

3. The Big Discovery: Deforming the Mirror

The authors discovered something very cool: You don't always have to use the standard mirror (the Legendre polarity). You can use any mirror, as long as it's a "quadratic" one (a specific mathematical shape).

• The Analogy: Imagine you have a funhouse mirror.
  • Option A: You can keep the mirror straight but stretch the hill before you put it in front of it.
  • Option B: You can keep the hill straight but bend the mirror itself.
• The Result: The paper proves that these two options produce the exact same result. Whether you deform the object or deform the mirror, the "dual" relationship remains consistent. This means we can use simple tools (matrices) to handle very complex shapes.

4. Measuring the Gap: Fenchel-Young Divergence

In machine learning and statistics, we often need to measure how "different" two things are. This is called a Divergence.

• The Analogy: Imagine you have a point on the hill and a point on the shadow-wall. The Fenchel-Young Divergence is the distance between them.
• The Paper's Contribution: They defined a new way to measure this distance using the polarity mirror.
  • Key Property: It's fair. If you swap the point on the hill with the point on the wall, the distance measurement stays the same (or transforms predictably). This is called Reference Duality. It's like saying, "It doesn't matter if I measure the distance from the hill to the wall, or from the wall to the hill; the relationship is symmetric."

5. The "Total" Distance: Normalizing the View

Sometimes, the raw distance isn't enough because the "wall" might be tilted or far away in a weird way. The paper introduces a Total Fenchel-Young Divergence.

• The Analogy: Imagine you are measuring the distance from a point to a wall, but the wall is slanted. If you just measure the straight line, it might be misleading. You need to "normalize" it—like adjusting for the angle of the sun so the shadow length is accurate.
• The Result: This new "Total" distance is actually the same as a famous tool used in medical imaging and data science called the Total Bregman Divergence. The paper shows that this complex tool is just a "normalized" version of their new polarity distance.

Why Does This Matter?

This paper is like finding a universal translator for geometry and data science.

1. Simplification: It shows that complex, curved problems can be solved using simple straight-line math (linear algebra) if you look at them through the right "polarity" lens.
2. Unification: It connects three big ideas:
   • Projective Geometry (the study of shapes and shadows).
   • Convex Analysis (the study of hills and valleys).
   • Information Geometry (the study of how data points relate to each other).
3. New Tools: It gives scientists a new way to build algorithms for Optimal Transport (moving mass efficiently, like shipping logistics) and Machine Learning by treating these problems as simple reflections in a high-dimensional room.

In a nutshell: The authors took a complex mathematical concept (duality), showed that it's just a geometric reflection (polarity), proved that you can bend the mirror or the object to get the same result, and used this to create better ways to measure distances between data points.

Technical Summary

Here is a detailed technical summary of the paper "Quadratic polarity and polar Fenchel-Young divergences from the canonical Legendre polarity" by Frank Nielsen, Basile Plus-Gourdon, and Mahito Sugiyama.

1. Problem Statement

The paper addresses the need to unify and generalize the Legendre-Fenchel transformation (a cornerstone of convex analysis, information geometry, and optimal transport) through the lens of projective geometry and polarity.

While the Legendre-Fenchel transform is well-understood as a conjugation operation between convex functions, its geometric interpretation via polarity (a reciprocal duality mapping points to hyperplanes) is often treated separately. Furthermore, generalized transformations of the Legendre transform (involving affine deformations) and the relationship between standard divergences (like Bregman) and their "total" or normalized counterparts (Total Bregman) lack a unified geometric framework. The authors aim to:

1. Formalize the Legendre-Fenchel transform as a specific instance of a quadratic polarity in projective space.
2. Demonstrate that generic quadratic polarities can be reduced to the canonical Legendre polarity via affine transformations.
3. Define a new class of Polar Fenchel-Young divergences that generalize standard divergences and recover key properties like non-negativity and duality.
4. Show that Total Bregman divergences arise naturally as normalized polar divergences.

2. Methodology

The authors employ a geometric approach rooted in projective geometry and homogeneous coordinates.

• Projective Embedding: They represent the epigraph of a function $F: \mathbb{R}^n \to \mathbb{R}$ as a set in $\mathbb{R}^{n+1}$ and embed it into the projective space $\mathbb{P}^{n+1}$ using homogeneous coordinates in $\mathbb{R}^{n+2}$. A point $(\theta, F(\theta))$ becomes $[\theta, F(\theta), 1]^\top$.
• Polarity Definition: A polarity $\Delta_C$ is defined by a non-degenerate cost matrix $C \in GL(n+2)$. It maps a set $A$ to its polar set $\Delta_C(A) = \{ [b] \mid \forall [a] \in A, [a]^\top C [b] \geq 0 \}$.
• Canonical Legendre Polarity: They identify a specific matrix $C_L$ (the Legendre polarity matrix) such that the boundary of the polar of a function's graph coincides with the graph of its convex conjugate.
• Transformation Analysis: The paper analyzes how arbitrary quadratic polarities relate to the canonical Legendre polarity through affine transformations ($T$ and $S$) acting on the convex bodies or the polarity itself.
• Divergence Construction: They define divergences based on the inner product induced by the polarity matrix and introduce a normalization factor (conformal factor) to derive "total" divergences.

3. Key Contributions

A. Legendre-Fenchel Transform as Polarity

The authors prove that the Legendre-Fenchel transform is geometrically equivalent to the Legendre polarity.

• Proposition 2: The boundary of the Legendre polarity of the graph of a function $F$ coincides exactly with the graph of its convex conjugate $F^*$.
• This establishes that the transform is not just an algebraic operation but a geometric duality between a convex body and its polar in projective space.

B. Equivalence of Quadratic Polarities

The paper establishes that any generic quadratic polarity can be expressed in terms of the canonical Legendre polarity via affine deformations.

• Theorem 1: An arbitrary quadratic polarity $\Delta_C$ is equivalent to applying an affine transformation $T$ to the result of the Legendre polarity: $\Delta_C(A) = T(\Delta_L(A))$.
• Theorem 2: Alternatively, $\Delta_C$ is equivalent to applying the Legendre polarity to a deformed convex body: $\Delta_C(A) = \Delta_L(S(A))$.
• Proposition 6: Provides the explicit algebraic relationship between the transformation matrices $T$ and $S$ derived from the cost matrix $C$.
• Significance: This allows complex generalized Legendre transforms to be manipulated efficiently using linear algebra on $(n+2) \times (n+2)$ matrices acting on homogeneous coordinates.

C. Polar Fenchel-Young Divergences

The authors define a new divergence measure based on the polarity.

• Definition 2 (Polar Fenchel-Young Divergence): For a convex set $A$ and a point $[b]$ in its polar $\Delta_L(A)$, the divergence is defined as $D_A(a:b) = [a]^\top C_L [b]$.
• Property 7: This definition recovers the standard Fenchel-Young divergence (and equivalently the Bregman divergence) when $A$ is the epigraph of a convex function.
• Property 9 (Reference Duality): The divergence satisfies a swap property: $D_A(a:b) = D_{\Delta_L(A)}(b:a)$. This geometrically explains the "reference duality" in information geometry ($BF(\theta_1:\theta_2) = BF^*(\eta_2:\eta_1)$).

D. Total Polar Fenchel-Young Divergences

The paper connects the concept of Total Bregman Divergence (which includes a normalization factor) to the polar framework.

• Definition 3: The total polar divergence is defined by normalizing the standard polar divergence by a conformal factor $\kappa(b)$, which represents the norm of the normal vector of the polar hyperplane in the affine plane.
• Theorem 3: The authors prove a duality identity for these total divergences involving the conformal factors of both the primal and dual points:
  $$\frac{1}{\kappa^*(a)} tD_A(a:b) = \frac{1}{\kappa(b)} tD_{\Delta_L(A)}(b:a)$$
  This provides a new geometric derivation of the duality of Total Bregman divergences.

4. Results

1. Geometric Unification: The Legendre transform, generalized Legendre transforms, Fenchel-Young divergences, and Total Bregman divergences are all unified under the single framework of quadratic polarity in projective space.
2. Computational Efficiency: The use of homogeneous coordinates and $(n+2) \times (n+2)$ matrices allows these transformations to be computed using standard linear algebra operations, simplifying the handling of complex deformations.
3. Duality Recovery: The "reference duality" (swapping parameters between primal and dual spaces) is shown to be a direct consequence of the symmetry of the polarity matrix and the definition of the polar set.
4. Optimal Transport Connection: The paper notes that the $c$-transform in optimal transport (where $c$ is a quadratic cost) corresponds directly to a quadratic polarity, bridging information geometry and optimal transport theory.

5. Significance

• Theoretical Insight: The paper offers a profound geometric reinterpretation of convex duality. By viewing the Legendre transform as a polarity, it reveals that the structure of convex conjugation is inherent to the projective geometry of the function's epigraph.
• Generalization: It provides a rigorous framework for understanding and constructing generalized Legendre transforms, which are crucial in modern machine learning (e.g., Fenchel-Young losses, Hopfield networks).
• Algorithmic Implications: The matrix-based formulation suggests that operations involving these divergences and transforms can be optimized using linear algebra libraries, potentially improving the efficiency of algorithms in information geometry and optimal transport.
• New Perspective on Total Divergences: The derivation of Total Bregman divergences as normalized polar divergences offers a fresh perspective on why the conformal factor is necessary and how it preserves duality properties.

In summary, this paper successfully bridges the gap between abstract projective geometry and practical convex analysis, providing a powerful, unified language for describing duality, transformations, and divergences in high-dimensional spaces.