🔢 mathematics

A quadratic Grassmann manifold optimization problem arising from quantum embedding methods

This paper analyzes a non-convex quadratic optimization problem on the Grassmann manifold arising in quantum embedding methods, demonstrating that its global minimizer can be found via an auxiliary convex problem in specific cases and that the auxiliary solution serves as a highly effective initialization for Riemannian optimization and self-consistent field algorithms when direct convexity does not apply.

Original authors: Thomas Ayral, Eric Cancès, Fabian M. Faulstich, Lin Lin, Alicia Negre

Published 2026-03-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Thomas Ayral, Eric Cancès, Fabian M. Faulstich, Lin Lin, Alicia Negre

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding the Perfect "Shadow"

Imagine you are a quantum chemist trying to understand a giant, complex molecule (like benzene). The molecule is made of many atoms, and the electrons are dancing around them in a chaotic, high-dimensional space. Trying to calculate the behavior of every electron at once is like trying to predict the weather for every single water molecule in the ocean simultaneously. It's impossible.

To solve this, scientists use a trick called Quantum Embedding. They say, "Let's just focus on one small piece of the molecule (a 'fragment') and pretend the rest of the universe is a simple 'bath' of water that interacts with it."

The problem this paper solves is: How do we design the perfect "bath" so that our small piece of the molecule behaves exactly like it would in the real, giant molecule?

The Mathematical Puzzle: The Grassmann Manifold

In the math world, this "bath" is represented by a special kind of matrix (a grid of numbers) called a projector. You can think of this projector as a spotlight.

The Goal: We need to find the perfect angle and shape for this spotlight to illuminate the right electrons.
The Constraint: The spotlight must be "rank-m," meaning it can only shine on a specific number of electrons (let's say $m$ ).
The Shape: The set of all possible spotlights forms a weird, curved shape called a Grassmann Manifold. Imagine a giant, multi-dimensional sphere where every point on the surface is a different way to aim your spotlight.

The scientists want to find the lowest point in a valley on this curved surface. This "lowest point" represents the most accurate, lowest-energy state for the molecule.

The Problem: The "Hilly" Landscape

Usually, finding the lowest point on a curved surface is hard because the landscape is full of fake valleys (local minima).

The Trap: If you roll a ball down the hill, it might get stuck in a small dip that looks like the bottom, but isn't the true bottom.
The Old Way: Scientists used to use algorithms (like the "Roothaan algorithm") that just roll the ball downhill. Sometimes, the ball gets stuck in a fake valley, and the simulation gives a wrong answer.

The Paper's Solution: The "Magic Mirror" (Convexification)

The authors discovered a clever trick. They realized that while the original problem (finding the spotlight) is a bumpy, confusing hill, there is a sibling problem that looks exactly the same on the surface but is perfectly smooth underneath.

The Original Problem (The Bumpy Hill): Minimizing the energy on the Grassmann manifold. It's full of traps.
The Sibling Problem (The Smooth Bowl): They created a new, "convex" function. Imagine turning the bumpy hill into a perfect, smooth bowl.
- The Magic: If you solve the smooth bowl problem, the solution you find is often the exact same solution as the bumpy hill.
- The Safety Net: Even if the smooth bowl doesn't give the perfect answer immediately, it gives you a perfect starting point. If you take that solution and roll it back onto the bumpy hill, you are so close to the bottom that you can't get stuck in a fake valley anymore.

The "Aufbau Principle": The Rule of the Lowest Seats

The paper also proves a rule called the Aufbau Principle (borrowed from chemistry).

The Metaphor: Imagine a theater with $M$ seats. The "cost" of sitting in a seat is determined by the matrix $G$ .
The Rule: To get the best (lowest energy) result, you must fill the $m$ cheapest seats. You never skip a cheap seat to sit in an expensive one.
Why it matters: This rule guarantees that the "ball" rolling down the hill will eventually settle into a stable position, rather than bouncing around chaotically.

Real-World Test: The Benzene Molecule

The authors tested their theory on Benzene (a ring of 6 carbon atoms).

They tried to build the "bath" for different sizes.
The Result:
- When the bath was small, their "Smooth Bowl" trick found the perfect answer instantly.
- When the bath was large, the "Smooth Bowl" gave a very good guess. When they used that guess to start the "bumpy hill" search, the algorithms converged (found the answer) much faster and more reliably than before.

Summary: Why This Matters

It's a Map: The paper provides a map for navigating a confusing mathematical landscape (the Grassmann manifold) that quantum chemists use every day.
It Avoids Traps: It offers a way to avoid getting stuck in "fake valleys" (local minima) that ruin simulations.
It's a Booster: Even when the trick doesn't give the final answer directly, it acts as a "turbo boost" for existing algorithms, making them faster and more accurate.

In a nutshell: The authors found a way to turn a treacherous, bumpy mountain climb into a smooth slide, ensuring that scientists can find the true "bottom of the valley" for their quantum simulations without getting lost.

1. Problem Statement

The paper addresses the minimization of a specific quadratic functional $J(P)$ over the real Grassmann manifold $\mathcal{M} = \text{Gr}(m, \mathbb{R}^M)$ , which represents the set of rank- $m$ orthogonal projectors in $\mathbb{R}^M$ . The optimization problem is defined as:

$\min_{P \in \mathcal{M}} J(P) = \text{Tr}(BP) - \frac{1}{2}\text{Tr}(APAP)$

where:

$A, B \in \mathbb{R}^{M \times M}_{\text{sym}}$ are symmetric matrices.
$A \succeq 0$ (positive semidefinite).
The manifold constraint is $P^2 = P$ and $\text{Tr}(P) = m$ .

Context: This problem arises naturally in Density Matrix Embedding Theory (DMET), a quantum embedding method used in computational chemistry and physics. Specifically, it corresponds to the bath construction step, where one seeks to find a subspace (the "bath") that minimizes the entanglement between a local fragment and its environment, effectively decoupling the system for efficient many-body calculations.

2. Methodology and Mathematical Analysis

The authors employ a multi-faceted approach combining differential geometry, convex analysis, and numerical linear algebra.

A. Mathematical Properties

Optimality Conditions: The paper derives first and second-order optimality conditions using Riemannian geometry. The Riemannian gradient is identified as the projection of the Euclidean gradient onto the tangent space of the Grassmann manifold.
The Aufbau Principle (Theorem 2): A key theoretical result is that any global minimizer $P_\star$ satisfies an "Aufbau principle." This means $P_\star$ is the orthogonal projector onto the subspace spanned by the eigenvectors of the matrix $G(P_\star) = B - AP_\star A$ corresponding to its $m$ smallest eigenvalues. This property is analogous to the Hartree-Fock problem but is not guaranteed for general quadratic Grassmann problems.
Convexification (Theorem 4): The authors introduce a convex relaxation of the problem. They define an auxiliary convex function $\tilde{J}(D)$ $\tilde{J} (D)$ over the convex hull of the manifold, $\mathcal{K} = \text{CH}(\mathcal{M}) = \{D \mid 0 \preceq D \preceq I, \text{Tr}(D)=m\}$ $K = CH (M) = {D ∣ 0 ⪯ D ⪯ I, Tr (D) = m}$ :
$\tilde{J}(D) = \text{Tr}(CD) + \frac{1}{4}\|[A, D]\|^2$
where $C = B - \frac{1}{2}A^2$ $C = B - \frac{1}{2} A^{2}$ .
- Crucially, $\tilde{J}(P) = J(P)$ for all $P \in \mathcal{M}$ .
- The gradient of $\tilde{J}$ at any minimizer $D_\star$ of the convex problem yields a matrix $H_\star = \nabla \tilde{J}(D_\star)$ that is independent of the specific minimizer found.
- Gap Condition: If the $m$ -th and $(m+1)$ -th eigenvalues of $H_\star$ are distinct ( $\mu_m < \mu_{m+1}$ ), then the unique minimizer of the convex problem lies on the manifold $\mathcal{M}$ and is the global minimizer of the original non-convex problem.

B. Special Cases

Commuting Matrices: If $A$ and $B$ commute, the problem reduces to the diagonalization of the matrix $C = B - \frac{1}{2}A^2$ .
Perturbation Analysis: The paper analyzes cases where $[A, B]$ is small, showing that the solution is a real-analytic perturbation of the commuting case.
DMET Specifics: For the DMET bath construction, the matrices $A$ and $B$ are derived from blocks of the global 1-RDM ( $\gamma$ ). The authors derive necessary and sufficient conditions for "full disentanglement" (where the minimum energy is zero), linking the required bath dimension to the spectral properties of $\gamma$ .

C. Numerical Algorithms

The paper evaluates three classes of numerical methods:

Riemannian Optimization: Direct minimization on the Grassmann or Stiefel manifolds using trust-region algorithms (e.g., via manopt.jl).
Self-Consistent Field (SCF) / Roothaan Algorithm: An iterative scheme based on the Aufbau principle: $P_{k+1} = \text{projector of lowest } m \text{ eigenvectors of } (B - AP_k A)$ $P_{k + 1} = projector of lowest m eigenvectors of (B - A P_{k} A)$ .
- Key Finding: Unlike the Hartree-Fock Roothaan algorithm, which can oscillate, the Roothaan algorithm for this specific quadratic problem always converges to a critical point (Proposition 10).
Convexified Approach (ODA): Using the Optimal Damping Algorithm (ODA) to solve the convex problem (4).
- If the spectral gap condition holds, this yields the global solution directly.
- If the gap does not hold, the solution $D_\star$ serves as an excellent initialization for Riemannian methods, significantly outperforming standard initial guesses.

3. Key Results

Global Optimality via Convexification: The paper establishes that solving the convex relaxation is sufficient to find the global minimizer of the non-convex problem provided a spectral gap exists in the gradient of the convexified objective.
Convergence of SCF: The authors prove that the Roothaan algorithm applied to this specific quadratic cost function is globally convergent to a critical point, a stronger result than typically found in general non-linear eigenvalue problems.
Local Minima Existence: Through low-dimensional examples ( $2\times2$ and $3\times3$ ), the authors demonstrate that the problem is non-convex and admits local minima that are not global. They show that standard SCF algorithms can get trapped in these local minima depending on the initialization, whereas the convexified approach (or ODA) provides a robust path to the global minimum or a superior starting point.
Benzene Molecule Benchmark: Numerical tests on the benzene molecule ( $C_6H_6$ $C_{6} H_{6}$ ) using CCSD-level data confirm the theory:
- For small bath dimensions ( $m \le 5$ ), the convex gap condition holds, and the ODA finds the global minimum.
- For larger dimensions ( $m \ge 6$ ), the gap closes, and the convex solution is not on the manifold. However, using the convex solution as an initialization allows Riemannian trust-region methods to reach the global minimum, whereas standard SCF with a naive initialization gets stuck in local minima (plateaus).

4. Significance and Contributions

Theoretical Rigor: The paper provides the first rigorous mathematical analysis of this specific quadratic Grassmann optimization problem, which had previously been unstudied in the mathematical literature despite its importance in quantum chemistry.
Algorithmic Improvement: It offers a practical strategy to overcome the non-convexity of DMET bath construction. By leveraging the convex relaxation, practitioners can avoid the "local minimum trap" that plagues standard SCF iterations.
Bridge between Fields: The work connects Riemannian optimization, convex analysis, and quantum embedding methods, demonstrating how convex relaxations can be used to initialize non-convex solvers in high-dimensional physics problems.
DMET Robustness: The results suggest that the "full disentanglement" condition in DMET can be systematically checked and achieved by analyzing the spectral properties of the convexified gradient, potentially improving the reliability of quantum embedding simulations for strongly correlated systems.

In summary, the paper transforms a challenging non-convex optimization problem arising in quantum chemistry into a tractable framework by identifying a convex surrogate that either solves the problem directly or provides a near-optimal initialization, thereby significantly enhancing the robustness of DMET calculations.