Critical point search and linear response theory for computing electronic excitation energies of molecular systems. Part I: General framework, application to Hartree-Fock and DFT

This paper presents a unified Kähler manifold framework that systematically derives linear response equations for computing electronic excitation energies across various variational models, offering a streamlined alternative to traditional methods like Casida's derivation for Hartree-Fock and DFT.

The Gist

The Big Picture: Finding the "Next Step" in a Molecule

Imagine you are trying to understand a molecule (a tiny cluster of atoms). You know how it behaves when it's calm and resting (its ground state). But what happens when you hit it with a flash of light? It gets excited, jumps to a higher energy level, and starts vibrating or changing shape. This is an excited state.

Calculating exactly what happens when a molecule gets excited is one of the hardest puzzles in chemistry. It's like trying to predict exactly how a complex piece of origami will unfold if you poke it in a specific spot.

This paper introduces a new, unified way to solve this puzzle. The authors, Laura Grazioli, Yukuan Hu, and Eric Cancès, are saying: "We have two different ways to guess the answer, but they are actually looking at the same mountain from different sides. Let's use a special map (mathematics) to see how they connect and where they might lead us astray."

---

The Two Main Strategies: Climbing vs. Shaking

The paper compares two main methods scientists use to find these excited states. Think of them as two different ways to explore a hilly landscape.

1. The "Climbing" Method (Critical Point Search - CP)

The Analogy: Imagine you are a hiker on a giant, foggy mountain range. You want to find the highest peaks (excited states).

• How it works: You start at the bottom (the ground state) and try to climb up to the next highest peak. You keep looking for "summits" (critical points) that are higher than the valley you are in.
• The Problem: The mountain is tricky. Sometimes, you might climb a small hill that looks like a peak but isn't actually a real mountain top. Or, you might find a "saddle point" (a pass between two mountains) and mistake it for a peak. In the math world, these are called spurious critical points—fake answers that look real but don't correspond to a real physical state.
• The Paper's Insight: The authors show that while this method can find higher, more complex excited states (like climbing a double-hump mountain), it is prone to finding these "fake peaks" that confuse the results.

2. The "Shaking" Method (Linear Response - LR)

The Analogy: Imagine you are standing at the very bottom of the valley (the ground state). Instead of trying to climb a mountain, you gently shake the ground.

• How it works: You give the system a tiny nudge and listen to how it vibrates. Every object has a natural frequency at which it likes to vibrate (like a guitar string). The paper explains that the energy required to excite the molecule is directly related to these natural vibration frequencies.
• The Advantage: This method is very stable and rarely gives you "fake" answers. It's like listening to a bell ring; you know exactly what note it is.
• The Limitation: It only tells you about the first few vibrations (single excitations). It's great for a simple nudge, but it can't easily predict what happens if you smash the molecule with a sledgehammer (complex, multi-step excitations).

---

The Secret Weapon: The "Kähler Manifold" Map

The authors use a fancy mathematical tool called a Kähler manifold. Don't let the name scare you.

The Analogy: Think of a Rubik's Cube.

• A Rubik's Cube has many possible positions (states).
• Some positions are "solved" (ground state).
• Some are "scrambled" (excited states).
• The "Kähler manifold" is like a perfect, transparent 3D map of every possible way you can twist the cube.

The authors use this map to show that both the "Climbing" method and the "Shaking" method are just different ways of navigating this same map.

• Climbing is looking for specific coordinates on the map where the energy stops changing.
• Shaking is looking at how the map curves around the bottom point to predict vibrations.

By using this map, the authors derived a new, simpler way to write the equations for the "Shaking" method (Linear Response). It's like finding a shortcut through a maze that everyone else has been walking around for decades.

---

The Experiment: Testing the Methods

To prove their theory, the authors ran computer simulations on simple molecules like Hydrogen ($H_2$) and Water ($H_2O$). They played a game of "What if?" by turning the interaction between electrons up and down (like turning a volume knob).

What they found:

1. When the volume is low (weak interaction): Both the "Climbing" and "Shaking" methods give the exact same answer. They agree perfectly with the "Gold Standard" (Full Configuration Interaction, or FCI), which is the most accurate but expensive way to calculate things.
2. When the volume is high (strong interaction): The methods start to disagree.
   • The "Shaking" method (LR) stays close to the Gold Standard. It's reliable.
   • The "Climbing" method (CP) starts to wander off. It finds "fake peaks" (spurious states) that don't exist in reality.

The H4 Molecule Lesson:
They tested a molecule made of four Hydrogen atoms in a rectangle. They found that the "Climbing" method found three different "peaks" (saddle points).

• One of them was a real excited state.
• The other two were ghosts—mathematical illusions created by the complexity of the method.
• If a chemist just looked at the "Climbing" results without checking, they might think the molecule has three excited states when it only has one.

---

The Takeaway: Why This Matters

This paper is a guidebook for chemists and physicists.

• For the "Shaking" method (Linear Response): The authors have provided a cleaner, more systematic way to derive the equations. It's like giving them a better blueprint for building a house.
• For the "Climbing" method (Critical Point Search): They are issuing a warning. "Be careful! Just because you found a peak on the map doesn't mean it's a real mountain. You might be looking at a ghost."

In simple terms:
If you want to know how a molecule reacts to a gentle tap, use the Shaking method. It's safe, accurate, and easy to understand.
If you want to know what happens during a violent crash, you might need the Climbing method, but you have to be very careful to filter out the fake answers it produces.

The authors promise that in their next papers (Parts II and III), they will apply this same "map" to even more complex and powerful methods used in modern chemistry, helping scientists avoid these mathematical traps in the future.

Technical Summary

1. Problem Statement

Computing excited states of many-body quantum Hamiltonians is a fundamental challenge in computational physics and chemistry. Current state-of-the-art methods generally fall into two distinct categories:

1. Variational Methods (Critical Point Search - CP): These involve finding critical points (minima or saddle points) of energy functionals on differentiable manifolds. This includes state-specific and state-averaged approaches.
2. Linear Response (LR) Methods: These compute excitation energies as resonant frequencies of the linearized dynamics around the ground state (e.g., LR-HF, LR-TDDFT, EOM-CC).

While these methods coincide for the exact Full Configuration Interaction (FCI) solution, they diverge significantly when approximations (like Hartree-Fock or DFT) are introduced. A key issue is that nonlinear models (like HF) can possess "spurious" critical points that do not correspond to physical excited states. Furthermore, the derivation of LR equations for nonlinear models (typically via Casida's equations) is often complex and lacks a unified geometric perspective. The paper aims to provide a unified theoretical framework to compare these approaches, analyze their differences, and understand the nature of critical points in nonlinear approximations.

2. Methodology

The authors employ the Kähler manifold formalism to create a unified geometric framework for both CP and LR approaches.

• Kähler Manifold Framework: The state space of the electronic system is modeled as a Kähler manifold $\mathcal{M}$ (a complex manifold with compatible Riemannian and symplectic structures).
  • Hamiltonian Dynamics: The time evolution is defined as $\frac{dx}{dt} = J_x \text{grad}_{\mathcal{M}} E(x)$, where $J_x$ is the symplectic operator and $\text{grad}_{\mathcal{M}} E$ is the Riemannian gradient.
  • Critical Point Search (CP): Excited states are identified as critical points $x_k$ where $\text{grad}_{\mathcal{M}} E(x_k) = 0$. The excitation energy is $\omega_k^{CP} = E(x_k) - E(x_0)$.
  • Linear Response (LR): Excitation energies are derived from the linearized dynamics around the ground state $x_0$. The eigenfrequencies are the symplectic eigenvalues of the Riemannian Hessian $\text{Hess}_{\mathcal{M}} E(x_0)$.
• Application to Grassmann Manifolds: The authors specialize this framework to the complex Grassmann manifold $\text{Gr}_{\mathbb{C}}(r, n)$, which represents the space of density matrices (or projectors) of rank $r$ in an $n$-dimensional space.
  • FCI Case ($r=1$): The energy functional is linear. The authors prove that CP and LR yield identical results, recovering the exact Schrödinger equation eigenvalues.
  • Mean-Field Case ($r=N$): For Hartree-Fock (HF) and adiabatic TDDFT, the energy functional is nonlinear. The authors derive the LR equations directly from the Hessian on the manifold, providing a systematic alternative to Casida's derivation.
• Perturbative Analysis: In the weakly-interacting regime (parameterized by a coupling constant $\eta$), the authors perform a theoretical comparison using perturbation theory. They expand the excitation energies up to the first order in $\eta$ for FCI, CP-HF, and LR-HF to analytically determine where the methods diverge.

3. Key Contributions

• Unified Formalism: The paper establishes a rigorous geometric link between CP and LR using Kähler manifolds. It demonstrates that LR excitation energies are intrinsically the symplectic eigenvalues of the Riemannian Hessian at the ground state.
• Systematic Derivation of LR Equations: The authors provide a direct, geometric derivation of the linear response equations for mean-field models (HF/TDDFT). This serves as a simpler, more systematic alternative to the traditional Casida derivation, naturally handling the symplectic structure of the problem.
• Theoretical Comparison in Weak Coupling:
  • For FCI, CP and LR are identical.
  • For Unrestricted HF (UHF), the authors prove that while both methods coincide at $\eta=0$ (non-interacting limit), their first-order derivatives with respect to the interaction strength differ.
  • Crucially, the LR approach yields the correct first-order perturbation result (matching FCI), whereas the CP approach yields an incorrect first-order derivative due to the nonlinear nature of the UHF energy functional.
• Identification of Spurious States: The study highlights that in nonlinear models, the CP approach can generate multiple critical points (saddle points) that do not correspond to physical excited states.

4. Numerical Results

The authors validate their theoretical findings through numerical simulations on $H_2$, $H_4$ (linear and rectangular), and $H_2O$ using the UHF method with various basis sets (STO-3G, 3-21G, etc.).

• Verification of Derivatives: Numerical finite-difference calculations of the first-order excitation energy derivatives at $\eta=0$ match the analytical predictions derived in Section 4.
• Comparison with FCI:
  • In the weakly interacting regime ($\eta \to 0$), LR-HF results converge to the FCI results, confirming the theoretical prediction that LR captures the correct first-order physics.
  • CP-HF results deviate from FCI even at first order.
  • As interaction strength ($\eta$) increases to physical values ($\eta=1$), LR-HF generally remains closer to FCI than CP-HF.
• Spurious Critical Points (H4 Example):
  • For the $H_4$ molecule in a rectangular geometry, the CP search identifies multiple index-1 saddle points (critical points with one negative Hessian eigenvalue).
  • By projecting these UHF critical points onto the exact FCI basis, the authors show that:
    • One critical point corresponds to a physical excited state (a symmetry-broken combination of singlet and triplet states).
    • Other critical points are spurious, arising purely from the nonlinearity of the UHF approximation and having no physical counterpart in the FCI spectrum.
  • This confirms that not all critical points found by CP are valid excited states.

5. Significance

• Theoretical Clarity: The work demystifies the relationship between variational and response theories, showing they are two sides of the same geometric coin (Hamiltonian dynamics on Kähler manifolds).
• Methodological Insight: It provides a robust justification for why Linear Response theory is often preferred over direct Critical Point Search for calculating excitation energies in mean-field theories: LR correctly captures the perturbative behavior of the system, whereas CP can introduce errors and spurious solutions due to the nonlinearity of the functional.
• Foundation for Future Work: This paper (Part I) sets the stage for Parts II and III, which will extend this framework to more advanced methods like CASSCF and Coupled Cluster (CC), aiming to resolve the issue of spurious critical points in those contexts.
• Practical Implication: For practitioners, the results suggest that while CP methods can access orbital relaxation and higher excitations, they require careful validation (e.g., checking Morse indices and overlaps with reference states) to avoid misinterpreting spurious saddle points as physical excited states.

In summary, the paper successfully unifies the mathematical treatment of excited states, proves the superiority of Linear Response over Critical Point Search in the perturbative regime for mean-field theories, and numerically demonstrates the existence of non-physical solutions in nonlinear variational searches.