Occupancy Extrapolation: Reaching Many Excited Electronic States from Ground State Calculations

This paper introduces an occupancy extrapolation (OE) method inspired by Landau Fermi liquid theory that efficiently computes accurate valence, Rydberg, and charge-transfer excitation energies at $O(N^3)$ cost by extrapolating from ground-state calculations, thereby avoiding the need for separate $\Delta$SCF computations for each excited state.

The Gist

Imagine you are trying to understand the behavior of a complex machine, like a car engine. Usually, to see how the engine behaves when you press the gas pedal (an "excited" state), you have to build a whole new engine, start it up, and run a separate test for every single speed you want to study. This is slow, expensive, and prone to errors because the engine might stall or behave unpredictably during the test.

This paper introduces a clever shortcut called Occupancy Extrapolation (OE). Instead of building new engines for every test, OE lets you predict how the car will behave at high speeds just by looking at the engine while it's idling (the "ground state").

Here is how it works, broken down with simple analogies:

1. The Problem: The "Stuck" Engine

In the world of atoms and molecules, scientists use a method called $\Delta$SCF to study excited states (like when a molecule absorbs light).

• The Analogy: Imagine trying to balance a broom on your finger. If you want to see how it balances when you tilt it slightly, you have to physically move your hand and try to balance it again.
• The Issue: Doing this for every single tilt (every excited state) is a nightmare. The broom often falls (the calculation crashes), and it takes forever to do it for a whole fleet of brooms (large molecules).

2. The Inspiration: The "Smooth Slope"

The authors were inspired by a theory from physics called Landau Fermi Liquid Theory.

• The Analogy: Imagine a smooth, curved hill. If you know the shape of the hill right at the bottom (the ground state), and you know how steep the hill is, you can mathematically predict exactly how high you will be if you take a few steps up, without actually walking up there.
• The Innovation: The authors realized that the energy of an atom behaves like this smooth hill. They developed a mathematical "map" (a Taylor expansion) that uses the ground state to predict the energy of excited states.

3. The Method: The "Crystal Ball" Calculation

The OE method works like this:

1. Run One Calculation: You calculate the energy of the molecule in its calm, resting state (ground state).
2. Look at the "Slope": The computer calculates how sensitive the energy is to tiny changes in the number of electrons in specific orbits (like how steep the hill is).
3. The Magic Formula: Using a formula that looks at these "slopes" and how they interact with each other, the computer extrapolates (predicts) the energy of the excited states.

The Result: You get the answer for many excited states from just one calculation. It's like predicting the weather for the next week by looking at the barometer right now, rather than waiting a day to see if it rains.

4. What Makes It Special?

• Speed: It is incredibly fast. Instead of running a separate, heavy calculation for every excited state, it does the math in the background of the ground-state calculation. It scales efficiently, meaning it works just as well for a small molecule as it does for a giant protein.
• Accuracy: The paper tested this on three tricky types of "excited" behaviors:
  • Valence: Electrons jumping to nearby seats.
  • Rydberg: Electrons jumping very far away (like a satellite).
  • Charge Transfer: Electrons jumping from one molecule to another.
  • The Result: OE was just as accurate as the slow, heavy methods, but much faster.
• Physical Meaning: It doesn't just give a number; it explains why. It breaks the energy down into "quasiparticles" (like individual players on a team) and their interactions. It tells you that the energy is the sum of the players' individual skills plus how much they like or dislike each other.

5. The Catch (and the Fix)

Like any shortcut, there are limits.

• The Issue: If the "hill" isn't perfectly smooth (due to errors in the mathematical models used for electrons), the prediction might be slightly off. Also, sometimes the math gets "spiky" in areas with very few electrons.
• The Fix: The authors added a "smoothing filter" to handle the spiky parts and used a technique called "spin purification" to ensure the math doesn't get confused between different types of magnetic spins (like making sure you don't mix up a spinning top with a spinning coin).

The Bottom Line

This paper gives scientists a super-efficient crystal ball. Instead of laboriously building and testing every possible excited state of a molecule, they can now use a single, stable calculation to predict how that molecule will react to light, electricity, or chemical changes. This opens the door to simulating massive, complex systems (like solar cells or biological processes) that were previously too expensive or slow to study in detail.

Technical Summary

1. Problem Statement

Electronic excitations are fundamental to spectroscopy, optoelectronics, and materials design. While various theoretical frameworks exist (e.g., TD-DFT, EOM-CC, BSE), the $\Delta$SCF (Delta Self-Consistent Field) method remains a robust approach for calculating excited states by optimizing non-Aufbau electron configurations. However, $\Delta$SCF suffers from three critical limitations:

1. Convergence Issues: Variational orbital optimization often causes excited-state wavefunctions to collapse back to the ground state.
2. Computational Cost: Calculating multiple excited states requires separate, expensive SCF cycles for each state, scaling poorly for large systems.
3. Symmetry Breaking: As a single-determinant method, $\Delta$SCF can break spin and spatial symmetries (e.g., in singlet excitations or diatomic molecules).

The authors aim to retain the physical accuracy of $\Delta$SCF while eliminating the need for individual state-specific SCF calculations, enabling efficient large-scale excited-state simulations directly from a single ground-state calculation.

2. Methodology: Occupancy Extrapolation (OE)

Inspired by Landau Fermi Liquid (LFL) theory, the authors develop the Occupancy Extrapolation (OE) method. The core idea is to treat the system energy as a continuous function of orbital occupancies and use a Taylor expansion to extrapolate from a reference ground state to excited states.

Theoretical Framework

• Energy Functional: The system energy $E_v$ is defined as a function of the non-interacting 1-particle density matrix $\gamma_s$, which depends on orbital occupations $\{n_{q\sigma}\}$.
• Continuous Extension: While $\Delta$SCF typically deals with integer occupations, OE extends this to fractional occupations (zero-temperature ensembles), allowing for a continuous energy surface.
• Taylor Expansion: The energy difference between an excited state and a reference ground state is expanded in terms of occupation fluctuations $\delta n_{q\sigma} = n_{q\sigma} - n^0_{q\sigma}$:
  $$\Delta E = \sum_q \frac{\partial E}{\partial n_q} \delta n_q + \frac{1}{2} \sum_{qr} \frac{\partial^2 E}{\partial n_q \partial n_r} \delta n_q \delta n_r + O(\delta n^3)$$
• Derivatives:
  • First-order derivatives ($\partial E / \partial n$): Correspond to the ground-state Kohn-Sham (or Generalized Kohn-Sham) orbital energies ($\varepsilon$).
  • Second-order derivatives ($\partial^2 E / \partial n^2$): Represent the generalized screened interaction ($\kappa$) between quasiparticles. This term includes the Hartree-exchange-correlation kernel and the density response function.

Physical Interpretation

The method reinterprets excitation energies as sums of quasiparticle energies and their generalized screened interactions:

• Quasiparticles (Electron addition): Adding an electron to an unoccupied orbital.
• Quasiholes (Electron removal): Removing an electron from an occupied orbital.
• Excitation Energy ($\Delta E$): For a 1-particle-1-hole (1p1h) excitation, the energy is the difference between the quasiparticle and quasihole energies, corrected by the attractive screened interaction between them (analogous to the Bethe-Salpeter equation but derived from DFT).

Technical Implementation

• Spin Purification: To address spin symmetry breaking in open-shell singlet states, the authors employ a spin-purification scheme ($E_S = 2E_{BS} - E_T$) using the broken-symmetry singlet and triplet energies.
• Numerical Stability: To avoid singularities in the exchange-correlation kernel (common in low-density regions of local functionals), the authors use analytical curvatures calculated slightly away from integer points with fixed orbitals, avoiding additional SCF cycles.
• Computational Scaling: By utilizing a Sherman-Morrison-based Resolution of the Identity (RI) approximation, the cost is reduced to $O(N^3)$, making it suitable for large-scale simulations.

3. Key Contributions

1. Single-Reference Efficiency: OE allows the calculation of multiple excited states (valence, Rydberg, charge-transfer) from a single ground-state DFT calculation, bypassing the need for separate SCF optimizations for each state.
2. Physical Rigor: It provides a rigorous physical interpretation of excitation energies as quasiparticle energies plus screened interactions, bridging the gap between $\Delta$SCF and many-body perturbation theory (like GW/BSE).
3. Generalization of QE-DFT: The method generalizes the Quasiparticle Energy from DFT (QE-DFT) approach (originally for ionization/electron affinity) to arbitrary neutral excitations.
4. Handling Charge Transfer: Unlike standard TD-DFT which often fails for charge-transfer (CT) states without range-separated hybrids, OE naturally captures the correct long-range Coulomb attraction through the screened interaction term without requiring tuned parameters.

4. Results and Benchmarks

The authors validated OE using the BLYP and B3LYP functionals against high-level theoretical best estimates (TBE) and $\Delta$SCF results for a diverse set of molecules.

• Accuracy:
  • Valence Excitations: OE@BLYP achieved Mean Absolute Errors (MAEs) of 0.55 eV (singlets) and 0.29 eV (triplets), comparable to explicit $\Delta$SCF.
  • Rydberg States: OE@BLYP yielded MAEs of 0.44 eV (singlets) and 0.23 eV (triplets).
  • Charge-Transfer (CT): OE reproduced CT energies with an MAE of 0.30 eV against EOM-CCSDT-3 references, outperforming standard TD-DFT approaches that struggle with CT without tuning.
• Impact of Functionals: Using B3LYP (which includes exact exchange) significantly reduced delocalization errors, lowering the MAE for Rydberg singlets from 0.44 eV (BLYP) to 0.23 eV.
• Orbital Energies: The method successfully predicted excited-state orbital energies (Eqs. 12 & 13) that matched $\Delta$SCF results, confirming the validity of the derivative approach.

5. Significance and Future Outlook

• Scalability: The $O(N^3)$ cost and avoidance of multiple SCF cycles make OE a powerful tool for simulating excited states in large, complex systems where traditional $\Delta$SCF or high-level wavefunction methods are computationally prohibitive.
• Theoretical Bridge: OE unifies the concepts of $\Delta$SCF, Landau Fermi Liquid theory, and Green's function methods (GW/BSE), offering a unified DFT-based perspective on excitations.
• Limitations & Future Work:
  • The method relies on the accuracy of the underlying DFT functional; delocalization errors in approximate functionals remain a source of error, particularly for core excitations and extended systems.
  • It is inherently a single-determinant method and may struggle with states having strong multi-configurational character (though spin purification helps).
  • Future improvements could involve incorporating higher-order Taylor terms and advanced delocalization error corrections (e.g., lrLOSC) to further enhance accuracy for core-level and extended systems.

In conclusion, Occupancy Extrapolation represents a significant advancement in computational chemistry, offering a computationally efficient, physically transparent, and accurate route to excited-state properties directly from ground-state calculations.