Excited-state Properties Beyond the Excitation Energy from Orbital-Optimized Density Functional Calculations II: Absorption Spectra

This study extends Löwdin's formalism for nonorthogonal determinants to orbital-optimized density functional calculations within the projector augmented-wave framework, demonstrating that while the method qualitatively reproduces absorption spectra and peak intensities for single-determinant excited states, it struggles with multi-configurational states and shows no systematic improvement from exact exchange or self-interaction corrections.

The Gist

The Big Picture: Taking a Snapshot of a Dancing Molecule

Imagine a molecule as a tiny, complex dance troupe. When you shine light on it, the dancers (electrons) jump to higher energy levels, changing their dance moves. Scientists want to predict two things about this jump:

1. The Energy: How much energy does it take to make the jump? (Like the height of a jump).
2. The Brightness: How bright is the flash of light when they jump? (This is called the "oscillator strength" or "absorption intensity").

For a long time, computer programs were good at predicting the height of the jump but often got the brightness wrong. This paper introduces a new way of calculating these jumps called Orbital-Optimized (OO) Density Functional Theory.

The Problem: The "Non-Orthogonal" Dance Floor

In standard computer models, scientists usually assume that the "ground state" (the resting dance) and the "excited state" (the jumping dance) are completely separate and don't overlap, like two people standing in different rooms. This makes the math easy.

However, this new "Orbital-Optimized" method is more realistic. It lets the electrons rearrange themselves specifically for the jump. The problem is that this rearrangement means the "resting" state and the "jumping" state are no longer in separate rooms; they are now in the same room, slightly overlapping.

The Analogy: Imagine trying to measure the distance between two people who are hugging. If you assume they are standing apart, your measurement is wrong. Because these states "hug" (overlap), calculating how bright the light flash will be becomes very tricky. Previous studies mostly checked if the jump height was right, but they didn't check if the brightness calculation was accurate when the states were overlapping.

What They Did: Testing the New Method

The researchers took this new method and tested it on a small group of molecules (water, ammonia, formaldehyde, methanol, and ethylene). They wanted to see:

1. Does this method predict the brightness of the light flash correctly?
2. Does it matter what "tools" (mathematical basis sets) they use to do the calculation?
3. Does changing the "rules of the game" (the mathematical formulas used to describe electron behavior) fix the brightness errors?

The Results: A Mixed Bag

Here is what they found, broken down simply:

1. The "Toolbox" Matters (Basis Sets)
Think of the "basis set" as the resolution of a camera.

• Low Resolution (Simple tools): If you use a simple camera, you miss the fine details of the "diffuse" electrons (those that float far away from the molecule, like a Rydberg state). The brightness calculation is a bit off.
• High Resolution (Complex tools): When they used very detailed tools (like Plane Waves or double-augmented sets), the results for the brightness became much more consistent.
• Takeaway: To get the brightness right, you need a high-resolution camera, especially for those electrons that float far away.

2. The "Rules" Don't Fix Everything (Exchange-Correlation Functionals)
Scientists tried changing the "rules" of the simulation (using different mathematical formulas like PBE, PBE0, or adding corrections for self-interaction).

• The Result: Changing the rules helped fix the height of the jump (energy) in some cases, but it did not consistently fix the brightness.
• Analogy: Imagine you are trying to fix a blurry photo. You tried changing the lens, the lighting, and the filter. Sometimes the photo got sharper, but often the brightness was still wrong. There was no single "magic rule" that fixed the brightness for every molecule.

3. The Real Culprit: The "Solo" vs. The "Group" (Single vs. Multi-Configurational)
This is the most important finding. The method worked great when the excited state was like a solo dancer (a single, clear configuration).

• Solo Dancers: For simple jumps (like in Ammonia), the method predicted the brightness perfectly.
• Group Dancers: For complex jumps where the electron dance is a mix of many different possibilities at once (multi-configurational), the method failed to predict the brightness correctly.
• The Specific Failures: They found big errors in water, formaldehyde, and ethylene. In these cases, the excited state is a messy mix of different dance moves. Because the computer model forces the state to look like a single, clean "solo" move, it gets the brightness wrong.
• The Overlap Issue: They checked if the "hugging" (overlap) between the ground and excited states was causing the error. They found that even when they changed the overlap, the brightness error stayed the same. So, the overlap wasn't the main problem; the "solo vs. group" nature of the dance was.

Comparison with the Old Way (LR-TDDFT)

They compared their new method to the standard method (LR-TDDFT):

• Standard Method: Good at predicting the brightness of complex "group" dances (like the bright flashes in ethylene), but bad at predicting the energy of the "solo" dancers that float far away (Rydberg states).
• New Method (OO): Great at predicting the energy of the "solo" dancers (Rydberg states), but struggles with the brightness of the complex "group" dances.

The Bottom Line

This paper shows that the new "Orbital-Optimized" method is a powerful tool for predicting how molecules absorb light, but with a catch:

• It works very well for simple, single-dance moves (Rydberg states).
• It struggles when the dance is a complex mix of many moves (multi-configurational states).
• Simply changing the mathematical formulas or the "camera resolution" doesn't fix the errors for the complex dances. To fix those, we would need a method that can handle complex group dances, not just solo ones.

In short: The method is a great step forward for understanding the "height" of electron jumps and the brightness of simple jumps, but it still needs help to accurately predict the brightness of the most complex, chaotic electron dances.

Technical Summary

Technical Summary: Excited-State Properties Beyond the Excitation Energy from Orbital-Optimized Density Functional Calculations II: Absorption Spectra

Problem Statement
Orbital-optimized (OO) density functional calculations offer a state-specific approach to describing excited states by employing variationally optimized orbitals for each state, rather than treating them as linear responses of the ground state. While this approach has been shown to improve excitation energies, particularly for Rydberg states where standard linear-response time-dependent density functional theory (LR-TDDFT) often fails due to incorrect asymptotic potential behavior, it introduces significant complications for calculating transition properties. Because each electronic state is described by a distinct set of molecular orbitals, OO states are generally nonorthogonal. This nonorthogonality complicates the evaluation of transition dipole moments (TDMs) and oscillator strengths. Previous benchmarks have largely focused on excitation energies or restricted oscillator strength calculations to the first excited state (HOMO–LUMO transition). Consequently, the performance of OO methods for oscillator strengths across a broader spectrum of valence and Rydberg excitations, and the impact of nonorthogonality on these properties, remain poorly understood.

Methodology
This study extends the application of OO density functional calculations to the computation of absorption spectra for a set of small molecules (H₂O, CH₂O, NH₃, CH₃OH, and C₂H₄), covering both valence and Rydberg states up to 10 eV.

• Theoretical Framework: The authors employ a direct orbital optimization scheme to locate excited states as stationary points (saddle points) of the energy functional, avoiding variational collapse. To handle the nonorthogonality of the resulting states, Löwdin's formalism for nonorthogonal determinants is extended to the projector augmented-wave (PAW) framework. This allows for the rigorous calculation of matrix elements between nonorthogonal Slater determinants.
• Implementation: The methodology is implemented in the GPAW software. Transition matrix elements are derived within the PAW formalism, accounting for the transformation between smooth pseudo-wavefunctions and all-electron wavefunctions. The formalism includes corrections for the overlap matrix and one-electron operators (specifically the dipole operator) within the augmentation spheres.
• Computational Details: Calculations were performed using various exchange-correlation (xc) functionals: the generalized gradient approximation (PBE), the hybrid functional (PBE0), and PBE with explicit Perdew–Zunger self-interaction correction (SIC) using global scaling factors ($\alpha = 1$ and $\alpha = 1/2$). Both plane-wave (PW) and linear combination of atomic orbitals (LCAO) basis sets (aug-cc-pVDZ and d-aug-cc-pVDZ) were utilized within the PAW framework. Results were compared against high-level multireference reference data (MS-CASPT2, CCSD, CCSD(T), and CCSDT) and LR-TDDFT calculations.
• Spin Purification: For open-shell singlet states, energies and TDMs were obtained using spin-purification formulas derived from mixed-spin and triplet calculations.

Key Contributions

1. Extension of Löwdin's Rules to PAW: The paper derives and implements Löwdin's rules for nonorthogonal determinants specifically within the PAW formalism, enabling the calculation of transition properties for OO states using plane-wave and grid-based representations.
2. Systematic Benchmarking: The study provides a comprehensive assessment of OO oscillator strengths for multiple excited states (beyond the HOMO–LUMO transition) across a diverse set of molecules, comparing different basis sets and xc functionals.
3. Analysis of Nonorthogonality: The work explicitly investigates whether the nonzero overlap between ground and excited states in OO calculations introduces systematic errors in oscillator strengths, a concern raised in previous literature but not fully resolved for full absorption spectra.

Results

• Basis Set Dependence: Oscillator strengths for diffuse Rydberg states are sensitive to the basis set representation. Singly augmented LCAO basis sets (aug) often overestimate excitation energies and yield inaccurate oscillator strengths for highly diffuse states compared to doubly augmented (d-aug) and plane-wave (PW) representations. However, bright valence transitions are largely insensitive to the basis set.
• Functional Dependence: Changing the xc functional (PBE, PBE0, PBE-SIC/2, PBE-SIC) alters excitation energies and specific oscillator strengths but does not produce a systematic hierarchy of accuracy for intensities across the entire dataset. While SIC improves the asymptotic potential, it does not consistently correct oscillator strength errors.
• Single- vs. Multi-Configurational Character: A clear distinction emerges based on the electronic character of the excited states.
  • Single-Configurational States: For states dominated by a single determinant (e.g., many Rydberg states in ammonia), OO calculations yield excellent agreement with reference oscillator strengths and peak positions.
  • Multi-Configurational States: For states with strong multi-configurational character or significant configuration mixing (e.g., H₂O S3/S4, CH₂O S5, C₂H₄ S1), OO calculations exhibit substantial discrepancies. Specifically, the single-determinant description fails to capture the correct intensity distribution, leading to overestimation or underestimation of oscillator strengths that persists across all tested functionals.
• Nonorthogonality and Overlap: Analysis reveals no clear correlation between the magnitude of the ground-state/excited-state overlap and the error in oscillator strengths. For instance, reducing the overlap via hybrid functionals or SIC did not systematically improve the oscillator strength for problematic states like H₂O S3. Thus, nonorthogonality is not identified as the primary source of error for the largest deviations.
• Comparison with LR-TDDFT: OO methods generally outperform LR-TDDFT in describing Rydberg state energies and intensities. Conversely, LR-TDDFT often provides a better description of bright valence transitions with strong configuration mixing, where the single-determinant OO approach struggles.

Significance and Conclusions
The paper concludes that while OO density functional calculations are a robust tool for predicting excitation energies and absorption spectra of single-configurational Rydberg states, their quantitative reliability for oscillator strengths is limited by the single-determinant nature of the Kohn–Sham description. The method performs well when the excited state is adequately described by a single dominant configuration but fails for states with strong multi-configurational character, regardless of the choice of xc functional or basis set.

The authors assert that the large errors observed in oscillator strengths for specific transitions (e.g., water S3/S4, formaldehyde S5, ethylene S1) are intrinsic to the single-determinant approximation rather than artifacts of nonorthogonality or basis set incompleteness. Consequently, for systems where multi-configurational character is significant, OO-KS spectra should be interpreted qualitatively (e.g., for peak positions) rather than quantitatively regarding intensities. The study suggests that future improvements would require extending the OO framework to explicitly treat multi-configurational excited states, rather than relying solely on functional or basis set modifications.