🔢 mathematics

Multidimensional derivative-free optimization. A case study on minimization of Hartree-Fock-Roothaan energy functionals

This study systematically evaluates four derivative-free optimization algorithms for minimizing Hartree-Fock-Roothaan energy functionals involving non-integer Slater-type orbitals, demonstrating their effectiveness in handling the challenging, non-convex landscapes of atomic calculations where analytic derivatives are unavailable.

Original authors: A. Bagci

Published 2026-06-26

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: A. Bagci

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the absolute lowest point in a vast, foggy, and incredibly bumpy landscape. Your goal is to reach the bottom of the deepest valley (the "global minimum") to get the best possible result. In the world of quantum physics, this "landscape" is the energy of an atom, and finding the lowest point means finding the most stable, accurate way the electrons arrange themselves around the nucleus.

Usually, scientists use a map with steepness indicators (gradients) to guide them down the hill. But in this specific study, the author, Ali Bağcı, is dealing with a special kind of terrain where those maps don't exist or are too messy to read. The "hills" are made of mathematical shapes called Slater-type orbitals with non-integer numbers. Think of these as strange, slightly "fuzzy" or "fractional" versions of the standard electron clouds. Because they are so unusual, you can't calculate the slope (derivative) easily.

So, how do you find the bottom of the valley without a slope map? You have to use Derivative-Free Optimization (DFO). The paper tests four different "blindfolded hikers" to see which one is best at finding the bottom of this specific quantum valley.

Here is a breakdown of the four hikers (algorithms) tested:

Powell's Conjugate Direction (The Systematic Explorer):
Imagine a hiker who decides to walk in a set of fixed directions (North, East, South, West) one by one. After walking in all directions, they take a giant step in a new "diagonal" direction that combines their previous moves, hoping to cut across the valley faster. They keep rotating their directions to avoid getting stuck.
- Result: This hiker is great for small, simple valleys but gets tired and confused when the valley gets too complex (high dimensions).
Nelder-Mead Simplex (The Shapeshifting Tent):
Imagine a group of hikers holding hands to form a shape (a triangle in 2D, a tetrahedron in 3D). They look at who is standing at the highest point (the worst energy). They let go of that person and stretch the shape, fold it, or shrink it toward the lowest point. They constantly reshape their "tent" to slide down the hill.
- Result: This was the star performer. It was the most reliable, efficient, and consistent hiker. It found the bottom of the valley quickly and didn't get stuck, even when the terrain got tricky.
Pattern Search (The Grid Walker):
This hiker stands in one spot and takes small steps in every direction (like checking the four corners of a square). If they find a lower spot, they take a bigger step in that direction. If not, they shrink their steps and try again.
- Result: This hiker was very thorough but took a long time. It was like checking every single blade of grass. It worked, but it was slow and computationally expensive.
Model-Based RBF (The Architect):
This hiker doesn't just walk; they build a miniature 3D model of the terrain based on the few spots they've visited so far. They use this model to guess where the bottom is, then go check that spot.
- Result: While smart, this hiker spent so much time building and updating the model that they were the slowest. It was like trying to draw a perfect map of the forest while you are walking through it; the map took too long to make.

The Big Discovery:
The author applied these hikers to calculate the energy of atoms like Helium and Beryllium using these special "fractional" electron clouds. The main finding is that the Nelder-Mead "Shapeshifting Tent" method is the best tool for this specific job.

It managed to find the most accurate energy levels (the deepest valleys) with the least amount of effort. The other methods either got stuck, took too long, or required too much computing power.

Why does this matter?
Usually, scientists use "Gaussian" shapes for electron clouds because they are easy to calculate, but they aren't perfectly accurate near the nucleus. "Slater" shapes are more physically accurate, but they are hard to work with. This paper proves that you can use these more accurate, "fractional" Slater shapes to get better results for atoms, provided you use the right "blindfolded hiker" (Nelder-Mead) to find the solution.

In short: The paper is a race between four different strategies to solve a math puzzle about atoms. The "Shapeshifting Tent" (Nelder-Mead) won the race, proving it's the most effective way to optimize these tricky quantum calculations without needing a slope map.

Technical Summary: Multidimensional Derivative-Free Optimization for Hartree-Fock-Roothaan Energy Minimization

Problem Statement
This study addresses the challenge of minimizing Hartree-Fock-Roothaan (HFR) energy functionals for atomic systems employing noninteger Slater-type orbitals (n*-STOs). In these calculations, the objective function depends on nonlinear orbital parameters ( $\zeta$ ) and principal quantum numbers of noninteger order ( $n^*$ ). A critical difficulty arises because analytic derivatives of the energy functional with respect to these parameters are not readily available. This is due to the non-analyticity of power series expansions for functions with real exponents and the computational complexity of evaluating atomic integrals over n*-STOs, which often require specialized numerical methods (e.g., Hyper-radial functions) rather than closed-form expressions. Consequently, traditional gradient-based optimization techniques are inapplicable, necessitating the use of derivative-free optimization (DFO) methods.

Methodology
The authors implemented and compared four distinct DFO algorithms under identical numerical conditions using the Mathematica programming language. The implementations were explicit, avoiding built-in optimization commands to ensure algorithmic transparency. The methods evaluated were:

Powell's Conjugate Direction (Powell CD): A direct search method utilizing a set of conjugate directions updated via a rotation mechanism, employing line searches (Golden Section and Brent's method) within box constraints.
Nelder-Mead Simplex (NM Simplex): A direct search method evolving a simplex through reflection, expansion, contraction, and shrinkage operations, featuring an automatic restart mechanism to prevent premature convergence.
Coordinate-based Pattern Search (PS-C): A direct search method polling along coordinate directions with adaptively adjusted step sizes, utilizing an opportunistic strategy and acceleration steps.
Model-Based Radial Basis Function (MB-RBF): A surrogate-model approach constructing an interpolant using radial basis functions (RBF) within a trust-region framework. This method utilized adaptive diagonal regularization to ensure numerical stability without polynomial tails.

The algorithms were first benchmarked on the Powell Singular Function (PSF) in 4 and 8 dimensions, a test case known for its Hessian singularity at the global minimum. Subsequently, they were applied to HFR energy minimization for closed-shell atomic configurations, specifically the He-like and Be-like isoelectronic series. The optimization vectors included noninteger principal quantum numbers and orbital exponents, with problem dimensions ranging from 2 to 8 nonlinear parameters.

Key Results

Benchmarking (PSF): In the 4-dimensional PSF test, Powell CD and NM Simplex converged to small residual values with moderate computational cost. PS-C exhibited slower convergence and reduced accuracy. MB-RBF achieved intermediate accuracy but at a significantly higher computational cost due to surrogate model construction. In the 8-dimensional case, the computational cost (function evaluations and CPU time) for PS-C and MB-RBF increased substantially.
Atomic Calculations (Minimal Basis): For He-like and Be-like ions using minimal basis sets, the NM Simplex algorithm consistently required fewer function evaluations and less CPU time than Powell CD and PS-C. While MB-RBF produced acceptable energies, its computational overhead was deemed too high for practical repeated atomic calculations.
Extended Basis and High Dimensions: As the dimensionality increased (e.g., Be atom with 8 orbital parameters), the PS-C method became the primary source of computational expense, leading the authors to exclude it from further high-dimensional comparisons. Powell CD remained effective for lower-dimensional problems but showed increasing sensitivity to tolerance settings as the parameter space grew. NM Simplex maintained stability and yielded converged ground-state energies, though it required slightly more CPU time than Powell CD in specific high-precision extended basis set tests.
Accuracy: The NM Simplex algorithm successfully optimized the 8-parameter configuration for the Be atom, yielding a ground-state energy ( $E = -14.573023164$ a.u.) that agreed with re-optimized analytical Hartree-Fock benchmarks reported in the literature.

Claims and Significance
The paper claims to present the first systematic investigation of derivative-free optimization methods specifically for the minimization of Hartree-Fock-Roothaan energy functionals involving noninteger Slater-type orbitals. The study highlights that:

The Nelder-Mead Simplex algorithm delivers the most consistent and reliable performance among the tested DFO methods for this specific class of problems, balancing convergence speed and computational cost.
While Powell's CD method is effective for lower-dimensional problems, its performance degrades with increasing parameter space size and sensitivity to stopping criteria.
Model-based approaches (MB-RBF) and pattern search variants (PS-C) incur prohibitive computational costs for these specific atomic calculations, limiting their practical utility in this context.
The developed algorithms are transferable to other programming environments and are suitable for larger atomic systems and potentially molecular calculations involving exponential-type basis functions.
The study provides a benchmark for future developments in variational optimization strategies for noninteger Slater-type orbitals, a gap previously unaddressed in the literature.

The work demonstrates that effective treatment of n*-STOs is possible through direct minimization of the energy functional using robust DFO strategies, offering a pathway to approach the Hartree-Fock limit with fewer basis functions by optimizing nonlinear parameters.

More like this