H-NESSi: The Hierarchical Non-Equilibrium Systems Simulation package

H-NESSi is an open-source software package that enables efficient, long-time simulations of strongly correlated quantum systems out of equilibrium by combining high-order time-stepping with hierarchical low-rank compression techniques to overcome the prohibitive computational scaling of conventional Kadanoff-Baym equation solvers.

The Gist

Imagine you are trying to predict the weather, but instead of clouds and rain, you are tracking the chaotic dance of trillions of electrons inside a piece of material. This is the job of a physicist studying "nonequilibrium" systems—materials that are being jolted, heated, or shocked by lasers or electric fields, forcing them out of their comfortable, resting state.

The paper introduces a new software tool called H-NESSi (Hierarchical Non-Equilibrium Systems Simulation). Think of H-NESSi as a super-smart, ultra-efficient traffic controller for these electrons.

Here is the breakdown of the problem and how H-NESSi solves it, using simple analogies:

1. The Problem: The "Memory Explosion"

In the past, simulating these electron dances was like trying to write down the position of every single car on a highway at every single second.

• The Old Way: To predict the future, you had to remember every interaction that happened since the beginning of time. If you wanted to simulate 100 seconds, you needed a notebook that grew so huge it would fill a library.
• The Bottleneck: As time went on, the memory required to store this data grew cubically (like $10^3$, $100^3$, $1000^3$). It was like trying to fill a swimming pool with a teaspoon; eventually, the computer ran out of memory and crashed. This meant scientists could only simulate very short bursts of time (femtoseconds), missing out on the interesting long-term effects (like new superconducting states).

2. The Solution: The "Hierarchical Compression"

H-NESSi solves this by realizing that not every detail of the electron dance is unique. Many patterns repeat, and much of the data is redundant.

• The Analogy: Imagine you are describing a movie to a friend.
  • The Old Way: You describe every single frame, every pixel, and every shade of color for the entire 2-hour movie. It takes forever and requires a massive hard drive.
  • The H-NESSi Way: You realize that for 90% of the movie, the background is just a static blue sky. You don't need to describe the blue sky 10,000 times. You just say, "Blue sky, repeated." You only write down the details when the actors move or the plot changes.
  • The Tech: H-NESSi uses a technique called Hierarchical Low-Rank Compression. It breaks the massive data table into smaller blocks. If a block looks "simple" (repetitive), it compresses it into a tiny summary. If a block is "complex," it keeps the details. This is like using a zip file for your data, but doing it intelligently in real-time as the simulation runs.

3. The "Time-Travel" Trick (Kadanoff-Baym Equations)

The math behind this involves the Kadanoff-Baym equations.

• The Metaphor: Imagine a river. To know where the water is now, you need to know where it was yesterday, and the day before, and how the rocks (impurities) affected it.
• The Innovation: H-NESSi doesn't just look at the river; it uses a hierarchical map. It knows that the water flow from 100 years ago doesn't affect the ripples right now as much as the water from 1 second ago. It focuses its computing power on the "recent past" and compresses the "distant past," allowing it to simulate the river for much longer without drowning in data.

4. The "Thermal" Starter (Imaginary Time)

Before the simulation starts, the material is usually hot (in thermal equilibrium).

• The Analogy: Think of this as warming up a car engine before a race.
• The Tool: H-NESSi uses a method called the Discrete Lehmann Representation (DLR). Instead of checking the engine temperature at every single tiny fraction of a second, it picks the most important "checkpoints" to get an accurate reading. This lets the simulation start perfectly from a hot, stable state without wasting time on unnecessary calculations.

5. Why This Matters (The Results)

The paper shows that H-NESSi can simulate systems much larger and for much longer than before.

• The Superconductor Test: They simulated a superconductor (a material that conducts electricity with zero resistance) being hit by a laser pulse. H-NESSi could track how the material reacted for a long time, revealing new states of matter that were previously impossible to see because the computers would have run out of memory.
• The Speed: While old software took hours to simulate a few seconds of time, H-NESSi can simulate minutes or hours of "electron time" on the same hardware. It scales much better, meaning adding more computers makes it faster without hitting a wall.

Summary

H-NESSi is a revolutionary software package that allows scientists to simulate the chaotic behavior of quantum materials over long periods.

• Before: Simulating was like trying to carry a mountain of sand in a paper bag. It was heavy, messy, and limited to small amounts.
• Now: H-NESSi is like a magic vacuum that sucks the air out of the sand, compressing it into a tiny, manageable brick without losing any of the sand's shape or weight.

This allows researchers to finally answer big questions: What happens to a material if we hit it with a laser for a long time? Can we create new types of superconductors? H-NESSi gives us the superpower to watch these movies play out in full, high-definition, without the computer crashing.

Technical Summary

1. Problem Statement

The simulation of strongly correlated quantum systems out of equilibrium is a major challenge in computational physics. The standard theoretical framework for these problems is the Nonequilibrium Green's Function (NEGF) theory, specifically the Kadanoff–Baym equations (KBE).

• Computational Bottlenecks: Conventional two-time formulations of the KBE suffer from severe scaling limitations:
  • Time Complexity: $O(N_t^3)$, where $N_t$ is the number of time steps. This cubic scaling restricts simulations to very short timescales (femtoseconds), whereas many physical phenomena (e.g., transient superconductivity, metastable phases) occur on picosecond to nanosecond scales.
  • Memory Growth: $O(N_t^2)$, requiring massive amounts of RAM for long propagation times.
• Existing Limitations:
  • Approximate methods (e.g., Generalized Kadanoff–Baym Ansatz) reduce cost but often lose accuracy in strongly correlated regimes.
  • Exact methods using memory-cutting or tensor trains often fail for long-range memory or suffer from convergence issues at long times.
  • Standard implementations (like the NESSi package) cannot handle large lattice systems or long-time dynamics due to the cubic/quadratic scaling.

2. Methodology

The authors present H-NESSi, an open-source C++ package that solves the KBE using Hierarchical Off-Diagonal Low-Rank (HODLR) compression techniques combined with high-order time-stepping schemes.

Core Algorithmic Innovations

• HODLR Compression:
  • The retarded ($G^R$) and lesser ($G^<$) Green's functions are represented as lower-triangular matrices.
  • These matrices are hierarchically partitioned into blocks. Off-diagonal blocks are approximated using Truncated Singular Value Decomposition (TSVD).
  • Singular values below a user-defined threshold ($\epsilon_{SVD}$) are discarded. This exploits the fact that off-diagonal blocks of Green's functions often have low numerical rank, allowing for data compression.
  • Adaptive Rank Selection: The rank of each block is determined dynamically based on the compression threshold, ensuring controllable accuracy without fixed approximations.
• High-Order Time Stepping:
  • The KBE are integro-differential equations. H-NESSi uses high-order integration schemes (up to 6th order) to approximate derivatives and history integrals.
  • The integration class supports error scaling of $O(h^{k+1})$, allowing for larger time steps while maintaining precision.
• Discrete Lehmann Representation (DLR):
  • Imaginary-time quantities (Matsubara Green's functions and mixed components) are discretized using the DLR.
  • This allows for a compact, accurate representation of thermal initial states using a sparse grid of points, scaling as $O(\log(\Lambda)\log(1/\epsilon))$, rather than a dense equidistant grid.
• Multiorbital Support:
  • The code handles systems with multiple orbitals (spin, orbital, lattice site) by storing $N_o^2$ separate compressed representations, where $N_o$ is the number of orbitals.

Parallelization Strategy

• Hybrid MPI + OpenMP:
  • MPI (Distributed Memory): Used for lattice calculations with translational invariance. The Brillouin zone is partitioned across MPI ranks. Green's functions are communicated to evaluate self-energies (which depend on the global Brillouin zone), and self-energies are redistributed.
  • OpenMP (Shared Memory): Used within each MPI rank to parallelize time steps, DLR nodes, and orbital loops.
• Communication Efficiency: The mpi_comm class manages complex data transposition between "time-local" buffers (all time points for local $k$) and "momentum-local" buffers (all $k$ points for local time), minimizing communication overhead.

3. Key Contributions

1. Algorithmic Breakthrough: The first robust implementation of HODLR compression specifically tailored for the Kadanoff–Baym equations in a multiorbital, non-equilibrium context.
2. Reduced Complexity:
   • Time Complexity: Reduced from $O(N_t^3)$ to approximately $O(N_t^{2+\alpha})$, where $\alpha$ depends on the growth of the $\epsilon$-rank (typically $\alpha \approx 0.5$ for superconducting states and $\alpha \approx 0$ for normal states).
   • Memory Complexity: Reduced from $O(N_t^2)$ to $O(N_t \cdot N_\epsilon^S)$, where $N_\epsilon^S$ is the number of retained singular values.
3. User-Friendly Interface: Designed to mirror the existing NESSi package, allowing users to transition to H-NESSi with minimal code modifications. It transparently handles compression, so users interact with standard matrix interfaces.
4. Robust Self-Consistency: Implements a stable iterative scheme for solving the nonlinear KBE, including specific strategies for bootstrapping (initial steps) and handling density matrix conservation laws (via diagonal vs. horizontal integration).
5. Open-Source Availability: The package is publicly available, written in C++ with Python bindings for analysis, and supports standard scientific libraries (Eigen, HDF5, FFTW, MPI).

4. Results and Benchmarks

The authors validated H-NESSi using two distinct physical systems:

• Driven Superconductors (DMFT):
  • Simulated an attractive Hubbard model under a time-dependent electric field within Dynamical Mean-Field Theory (DMFT).
  • Scaling: The H-NESSi solver showed asymptotic scaling significantly better than the uncompressed NESSi solver. For the superconducting state, the time complexity scaled as $T^{2.5}$, compared to $T^3$ for NESSi.
  • Efficiency: Achieved propagation times roughly 3 times longer than NESSi under identical memory constraints.
• 2D Hubbard Model (Lattice):
  • Simulated the optical conductivity of a 2D square lattice Hubbard model using the second Born approximation.
  • Scale: Successfully ran simulations on lattices with $L=64$ ($N_k > 8000$) and $N_t = 16384$ time steps, requiring over 15,000 CPU cores and 10 TB of RAM (without compression, this would be infeasible).
  • Performance: The hybrid MPI/OpenMP implementation demonstrated near-linear speedup with respect to CPU count. The optimal configuration was found to be 8 OpenMP threads per MPI rank.
  • Compression: The $\epsilon$-rank of the Green's function saturated at small block sizes, confirming the efficiency of the compression even for large, complex systems.

5. Significance

H-NESSi represents a paradigm shift in the simulation of non-equilibrium quantum materials:

• Access to New Regimes: It enables the study of long-time dynamics (nanoseconds) and large system sizes that were previously computationally prohibitive.
• Controlled Accuracy: Unlike approximate methods, H-NESSi provides a systematic route to accuracy via the compression threshold, making it suitable for quantitative predictions in strongly correlated regimes.
• Scalability: The ability to scale to thousands of cores and petabytes of data (via compression) makes it a critical tool for next-generation exascale computing in condensed matter physics.
• Foundation for Future Work: The modular design supports extensions to more complex self-energies, embedding schemes, and coupling to other degrees of freedom (e.g., phonons), paving the way for systematic studies of complex multiorbital materials under strong driving.

In summary, H-NESSi overcomes the fundamental "curse of dimensionality" in non-equilibrium Green's function simulations by leveraging hierarchical low-rank structures, effectively extending the horizon of what can be simulated in quantum many-body physics.