Online Spectral Deflation for State Constrained Optimal Control Problems

This paper proposes an online spectral deflation strategy that accelerates the solution of parameter-dependent, state-constrained optimal control problems by reusing a single full-domain reference eigenbasis to precondition Krylov subspace solvers on varying inactive sets, achieving significant reductions in iteration counts and wall time across diverse PDE benchmarks.

The Gist

Imagine you are trying to keep a giant, complex machine (like a power transformer) running at the perfect temperature. You have a control knob (the "control") that you can turn to add or remove heat, but you have a strict safety rule: no part of the machine can ever get hotter than a specific limit.

This is a "state-constrained optimal control" problem. You want the machine to behave perfectly, but you must obey the safety limit at every single point.

The Problem: The "Moving Target"

To solve this on a computer, the machine is broken down into millions of tiny points (a grid). The computer tries to find the perfect setting for the control knob. However, because of the safety limit, some points on the machine are "locked" at the maximum temperature (the Active Set), while others are free to vary (the Inactive Set).

Here is the catch: As you change the operating conditions (like the outside weather or the load on the transformer), the pattern of which points are "locked" and which are "free" changes abruptly. It's like a game of musical chairs where the chairs suddenly disappear and reappear in different spots every time the music stops.

Because the "free" area changes so wildly, the mathematical equations the computer needs to solve change completely every time.

• Old Way: The computer tries to build a brand-new, custom map (a solver) for every single scenario. This is like hiring a new architect to draw a new blueprint every time you move a piece of furniture. It's incredibly slow and expensive.
• The Bottleneck: Even if the underlying physics of the machine doesn't change much, the fact that the "free" area keeps shifting makes it impossible to reuse the old maps.

The Solution: The "Master Blueprint" (Spectral Deflation)

The authors propose a clever trick called Online Spectral Deflation. Instead of building a new map for every scenario, they use a Master Blueprint.

1. The Master Blueprint (Reference Operator): Imagine you have a perfect, detailed map of the entire machine when it's in a standard, "all-free" state. You analyze this map once to find its "slowest" or "stiffest" parts (these are the eigenmodes). Think of these as the fundamental vibrations or patterns of the machine.
2. The Shortcut: When you need to solve a specific scenario where some parts are locked, you don't throw away the Master Blueprint. Instead, you simply crop the Master Blueprint to fit only the "free" parts of the current scenario.
3. The Magic: Even though the "free" area has changed, the fundamental patterns (the vibrations) from the Master Blueprint still match the current situation very well. It's like having a master key that fits almost every lock in a building, even if the locks are slightly different. You just have to trim the key a little bit to fit the specific lock you're facing right now.

How It Works in Practice

• The "Deflation" Step: The computer uses these cropped patterns to "deflate" the problem. It says, "We already know how to handle these tricky, slow-moving parts based on our Master Blueprint, so let's solve those first and ignore them." This leaves the computer to only solve the easy, fast-moving parts.
• The Result: The computer solves the problem 55% to 98% faster in terms of calculation steps.

The Hardware Advantage (GPU vs. CPU)

The paper also tested this on modern graphics cards (GPUs) versus traditional processors (CPUs).

• The CPU approach: Like a team of accountants who are very good at math but have to stop and re-calculate their entire filing system every time a new document arrives.
• The GPU approach: Like a massive army of robots that can process thousands of simple calculations simultaneously. Because the "Master Blueprint" is built only once and then just "trimmed" for each new problem, the robots can work incredibly fast.
• The Outcome: On the GPU, this method was hundreds of times faster than the traditional CPU methods for large problems.

Why It's Not "Guessing"

It is important to note that this method does not use AI or machine learning to guess the answer. It does not replace the high-precision math with a shortcut.

• It still solves the exact same difficult equations.
• It still gets the exact same precise result.
• It just finds a much faster way to get there by reusing a "reference" map that is mathematically proven to be helpful, even when the problem changes.

Summary

Think of it like this: If you have to navigate a city where the roads close and open randomly every day, a normal driver (the old method) would stop and draw a new map every time. This new method says, "Let's keep a master map of the whole city. When roads close, we just fold the map to show only the open roads. We know the main highways (the patterns) are still there, so we can drive much faster without getting lost."

This allows engineers to run complex safety simulations for power grids and other critical systems much faster, without sacrificing accuracy.

Technical Summary

Technical Summary: Online Spectral Deflation for State Constrained Optimal Control Problems

1. Problem Statement

The paper addresses parametric PDE-constrained optimal control problems featuring pointwise state constraints. A representative application is the thermal management of large energy assets (e.g., power transformers), where a distributed heat source $u$ must steer a temperature field $y$ toward a desired profile $y_d$ while respecting a safety bound $\psi$.

The core computational challenge arises in the many-query setting (design studies, digital twins, uncertainty quantification), where the same constrained optimization problem must be solved repeatedly for varying parameters $\theta$ (e.g., operating conditions, material properties).

The Active-Set Volatility Obstacle

State constraints are typically enforced via a Primal Active-Set (PAS) strategy. At each iteration, the domain is partitioned into:

• Active Set ($\mathcal{A}$): Where the state constraint is active ($y = \psi$).
• Inactive Set ($\mathcal{I}$): Where the state is free ($y < \psi$).

After eliminating control and adjoint variables, the problem reduces to solving a Schur-complement system restricted to the inactive set:
$$M_{\mathcal{I}\mathcal{I}}(\theta) \mathbf{y}_{\mathcal{I}} = \mathbf{b}_{\mathcal{I}}$$
This system is Symmetric Positive Definite (SPD). However, as the parameter $\theta$ changes, the active/inactive partition changes non-smoothly (discontinuously). Consequently, the restricted operator $M_{\mathcal{I}\mathcal{I}}$ varies in:

1. Dimension (size of the inactive set).
2. Sparsity pattern.
3. Spectrum.

This volatility renders standard reuse strategies ineffective:

• Sparse direct factorizations must be rebuilt for every instance.
• Algebraic Multigrid (AMG) hierarchies require reconstruction.
• Instance-to-instance Krylov recycling fails because the spectral information from a previous restricted system does not necessarily transfer to the next due to the abrupt change in the operator's structure.

2. Methodology: Reusable Spectral Deflation

The authors propose a strategy that avoids recycling spectral information between consecutive restricted systems. Instead, they anchor the reusable information to a single full-domain reference Schur complement ($M_{\text{ref}}$).

Core Algorithm (A-DEF2)

The method employs A-DEF2 deflated Conjugate Gradient (CG):

1. Reference Basis Construction: Compute the $r_{\text{pool}}$ smallest eigenpairs $(\lambda_j, \phi_j)$ of a fixed reference operator $M_{\text{ref}} = \alpha A(\theta_{\text{ref}})^\top A(\theta_{\text{ref}}) + I$ once (offline).
2. Online Restriction: For a new parameter instance $\theta$ with inactive set $\mathcal{I}$, restrict the precomputed eigenmodes to $\mathcal{I}$: $\phi_j|_{\mathcal{I}}$.
3. Orthonormalization: Apply QR factorization to the restricted vectors to form the deflation basis $Z$.
4. Deflated Solve: Use $Z$ as a deflation basis in the A-DEF2 algorithm. This involves a coarse correction step followed by preconditioned CG on the projected operator $P^\top M_{\mathcal{I}\mathcal{I}} P$, where $P$ is the $M_{\mathcal{I}\mathcal{I}}$-orthogonal projector.

Basis Construction Variants

To handle different regimes and reduce costs, the paper introduces several enhancements:

• Rayleigh–Ritz Re-selection: An optional step where, after restriction, a small dense eigenproblem is solved on the candidate pool to select the best $r_{\text{defl}}$ vectors. This stabilizes the basis in fragile regimes (e.g., 2D high-rank Laplacian) where simple restriction leads to ill-conditioning.
• POD Enrichment: Online enrichment using Proper Orthogonal Decomposition (POD) snapshots from previously solved instances. These are merged with the reference eigenmodes via QR orthogonalization, subject to conditioning safeguards.
• Coarse-Grid Prolongation: To avoid the high cost of computing fine-grid eigenmodes, eigenmodes are computed on a coarser grid and prolonged to the fine grid via multilinear interpolation. This reduces the offline eigensolve cost by a factor of $c^d$ (where $c$ is the coarsening factor).
• Analytical Eigenmodes: For tensor-product grids with constant-coefficient Laplacian references, eigenmodes are available in closed form (Kronecker products of sine vectors), reducing the offline precompute to sub-second times.

Conditioning Safeguards

The method includes two thresholds to ensure stability:

• $\tau_{\text{safe}}$ (Construction): Stops the enrichment of POD modes if the coarse Gram matrix condition number exceeds a conservative limit ($10^4$).
• $\tau_{\text{cond}}$ (Solve-time): If the condition number of the coarse system at solve time exceeds a higher limit ($10^{10}$), the solver falls back to standard Jacobi-preconditioned CG.

3. Theoretical Rationale: Spectral Coherence

The effectiveness of the method relies on a spectral-coherence phenomenon. Although the restricted operator $M_{\mathcal{I}\mathcal{I}}$ changes dimension and spectrum, its low-frequency subspace remains aligned with the low-frequency subspace of the full-domain reference operator $M_{\text{ref}}$.

• This is motivated by Cauchy interlacing (for principal submatrices) and the Davis–Kahan sin $\Theta$ theorem (for eigenspace perturbation).
• Empirical diagnostics (principal angles) show that while worst-case angles between subspaces can reach $90^\circ$, the angles for the leading deflation modes (e.g., the first 20) remain small (e.g., $< 8.4^\circ$), ensuring the deflation basis remains effective.

4. Key Contributions

1. Reusable Spectral Deflation Strategy: A method for repeated inactive-set solves that anchors the basis to a full-domain reference operator rather than recycling between restricted instances.
2. Reference-Operator Viewpoint: A framework where low eigenmodes of a full-domain Schur complement are restricted online to parameter-dependent inactive sets.
3. Spectral-Coherence Interpretation: An explanation of the method's effectiveness based on principal-submatrix structure and subspace perturbation, validated by principal-angle diagnostics.
4. Practical Basis Construction: Development of variants including coarse-grid prolongation, analytical modes, POD enrichment, and Ritz stabilization, with explicit conditioning safeguards.

5. Numerical Results

The method was evaluated on 15 benchmark configurations (diffusion, convection-diffusion, nonlinear thermal, conjugate heat transfer) in 2D and 3D, plus a space–time extension.

Iteration Reduction

• Steady-State: Deflation reduced CG iterations by 55–98% compared to cold Jacobi-preconditioned CG.
  • 3D configurations showed reductions of 60–92% at rank $r=500$.
  • 2D configurations required Ritz reselection or POD merging to achieve 86–98% reduction due to conditioning issues at high ranks.
• Space-Time: For parabolic problems, a Kronecker temporal basis achieved 56–85% iteration reduction, whereas constant-in-time bases failed to capture temporal dynamics.

Wall-Time Performance (GPU vs. CPU)

• Hardware: GPU (NVIDIA H200/H100) vs. CPU (Intel Xeon/Azure).
• Speedup:
  • 3D: GPU deflated CG was 591–973× faster per instance than CPU sparse direct solvers at $50^3$ grids (125K DOF).
  • Comparison with AMG: Despite AMG requiring fewer iterations, GPU deflated CG was 13–18× faster than CPU AMG-RS due to the lower per-iteration cost on GPU and the fact that AMG hierarchies must be rebuilt for every instance, whereas the deflation basis is reused.
• Amortization:
  • With fine-grid eigensolves, the offline cost dominates for small batch sizes.
  • With coarse-grid prolongation or analytical modes, the offline cost is drastically reduced.
  • Break-even: With coarse-grid methods, the amortized speedup over CPU direct solvers is realized within 4–32 instances (depending on dimension and coarsening). With analytical modes for tensor-product grids, the break-even occurs at the first instance.

Accuracy

All deflated solves matched the CPU sparse direct solution to the prescribed Krylov tolerance ($10^{-10}$), with relative state errors comparable to cold CG (medians $\sim 10^{-10}$). The speedups are not due to accuracy relaxation.

6. Significance and Claims

The paper claims that this approach provides a hardware-independent measure of acceleration (iteration reduction) and substantial wall-time gains in GPU deployments.

• Complementarity: The method accelerates the repeated high-fidelity linear algebra without replacing the optimal control solve with a surrogate (unlike Reduced Order Modeling or DeepONets). It preserves the discrete constrained optimality system.
• Robustness: It is robust to the combinatorial changes induced by active-set updates, a scenario where standard Krylov recycling fails.
• Scalability: The method is well-suited for GPU execution, utilizing sparse matrix-vector products and small dense coarse solves.
• Limitations: The spectral coherence is empirically validated but not formally proved for all regimes. The method is currently limited to structured grids for analytical modes and relies on explicit Schur-complement assembly for the eigensolve, limiting the tested grid sizes to $\sim 125,000$ DOF (steady) and $\sim 540,000$ DOF (space-time). Scaling beyond $10^6$ DOF would require matrix-free eigensolvers.

The authors conclude that by anchoring the deflation basis to a stable reference operator, they can effectively solve state-constrained optimal control problems across many parameter instances, overcoming the volatility of active-set changes that typically hinder solver reuse.