Technical Summary: Continuous-Time Covariance Steering with Common Free-Final Time

1. Problem Formulation

This paper addresses the optimal common free-final time problem for steering the state covariance of a continuous-time stochastic linear time-invariant (LTI) system between prescribed initial and terminal covariance matrices.

The system is governed by the stochastic differential equation (SDE):
$$d\mathbf{x}(t) = (A\mathbf{x}(t) + B\mathbf{u}(t))dt + \Gamma d\mathbf{w}(t)$$
where $\mathbf{x}(t) \in \mathbb{R}^n$ is the state, $\mathbf{u}(t) \in \mathbb{R}^m$ is the control, and $\mathbf{w}(t)$ is a standard Brownian motion. The objective is to find a control policy $\mathbf{u}(\cdot)$ and a common final time $t_f > 0$ that minimize the expected cost:
$$J(t_f, \mathbf{u}(\cdot)) = \mathbb{E}\left\{ \frac{1}{2} \int_0^{t_f} [\varpi + \mathbf{x}(t)^T Q \mathbf{x}(t) + \mathbf{u}(t)^T R \mathbf{u}(t)] dt \right\}$$
subject to the boundary conditions $\mathbf{x}(0) \sim \mathcal{N}(0, \Sigma_0)$ and $\mathbf{x}(t_f) \sim \mathcal{N}(0, \Sigma_f)$.

Unlike standard Covariance Steering (CS) problems where $t_f$ is fixed, this formulation treats $t_f$ as a decision variable. The authors highlight that fixing $t_f$ a priori can be inefficient or conservative in applications such as spacecraft maneuvers, robotic motion planning, and multi-agent reconfiguration, where the task duration is a critical design variable.

2. Methodology

Deterministic Reformulation

The authors first establish that the optimal control for the stochastic problem admits a linear state-feedback representation $\mathbf{u}^*(t) = K^*(t)\mathbf{x}(t)$. This allows the stochastic optimal control problem (SOCP) to be reformulated as a deterministic optimal control problem (OCP) involving only the covariance matrix $\Sigma(t)$:
$$\dot{\Sigma}(t) = (A + BK(t))\Sigma(t) + \Sigma(t)(A + BK(t))^T + D$$
where $D = \Gamma\Gamma^T$. The cost functional becomes deterministic, depending solely on the trace of the covariance and the control gain.

Optimality Conditions and Hamiltonian-Zeros

Using the Pontryagin Maximum Principle (PMP) on the deterministic reformulation, the authors derive the necessary conditions for optimality. A key contribution is the derivation of the Hamiltonian transversality condition for the free final time.
Let $\mathcal{H}_{t_f}(\Sigma, \Pi)$ be the Hamiltonian associated with the covariance and co-state $\Pi$. For a candidate optimal final time $t_f^*$, the transversality condition requires:
$$\mathcal{H}_{t_f^*}(\Sigma(t), \Pi(t)) = 0 \quad \forall t \in [0, t_f^*]$$
The authors define Hamiltonian-zeros as candidate final times satisfying this condition. They distinguish between:

1. Finite Hamiltonian-zeros: $t_f^* < \infty$.
2. Asymptotic Hamiltonian-zeros: The condition is approached only as $t_f \to \infty$.

Asymptotic Analysis: Gateways and Connections

To characterize the behavior of solutions as $t_f \to \infty$, the paper introduces two novel concepts:

• Asymptotic Gateways (AG): A set $\mathcal{G}_\infty$ of stationary covariance states $\Sigma_s$ that satisfy the uncontrolled Lyapunov equation ($A\Sigma_s + \Sigma_s A^T + D = 0$) and incur zero running cost ($\text{trace}(Q\Sigma_s) = 0$).
• Asymptotic Connections (AC): Limiting trajectories of finite-horizon optimal solutions that approach $\mathcal{G}_\infty$, remain in its neighborhood (turnpike behavior), and then depart to satisfy the terminal constraint.

The authors prove that an asymptotic Hamiltonian-zero exists if and only if $\varpi = 0$ and $\mathcal{G}_\infty$ is non-empty. If $\mathcal{G}_\infty$ is empty (e.g., if $\varpi > 0$ or $Q \neq 0$), a finite optimal final time is guaranteed to exist.

Numerical Algorithm

Leveraging the sensitivity of the Hamiltonian with respect to $t_f$, the authors develop a trust-region line-search algorithm. This algorithm:

1. Iteratively updates $t_f$ to find zeros of the Hamiltonian.
2. Computes sensitivities analytically (for specific diffusion structures) or numerically.
3. Includes a Detector module to identify when the trajectory approaches an Asymptotic Gateway, distinguishing between finite-time convergence and asymptotic regimes where the optimal time tends to infinity.

3. Key Contributions

The paper claims the following technical contributions:

• First Systematic Study: It provides the first systematic treatment of continuous-time covariance steering with a common free-final time in the general stochastic LTI setting.
• Transversality Condition: It derives the correct Hamiltonian transversality condition for the common final time via a deterministic reformulation, addressing a gap where classical Stochastic Minimum Principles do not provide necessary conditions for common free-final times.
• Finiteness vs. Asymptoticity: It establishes sufficient conditions for the existence of a finite optimal final time and characterizes the mechanism (via Asymptotic Gateways) by which the optimal time may only be approached asymptotically.
• Turnpike Interpretation: It introduces Asymptotic Gateways and Connections to describe the invariant structures governing long-horizon optimal trajectories, providing a turnpike-type interpretation for the large-horizon regime.
• Algorithmic Solution: It proposes a trust-region algorithm capable of detecting both finite optimal times and infinite-horizon solution regimes, validated through numerical examples.

4. Results and Numerical Examples

The authors validate their theory and algorithm using three examples:

1. Effect of Noise Intensity: A 2D example demonstrating three behaviors:
   • Convergence to a finite optimal $t_f$ (when $Q=I$).
   • Convergence toward an unbounded asymptotic gateway (when $Q=0$ and system modes approach marginal stability).
   • Convergence toward a singular asymptotic gateway (when diffusion is nearly singular).
2. Spacecraft Maneuver: A demonstration of steering a deputy spacecraft's covariance under Clohessy-Wiltshire-Hill (CWH) dynamics. The algorithm successfully handles different initial guesses for $t_f$, including cases where the Hamiltonian sensitivity is nearly zero (flat regions), showing robustness.
3. Gaussian Mixture Steering: An application to steering a Gaussian Mixture Model (GMM) to a target GMM. The authors define the "Price of Synchronization" as the cost gap between enforcing a common final time for all mixture components versus allowing each component pair to have its own optimal final time. Results show that the common time constraint can increase costs (e.g., by 14% in the example), and this gap shrinks as mixture components merge.

5. Significance and Claims

The paper positions its work as a necessary advancement for uncertainty-aware control in safety-critical domains (e.g., aerospace, robotics) where task duration is a performance metric.

• Theoretical Insight: It clarifies that free-final time problems are "delicate," exhibiting phenomena like short-horizon singularities and long-horizon turnpike behaviors that are absent in fixed-horizon formulations.
• Practical Utility: The proposed framework avoids the inefficiency of arbitrarily fixing $t_f$ and provides a mechanism to detect when a problem is ill-posed for finite-time solutions (i.e., when the optimal solution requires an infinite horizon).
• Generalization: The formulation offers a natural extension to general distribution steering problems, such as allowing different components of a mixture to arrive at different times, thereby avoiding the "cost of synchronization."

The authors remain modest regarding future work, noting that extending these results to weaker hypotheses (e.g., relaxing controllability assumptions or the existence of fixed asymptotic limits) remains an open area for investigation.