Gaussian behaviors: representations and data-driven control

1. Problem Statement

Data-driven control methods based on behavioral systems theory (e.g., DeePC, Subspace Predictive Control) have shown success by representing Linear Time-Invariant (LTI) systems directly as sets of trajectories. However, existing frameworks primarily rely on deterministic behavioral descriptions. When applied to stochastic systems, these methods often rely on regularization to implicitly handle uncertainty, but the theoretical link between regularization and stochastic uncertainty remains abstract.

The paper addresses the following gaps:

• Lack of a unified, tractable modeling framework that describes finite-length trajectories of stochastic systems as Gaussian processes directly from data.
• The need to rigorously explain how existing data-driven methods (like regularized DeePC) relate to stochastic uncertainty quantification.
• The need for a control formulation that explicitly accounts for distributional uncertainty without relying solely on "optimistic" assumptions that may lead to poor closed-loop performance.

2. Methodology

A. Gaussian Behaviors Framework

The authors propose a new modeling framework called Gaussian Behaviors.

• Definition: A finite-length trajectory $w_{[t, t+L-1]}$ of a stochastic system is modeled as a multivariate Gaussian random vector: $w \sim \mathcal{N}(\mu_w, \Sigma_w)$.
• Linearity: A Gaussian behavior is linear if the mean $\mu_w$ lies in the image of a matrix $B$ (i.e., $\mu_w \in \text{im}(B)$).
• Data Estimation: The covariance matrix $\Sigma_w$ is estimated directly from observed data trajectories using the empirical sample covariance: $\hat{\Sigma}_w = \frac{1}{D} W W^\top$, where $W$ is the data matrix.
• Prediction: Given observed past data ($w_{free}$), the future trajectory ($w_{dep}$) is predicted using the conditional distribution of the Gaussian process. This yields a predictive mean $\mu_{pred}$ and covariance $\Sigma_{pred}$.

B. Theoretical Connections

The paper establishes that this framework unifies existing descriptions:

1. Deterministic LTI: A deterministic system is a special case where the covariance is zero (or singular), and the trajectory lies strictly in a subspace.
2. Stochastic State-Space: Standard LTI state-space models with Gaussian noise and initial conditions are shown to be special cases of Gaussian behaviors with specific structures in $\mu_w$ and $\Sigma_w$.
3. Maximum Likelihood: The sample covariance $\hat{\Sigma}_w$ is proven to be the Maximum Likelihood Estimate (MLE) of the true covariance under i.i.d. Gaussian assumptions.

C. Control Formulations

The authors derive three distinct control formulations based on the Gaussian behavior model:

1. Certainty-Equivalence Control (SPC):

   • Minimizes the expected cost using the estimated predictive mean $\hat{\mu}_{pred}$.
   • Result: This is mathematically equivalent to the classical Subspace Predictive Control (SPC). The term involving the covariance ($\text{tr}(Q\hat{\Sigma}_{pred})$) is constant with respect to the decision variable and thus does not affect the optimal input.

2. Distributionally Optimistic Control (Regularized DeePC):

   • Acknowledges that the estimated mean $\hat{\mu}_{pred}$ may be inaccurate.
   • Formulates a min-min problem: Minimize cost over inputs $u_f$ and a perturbed mean $\mu$, subject to a Kullback-Leibler (KL) divergence constraint between the true distribution and the estimated one.
   • Result: By relaxing the KL constraint into a penalty term, the authors prove that this formulation is equivalent to the regularized DeePC method. The regularization term in DeePC ($\lambda_g \|g\|^2$) corresponds to the KL divergence penalty, effectively optimizing "optimistically" within an uncertainty set.

3. Distributionally Robust Control (Novel Proposal):

   • To mitigate the risks of optimism (which can lead to constraint violations or poor performance under high uncertainty), the authors propose a min-max formulation.
   • Objective: Minimize the expected cost over inputs $u_f$ while maximizing the cost over the worst-case mean $\mu$ within the KL divergence uncertainty set.
   • Reformulation: Using Lagrange duality, the inner maximization is solved analytically. The resulting outer problem is a convex optimization problem (quadratic in $u_f$) that can be solved efficiently.
   • Tuning: A parameter $\lambda$ controls the size of the uncertainty set. As $\lambda \to \infty$, the robust formulation converges to the certainty-equivalence (SPC) solution.

3. Key Contributions

1. Gaussian Behaviors Model: Introduced a pragmatic, data-driven framework modeling system trajectories as Gaussian processes, enabling direct estimation of uncertainty (covariance) from data without explicit system identification.
2. Unification of Methods: Provided a rigorous theoretical explanation showing that:
   • Subspace Predictive Control (SPC) is a certainty-equivalence formulation.
   • Regularized DeePC is a distributionally optimistic formulation that implicitly optimizes for the worst-case within a specific uncertainty set defined by KL divergence.
3. Novel Robust Control: Proposed a Distributionally Robust Control (DRC) formulation that explicitly guards against model uncertainty. Unlike the optimistic DeePC, this approach minimizes the worst-case expected cost.
4. Convex Reformulation: Demonstrated that the proposed robust control problem can be reformulated as a convex quadratic program, ensuring computational tractability.

4. Results and Analysis

• Theoretical Equivalence: The paper proves that the regularizer in DeePC is proportional to the KL divergence between the estimated predictive distribution and a perturbed distribution. This explains why regularization improves robustness in data-driven control.
• Optimism vs. Robustness:
  • The "optimistic" approach (DeePC) balances exploration and exploitation but may degrade performance if uncertainty is high.
  • The proposed "robust" approach (DRC) provides a conservative bound, ensuring performance guarantees even when the estimated model is inaccurate.
• Convexity: The robust control problem is shown to be convex for sufficiently large tuning parameters ($\lambda$), allowing for efficient numerical solution.
• Consistency: The framework naturally recovers deterministic LTI behaviors and standard stochastic state-space models as special cases, ensuring backward compatibility with existing theory.

5. Significance

This work bridges the gap between behavioral systems theory and stochastic control.

• Interpretability: It demystifies the role of regularization in data-driven control, revealing it as a mechanism to handle distributional uncertainty via an optimistic lens.
• Safety and Reliability: By introducing a distributionally robust alternative, the paper offers a solution for safety-critical applications where "optimism" is insufficient.
• Practicality: The proposed methods rely on simple sample covariance estimation and result in convex optimization problems, making them highly suitable for real-time implementation in complex stochastic environments.
• Future Direction: It opens the door for extending data-driven control to broader classes of stochastic systems beyond Gaussians, potentially using similar distributional robustness principles.