Distributionally Robust PAC-Bayesian Control

1. Problem Statement

The paper addresses a critical limitation in learning-based control: the vulnerability of controllers trained on finite datasets to distribution shifts (the "sim-to-real" gap).

• The Challenge: Standard learning-based control assumes the training data distribution matches the deployment environment. In reality, unmodeled disturbances and environmental changes act as adversarial perturbations, degrading performance.
• Limitations of Existing Methods:
  • PAC-Bayes: Existing PAC-Bayesian control frameworks typically assume bounded loss functions and identical training/deployment distributions. They often fail to handle unbounded losses (common in control) or explicitly model distribution shifts.
  • Distributionally Robust Optimization (DRO): While DRO handles distribution shifts, it often lacks the finite-sample generalization guarantees provided by PAC-Bayes or relies on information-theoretic measures (like $f$-divergences) that inflate complexity terms without directly linking to controller stability.
• Goal: To develop a framework that provides high-probability finite-sample safety certificates for learning-based controllers that remain valid even when the deployment distribution differs from the training distribution, specifically handling unbounded losses.

2. Methodology

The authors propose a Distributionally Robust (DR) PAC-Bayesian framework that integrates three key theoretical pillars:

A. Theoretical Foundation: Wasserstein DR-PAC-Bayes

The core innovation is extending the PAC-Bayes generalization bound to account for distribution shifts using the 1-Wasserstein distance.

• Ambiguity Set: The deployment distribution $D'$ is assumed to lie within a Wasserstein ball of radius $\rho$ around the training distribution $D$.
• Lipschitz Penalty: Leveraging the Kantorovich-Rubinstein duality, the authors derive a bound where the generalization error includes a penalty term proportional to the Lipschitz constant of the loss function with respect to the disturbance.
• Unbounded Losses: Unlike previous works requiring bounded losses, this framework accommodates sub-Gaussian losses where the variance proxy depends on the specific controller (hypothesis).

B. System Level Synthesis (SLS) Reparametrization

To make the problem computationally tractable for Linear Time-Invariant (LTI) systems, the authors utilize the System Level Synthesis (SLS) framework.

• Reparametrization: Instead of optimizing over controller gains directly, they optimize over the closed-loop system responses (state and input trajectories as linear maps of disturbances).
• Closed-Loop Map ($M(\theta)$): The loss function is expressed as the norm of the closed-loop map applied to the disturbance trajectory: $\ell(\theta, w) = \|M(\theta)w\|$.
• Key Insight: In the SLS formulation, both the sub-Gaussian variance proxy (for concentration) and the Lipschitz constant (for robustness) are explicitly determined by the operator norm of the closed-loop map $M(\theta)$.

C. The Optimization Objective

The learning algorithm seeks a posterior distribution $Q$ over controllers that minimizes a robust upper bound on the risk:
$$\min_{Q} \mathbb{E}_{\theta \sim Q} \left[ \hat{R}_S(\theta) + \rho \|M(\theta)\|_{op} \right] + \text{Complexity Term}$$
Where:

• $\hat{R}_S(\theta)$: Empirical risk on training data.
• $\rho \|M(\theta)\|_{op}$: The Wasserstein robustness penalty (Lipschitz constant $\times$ shift radius).
• Complexity Term: A PAC-Bayes term involving the KL divergence between the posterior $Q$ and prior $P$, scaled by the expected sub-Gaussian variance.

3. Key Contributions

1. Wasserstein DR-PAC-Bayes Extension: The first framework to combine PAC-Bayes theory with 1-Wasserstein distributional robustness for control, specifically handling unbounded, Lipschitz continuous loss functions.
2. Controller-Dependent Certificates: Derivation of explicit certificates where both the concentration bound (variance) and the robustness penalty (Lipschitz constant) are tied directly to the operator norm of the closed-loop map. This allows the optimization to actively "immunize" the system by minimizing the sensitivity of the closed-loop dynamics to disturbances.
3. Tractable SLS Formulation: Specialization of the abstract theory to finite-horizon LTI systems via SLS, resulting in a computationally tractable optimization problem (solvable via gradient-based methods on Gaussian posteriors).
4. No Distribution Identification Required: The framework provides guarantees for deployment distributions that differ from training, without needing to explicitly identify the true deployment distribution.

4. Results and Numerical Validation

The authors validated their approach using a Double Integrator system with a finite horizon ($T=10$).

• Setup: Controllers were trained on Gaussian disturbance trajectories and tested under an adversarial distribution shift (mean shift) within the certified Wasserstein radius ($\rho = 0.08$).
• Comparison:
  • Vanilla PAC-Bayes ($\rho=0$): Failed to provide a valid upper bound on the test risk when distribution shifts occurred. The actual risk exceeded the theoretical bound.
  • Robust PAC-Bayes ($\rho > 0$): Successfully provided a valid upper bound on the test risk under distribution shift.
• Performance: The robust method not only guaranteed safety but also resulted in controllers with lower empirical costs on the shifted test data compared to the vanilla approach.
• Decomposition: The results showed that as sample size increases, the PAC-Bayes complexity term decreases, while the optimization naturally balances the empirical risk and the Wasserstein penalty.

5. Significance

This work bridges a critical gap between statistical learning theory and robust control:

• Safety Certificates: It offers a rigorous, high-probability guarantee that a learning-based controller will perform safely even in environments different from training, a prerequisite for real-world deployment (e.g., robotics, autonomous vehicles).
• Handling Realism: By addressing unbounded losses and distribution shifts, it moves beyond idealized assumptions common in theoretical control literature.
• Optimization Efficiency: The use of SLS transforms a complex non-convex control problem into a convex (or tractable) optimization over system responses, making the theoretical bounds practically implementable.
• Future Impact: The framework paves the way for learning-based Model Predictive Control (MPC) and robust control under model uncertainty, providing a mathematical foundation for "safe exploration" and deployment.