Distributionally robust two-stage model predictive control: adaptive constraint tightening with stability guarantee

This paper proposes a Two-Stage Distributionally Robust Model Predictive Control (TSDR-MPC) framework that utilizes a Wasserstein ambiguity set and a cutting-plane algorithm to achieve adaptive constraint tightening and guaranteed closed-loop stability for systems subject to disturbances with unknown, time-varying distributions.

The Gist

Here is an explanation of the paper using simple language, creative analogies, and metaphors.

The Big Picture: Driving a Car in the Fog

Imagine you are driving a car (the System) and you want to get from point A to point B as efficiently as possible. You have a GPS that tells you the best route (the Controller).

However, there are two problems:

1. The Road is Bumpy: There are potholes, wind gusts, and slippery patches (the Disturbances).
2. The Weather Report is Wrong: Sometimes the weather app says "sunny," but it's actually raining. Sometimes it says "light rain," but it's a hurricane. You don't know the exact probability of the weather; you only have a vague idea based on past data (the Unknown Distribution).

The Goal: You need a driving strategy that keeps you on the road (satisfies constraints) and gets you to the destination quickly (minimizes cost), even when you aren't sure what the weather will be like.

---

The Three Approaches to Driving

The paper compares three ways to handle this uncertainty:

1. Robust Control (The "Paranoid" Driver):

   • The Logic: "I will assume the worst possible storm imaginable. If I can drive safely in a Category 5 hurricane, I'll be safe in a drizzle."
   • The Problem: This is too conservative. You drive so slowly and stay so far from the edge of the road that you never make good time. You treat a light breeze like a tornado.

2. Stochastic Control (The "Gambler" Driver):

   • The Logic: "I know the exact weather forecast. There is a 95% chance of sun and a 5% chance of rain. I'll drive normally but accept that I might get wet 5% of the time."
   • The Problem: This requires perfect knowledge of the weather. In the real world, we rarely have perfect forecasts. If the forecast is wrong, you might end up in a ditch.

3. Distributionally Robust Control (The "Smart" Driver - This Paper):

   • The Logic: "I don't know the exact weather, but I know it's probably close to what I've seen before. I'll prepare for the worst-case scenario within a reasonable range of what could happen."
   • The Innovation: This paper introduces a Two-Stage strategy that adapts on the fly.

---

The Core Innovation: The "Two-Stage" Strategy

The authors propose a method called TSDR-MPC (Two-Stage Distributionally Robust Model Predictive Control). Think of it as a two-step decision process for every turn you take:

Stage 1: The Plan (The "Here-and-Now")

You decide your steering angle and speed for the next few seconds. You try to minimize fuel usage and time.

• The Twist: You don't just plan for the "average" road. You plan for the worst road that is plausible given your data.

Stage 2: The Safety Net (The "Wait-and-See")

This is the paper's big idea. Imagine you have a safety budget.

• If you hit a pothole (a disturbance) that pushes you slightly off course, you pay a small "penalty" from your budget to fix it.
• If you hit a massive storm that pushes you off the road, the penalty is huge.
• The Magic: Instead of guessing how much to tighten your safety margin before you start, the system calculates the penalty after seeing the disturbance. It asks: "How bad would it have been if the wind blew this way?"
• If the wind is unpredictable, the system automatically tightens your lane boundaries (Adaptive Constraint Tightening) to keep you safe. If the wind is predictable, it relaxes the boundaries so you can drive faster.

The "Wasserstein Ambiguity Set": The Bubble of Possibility

How does the system know what "plausible" means? It uses a mathematical tool called a Wasserstein Ambiguity Set.

• The Analogy: Imagine you have a bag of marbles representing past weather data. You draw a bubble around these marbles.
• The system assumes the true weather (the real storm) is somewhere inside that bubble.
• It doesn't assume the storm is exactly where the average marble is; it assumes the storm could be anywhere inside the bubble.
• The controller then plans for the worst storm inside that bubble.
• Why it's cool: As you get more data, the bubble shrinks, and your plan gets more precise. If you have very little data, the bubble is big, and you drive more cautiously.

The "Cutting-Plane" Algorithm: Chipping Away at the Problem

Solving this math problem is like trying to find the lowest point in a mountain range that is covered in thick fog. It's very hard to calculate directly.

• The Solution: The authors use a Cutting-Plane Algorithm.
• The Analogy: Imagine you are trying to find the center of a hidden shape in a dark room. You throw a dart. If you miss, you draw a line on the wall saying, "The center is not on this side." You keep throwing darts and drawing lines (cutting planes) until the remaining area is so small you know exactly where the center is.
• This method is fast enough to run on a computer in real-time, making it usable for actual robots or cars.

The "Terminal Constraint": The Safety Anchor

One of the hardest parts of control theory is proving that the system won't go crazy after a long time (Stability). Usually, if the wind keeps blowing you in one direction (non-zero mean), the car might drift off forever.

• The Fix: The authors put a "safety anchor" on the car. They force the planned path (ignoring the wind for a moment) to eventually return to the center of the road.
• This anchor is proportional to where you are right now. If you are far off, the anchor pulls harder. This ensures that even if the wind is weird, the car eventually stabilizes and doesn't drift away forever.

Summary of Results

The paper tested this on a "Double Integrator" system (a simple model of a moving object, like a drone or a car).

1. Zero Disturbance: It behaves like a perfect, deterministic driver.
2. Unknown Wind (Non-zero Mean): It automatically adjusts to counteract the wind drift without crashing.
3. Big Storms (Large Variance): It gets more conservative, widening its safety margins, but still keeps the car on the road.

The Bottom Line

This paper gives us a new way to control machines (robots, power grids, self-driving cars) that are smart enough to handle uncertainty without being paranoid.

Instead of assuming the worst-case scenario (which is too slow) or trusting a perfect forecast (which doesn't exist), it builds a flexible safety bubble around the data. It constantly asks, "What is the worst thing that could reasonably happen right now?" and adjusts its driving style accordingly. This makes systems safer, more efficient, and ready for the messy reality of the real world.

Technical Summary

Here is a detailed technical summary of the paper "Distributionally robust two-stage model predictive control: adaptive constraint tightening with stability guarantee."

1. Problem Statement

The paper addresses the challenge of controlling discrete-time linear stochastic systems subject to unknown time-varying disturbances with unknown distributions.

• Limitations of Existing Methods:
  • Robust MPC: Guarantees constraint satisfaction under worst-case scenarios but is often overly conservative, leading to poor performance.
  • Stochastic MPC: Balances performance and conservatism using chance constraints but relies on precise knowledge of the disturbance distribution, which is rarely available in practice.
  • Existing Distributionally Robust MPC (DRMPC): Often assumes disturbances have zero mean or known moments. In many real-world scenarios, disturbances exhibit unknown non-zero means and time-varying covariances that only lie within known bounds.
• Core Objective: Develop an MPC framework that handles unknown, time-vaying disturbance statistics (mean and covariance) without being overly conservative, while guaranteeing recursive feasibility and closed-loop stability.

2. Methodology: Two-Stage Distributionally Robust MPC (TSDR-MPC)

The authors propose a novel Two-Stage Distributionally Robust MPC scheme based on the following key components:

A. Two-Stage Optimization Formulation

The control problem is formulated as a two-stage distributionally robust program:

1. First Stage (Here-and-Now): Decides the control input sequence ($\bar{u}_k$) to minimize a quadratic cost function ($V_q$) representing state and input penalties.
2. Second Stage (Wait-and-See): Models the penalty for constraint violations ($V_c$) as a separate optimization problem. Instead of hard constraints, violations are penalized via a linear program inspired by $L_1$ exact penalty methods.
   • The total objective is to minimize the worst-case expected total cost over a Wasserstein ambiguity set.
   • This structure allows the controller to adaptively tighten constraints based on current state and sample data, rather than using pre-specified, static tightening parameters.

B. Wasserstein Ambiguity Set

The uncertainty is modeled using a 2-Wasserstein distance based ambiguity set ($\mathcal{P}_k$).

• The set contains all probability distributions within a radius $\epsilon$ of the empirical distribution derived from historical data.
• This approach provides strong out-of-sample performance guarantees and handles the lack of precise distributional knowledge.
• The paper derives bounds for the worst-case mean ($\hat{\mu}_k$) and covariance ($\hat{\Sigma}_k$) using the Gelbrich bound, acknowledging that even if the true mean is zero, the worst-case mean within the ambiguity set is generally non-zero.

C. Tractable Reformulation

To solve the resulting non-convex minimax problem efficiently:

• Strong Duality: The inner maximization over the ambiguity set is transformed into a minimization problem using strong duality.
• Reformulation: The problem is converted into a finite-dimensional optimization problem involving a dual variable $\gamma$ and the empirical samples.
• Cutting-Plane Algorithm: A specialized algorithm is developed to solve the reformulated problem. It iteratively solves a master problem (lower bound) and separation/dual problems to generate new cutting planes (support points and dual extreme points). The algorithm is proven to converge in a finite number of iterations, making it suitable for real-time implementation.

D. Stability Guarantee via Terminal Constraints

A critical innovation is the handling of non-zero mean disturbances for stability:

• Terminal Constraint: A constraint is applied solely to the nominal system (ignoring the disturbance distribution). It bounds the terminal state of the nominal trajectory proportionally to the current state ($\|z_{N|k}\|^2 \leq l_c \|x_k\|^2$).
• Significance: This prevents cross-terms involving the worst-case mean from introducing persistent offsets in the Lyapunov-like stability analysis. It ensures that the system remains stable and the average cost remains bounded even when the disturbance has a non-zero mean.
• Feasibility: Unlike traditional robust MPC, this method does not require explicit robust tubes or invariant sets for the disturbed system, as feasibility is maintained through the adaptive penalty mechanism.

3. Key Contributions

1. Novel Framework: Introduction of a Two-Stage Distributionally Robust MPC (TSDR-MPC) that explicitly handles unknown time-varying means and covariances.
2. Adaptive Constraint Tightening: A mechanism where constraint tightening emerges naturally from the optimization problem via the second-stage penalty, eliminating the need for manual tuning of tightening parameters or pre-computed tubes.
3. Stability under Non-Zero Mean: A rigorous theoretical proof of recursive feasibility and asymptotic performance bounds for systems with non-zero mean disturbances, achieved through a novel nominal terminal constraint.
4. Computational Efficiency: Development of a finite-convergent cutting-plane algorithm that renders the complex distributionally robust problem tractable for real-time control.
5. Theoretical Consistency: Proof that the framework degenerates to classical deterministic MPC (when uncertainty vanishes) and moment-based DRMPC (when mean is zero), validating its generality.

4. Results

Numerical simulations were conducted on a double-integrator benchmark system under four distinct disturbance scenarios:

• Scenario A (Zero Mean, Low Variance): The controller behaves nearly deterministically, with trajectories tightly clustered around the nominal path.
• Scenario B (Non-Zero Mean, Low Variance): The controller successfully counteracts the bias introduced by the unknown mean, maintaining constraint satisfaction without manual retuning.
• Scenario C (Zero Mean, High Variance): The controller adapts to large covariance, showing occasional minor constraint violations (expected due to probabilistic guarantees) but maintaining overall stability.
• Scenario D (Non-Zero Mean, High Variance): The most challenging case. The controller maintains stability and prevents divergence, outperforming traditional robust tube-based methods which would likely fail or become infeasible under such high uncertainty.

Key Findings:

• The controller automatically adjusts its conservatism based on the prevailing uncertainty level.
• The closed-loop system converges to a neighborhood of the origin, consistent with the derived asymptotic performance bound.
• The cutting-plane algorithm terminates efficiently, confirming real-time feasibility.

5. Significance

This paper makes a significant contribution to the field of control theory by bridging the gap between robustness and performance in the presence of distributional uncertainty.

• Practical Impact: It offers a solution for real-world systems where disturbance statistics are not only unknown but also time-varying (e.g., varying means due to sensor drift or changing environmental conditions).
• Theoretical Advancement: It resolves the difficulty of proving stability for distributionally robust controllers when the worst-case distribution has a non-zero mean, a common limitation in previous literature.
• Implementation: By providing a tractable reformulation and a convergent algorithm, it moves distributionally robust control from theoretical concepts to practical, real-time applications.