⚡ electrical engineering

Covariance Steering of Discrete-Time Markov Jump Linear Systems with Multiplicative Noise

This paper addresses the finite-horizon covariance steering problem for discrete-time Markov jump linear systems with multiplicative noise by proving that affine feedback is insufficient, introducing a lifted-state formulation to derive an equivalent SDP reformulation for the unconstrained case, and developing tractable convex surrogates with an iterative scheme to handle chance-constrained state and control limits.

Original authors: Fangji Wang, Siddhartha Ganguly, Panagiotis Tsiotras

Published 2026-04-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Fangji Wang, Siddhartha Ganguly, Panagiotis Tsiotras

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a ship navigating through a stormy sea. Your goal isn't just to get from Point A to Point B; you have very specific rules about how you arrive. You need to arrive at a specific time, with the ship sitting at a precise location, and you need the passengers to be spread out in a specific pattern (not too bunched up, not too scattered).

This is the essence of Covariance Steering. "Covariance" is just a fancy math word for "spread" or "uncertainty." You are trying to steer the spread of your ship's position, not just the ship itself.

Now, make the problem harder. Imagine your ship is a chameleon. It can instantly change its color and behavior based on a random signal (like a traffic light that changes unpredictably). Sometimes it's a fast, agile speedboat (Mode 1); other times, it's a slow, heavy barge (Mode 2). This is a Markov Jump Linear System (MJLS).

Finally, add the twist: The storm isn't just random rain; the wind gets stronger the faster you go or the bigger the waves get. This is Multiplicative Noise. The uncertainty scales with your actions.

This paper solves the puzzle of how to steer this shape-shifting, storm-affected ship to hit its target perfectly, while minimizing fuel (cost) and obeying safety rules.

Here is the breakdown of their solution using simple analogies:

1. The Problem: Why Old Rules Don't Work

In the past, if you had a normal ship (no shape-shifting, no scaling storms), you could steer it using a simple rule: "If we are off course, turn the wheel by X amount." This is called Affine State-Feedback.

However, the authors discovered that for their shape-shifting, storm-scaling ship, this simple rule fails.

The Analogy: Imagine trying to balance a broom on your hand. If the broom is heavy and the wind gets stronger the faster you move your hand, simply reacting to the tilt isn't enough. You need to add a little bit of "random jitter" to your hand movements to counteract the unpredictable wind.
The Discovery: The authors proved that to hit the target perfectly, your steering command must have three parts:
1. Feedback: Reacting to where you are now.
2. Feedforward: A pre-planned path based on where you think you should be.
3. Random Noise: A deliberate, calculated "jitter" (like shaking the steering wheel slightly) to manage the uncertainty.

2. The Magic Trick: The "Lifted" View

The math for this problem is incredibly messy because the "spread" of the ship depends on where the ship is, and the ship's path depends on the spread. It's a tangled knot.

The authors untangled it by using a Lifted-State Formulation.

The Analogy: Imagine you are trying to track a flock of birds. Instead of tracking every single bird's position and how far apart they are (which is thousands of variables), you lift your view up to a drone. From the drone, you see the center of the flock and the size of the flock as a single, unified object.
The Result: They combined the "where" (mean) and the "spread" (covariance) into one giant matrix. This turned a messy, non-linear problem into a clean, structured one that computers can solve easily using a method called Semidefinite Programming (SDP). Think of SDP as a super-smart calculator that finds the perfect path through a maze of constraints.

3. The Safety Net: Chance Constraints

In the real world, you can't just hope the ship stays safe; you need guarantees. You might say, "There must be a 95% chance the ship stays within this safe zone."

The authors developed a way to turn these "maybe" rules into hard math rules the computer can solve.

The Analogy: Imagine you are walking through a minefield. You can't see the mines, but you know they are mostly in the middle. You want to walk a path where there is a 99% chance you don't step on one.
The Solution: They created a "conservative" safety bubble. If you stay inside this bubble, you are mathematically guaranteed to be safe. However, this bubble can be too big (too conservative), making the path longer than necessary.

4. The Refinement: Iterative Updates

To fix the "too big bubble" problem, they invented an Iterative Reference-Update Scheme.

The Analogy: Imagine you are trying to hit a bullseye with a bow and arrow, but your sight is blurry.
1. First try: You aim at a wide, safe target. You miss the bullseye but hit the safe zone.
2. Second try: You look at where your arrow actually landed. You realize, "Oh, I was aiming too far left!" You adjust your sight to aim closer to where you actually landed.
3. Repeat: You keep adjusting your aim based on the previous shot.
The Result: With every step, the "safety bubble" shrinks to fit the reality more tightly, allowing for a more efficient path without sacrificing safety.

5. Real-World Application: The Financial Hedge

The paper tests this on a Finance problem.

The Ship: A financial portfolio (a mix of stocks and options).
The Modes: The market can be "Normal" or "Crisis" (Regime Switching).
The Noise: The risk (volatility) gets higher the more you trade (Control-Dependent Noise).
The Goal: You want to "hedge" (protect) your portfolio so that at the end of the day, your losses are minimized and your risk is within a specific limit.
The Outcome: Their method helps a trader navigate a volatile market that changes its rules randomly, ensuring they don't lose too much money while keeping their risk under control.

Summary

This paper is about steering uncertainty. It tells us that when the world changes its rules randomly and the risks grow with your actions, you can't just react simply. You need a plan that includes a little bit of calculated randomness. By looking at the problem from a "higher angle" (the lifted state) and refining your safety margins step-by-step, you can navigate even the most chaotic systems to hit your target perfectly.

1. Problem Statement

The paper addresses the finite-horizon covariance steering problem for Discrete-Time Markov Jump Linear Systems (MJLS) subject to both state-dependent and control-dependent multiplicative noise.

System Dynamics: The system evolves according to linear modes that switch abruptly according to a Markov chain. The dynamics include additive noise (standard disturbance) and multiplicative noise (where uncertainty scales with the current state or control input).
Objective: Minimize a quadratic running cost (state and control penalties) while steering the system from a given set of mode-conditioned initial means and covariances to a prescribed terminal mean and covariance.
Constraints: The problem considers both unconstrained cases and cases with chance constraints (probabilistic bounds) on the state and control, specifically ball constraints (norm bounds) and half-space constraints.
Core Challenge: Unlike standard linear systems or MJLS with only additive noise, the presence of multiplicative noise creates a fundamental structural difference: purely affine state-feedback laws are generally insufficient to achieve the desired terminal moments.

2. Methodology

The authors develop a rigorous framework to solve this non-convex, stochastic control problem by transforming it into a convex optimization problem.

A. Control Parametrization

The paper first establishes that without loss of generality, the optimal control policy can be represented as a combination of:

Mode-dependent linear feedback: $K_k(q_k)(x_k - \mu_k(q_k))$
Feedforward term: $v_k(q_k)$
Independent random component: $\nu_k$ (a zero-mean random variable with a specific covariance).

Crucially, the authors prove that purely affine state-feedback (terms 1 and 2 only) is not sufficient when multiplicative noise is present. The random component is necessary to decouple the mean and covariance dynamics sufficiently to reach arbitrary terminal targets.

B. Lifted-State Formulation

To handle the coupling between mean and covariance dynamics caused by multiplicative noise, the authors introduce a lifted-state formulation.

They define a lifted state $\tilde{x}_k = [x_k^T, 1]^T$ .
This allows the second moments (which contain both mean and covariance information) to be treated as a unified matrix variable $\tilde{I}_k = \mathbb{E}[\tilde{x}_k \tilde{x}_k^T]$ .
The original covariance steering problem is shown to be exactly equivalent to a "moment steering" problem on this lifted system.

C. Lossless Relaxation and SDP Reformulation

The moment propagation equations are inherently non-linear due to the control gain matrices. The authors propose a lossless relaxation:

Instead of optimizing over control gains directly, they optimize over the moment variables (conditional second moments of the state and control).
They prove that any feasible solution to the relaxed moment problem can be realized by a valid control policy (using a decomposition lemma).
This transforms the problem into a Semidefinite Programming (SDP) problem, which can be solved efficiently and globally.

D. Chance-Constrained Extension

For problems with probabilistic constraints (e.g., $\mathbb{P}(\|x_k - a\| \leq r) \geq 1-\epsilon$ ):

The authors derive tractable sufficient convex surrogates using Markov's inequality (for ball constraints) and Cantelli's inequality (for half-space constraints).
These surrogates depend on "reference variables" (e.g., reference means).
To reduce the conservatism introduced by these surrogates, they propose an iterative reference-update scheme:
1. Solve the SDP with initial reference values.
2. Update the reference values based on the resulting moments.
3. Re-solve the SDP.
4. Additionally, they introduce a tolerance relaxation mechanism (starting with looser constraints and tightening them) to ensure feasibility during the early iterations.

3. Key Contributions

Novel Problem Formulation: The first formulation of finite-horizon covariance steering for MJLS with both state- and control-dependent multiplicative noise.
Structural Insight: Proof that affine state-feedback is insufficient for MJLS with multiplicative noise, necessitating a control law with an independent random component.
Exact SDP Reformulation: Development of a lifted-state framework that yields an exact, lossless SDP reformulation for the unconstrained problem.
Chance-Constrained Framework: Derivation of convex surrogates for ball and half-space chance constraints, coupled with an iterative algorithm to minimize conservatism and ensure feasibility.
Application Demonstration: Validation of the framework on a dynamic hedging problem in finance, demonstrating its ability to handle regime-switching markets, volatility-scaled uncertainty, and execution risk.

4. Numerical Results

The paper validates the approach through two numerical experiments:

Iterative Reference Updates: In a standard MJLS example, the iterative update of reference variables was shown to progressively tighten the constraints, moving the state trajectories closer to the constraint boundaries and reducing the cost function monotonically.
Tolerance Relaxation: In a highly constrained scenario where the initial SDP was infeasible, the tolerance relaxation scheme successfully recovered a feasible solution by gradually tightening the violation tolerances.
Financial Hedging: The method successfully managed a portfolio tracking error under regime-switching conditions (Normal vs. Distressed markets), enforcing probabilistic bounds on Delta/Vega errors and trade execution risks while minimizing cost.

5. Significance and Impact

This work bridges a critical gap in stochastic control theory. While covariance steering is well-understood for linear systems with additive noise, the inclusion of multiplicative noise (common in finance, robotics with uncertain actuators, and networked systems) and regime switching makes the problem significantly harder.

Theoretical Advance: It resolves the question of control parametrization, showing that randomization is structurally required for optimality in this setting.
Computational Tractability: By converting a complex non-convex stochastic control problem into a convex SDP, it makes these problems solvable for practical horizons.
Practical Relevance: The framework is directly applicable to high-stakes domains like financial risk management (dynamic hedging), aerospace (fault-tolerant control), and robotics, where systems operate under changing environments and uncertainties that scale with system activity.

In summary, the paper provides a mathematically rigorous, computationally efficient, and practically applicable solution for steering the statistical distribution of complex stochastic systems under uncertainty and regime changes.