⚡ electrical engineering

Linear Quadratic Control with Non-Markovian and Non-Semimartingale Noise Models

This paper utilizes rough path theory, specifically signature representations and controlled rough paths, to solve a generalized linear quadratic optimal control problem involving non-Markovian and non-semimartingale noise processes with low Hölder regularity.

Original authors: Mostafa M. Shibl, Sharan Srinivasan, Harsha Honnappa, Vijay Gupta

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Mostafa M. Shibl, Sharan Srinivasan, Harsha Honnappa, Vijay Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a self-driving car how to stay in its lane on a highway.

The Old Way: The "Smooth Road" Assumption

Standard engineering math (called LQG control) assumes the world is a bit like a gentle, predictable breeze. It assumes that any "noise"—like wind hitting the car or a bump in the road—follows a very specific pattern called "Brownian motion."

Think of Brownian motion like a series of tiny, random, but relatively "polite" nudges. Because these nudges are predictable in their randomness, engineers have a perfect mathematical playbook to counteract them. If the wind nudges the car left, the math says, "Don't panic, just nudge it right by exactly this much."

The Problem: The "Wild World" Reality

The authors of this paper argue that the real world isn't "polite." Real-world noise is often non-Markovian and non-semimartingale. That sounds like a mouthful, but here is what it actually means:

Non-Markovian (The "Memory" Problem): Standard math assumes the future only depends on where you are now. But real noise has a memory. Imagine driving through a storm where the wind isn't just random; it has a rhythm. If a gust hits you now, it’s highly likely a similar gust will hit you in three seconds. The "noise" has a history.
Non-Semimartingale (The "Jagged" Problem): Standard math assumes the road is mostly smooth. But some real-world noise is incredibly "jagged" and "rough"—like driving over a field of broken glass or through a sudden earthquake. The movements are so sharp and irregular that the old mathematical tools literally break; they can't even calculate the "average" movement because the path is too chaotic.

If you use the "Smooth Road" playbook in a "Broken Glass" world, your self-driving car will overreact, panic, and eventually fly off the road.

The Solution: "Rough Path Theory" (The High-Definition Map)

The researchers introduced a new mathematical framework called Rough Path Theory.

If standard calculus is like looking at a blurry, low-resolution photo of a road, Rough Path Theory is like looking at a high-definition, 3D topographical map.

Instead of just looking at where the car is, this theory looks at the "signature" of the noise. It doesn't just see a nudge; it sees the shape, the rhythm, and the intensity of the turbulence. It uses something called "iterated integrals"—think of this as the math's way of saying, "I'm not just looking at the current bump; I'm calculating how the last five bumps are likely to influence the next one."

The Results: Staying on the Road

To prove this works, they tested it on a digital "Inverted Pendulum" (a classic physics test where you try to balance a stick on your finger).

The Old Controller: When they hit the pendulum with "jagged" noise (like a sudden earthquake or a heavy-tailed jump), the old controller panicked. It tried to correct too hard, hit its limit, and the pendulum went flying wildly out of control.
The New "gLQ" Controller: The new controller saw the "jaggedness" coming. It used its "memory" and its "high-def map" to make smooth, intelligent corrections. Even when the noise was chaotic and unpredictable, the pendulum stayed perfectly upright.

Summary in a Nutshell

The Paper's Core Message: Most control systems are designed for a "polite" world. This paper provides a new mathematical toolkit for a "wild" world, allowing machines to remain stable and safe even when faced with the most jagged, unpredictable, and "memory-heavy" chaos nature can throw at them.

Technical Summary: Linear Quadratic Control with Non-Markovian and Non-Semimartingale Noise Models

1. Problem Statement

The paper addresses a fundamental limitation in classical stochastic control theory: the Linear Quadratic Gaussian (LQG) framework. Standard LQG assumes that system and measurement noises are Brownian processes (semimartingales), which allows for the use of Itô calculus and the derivation of the separation principle (Kalman filtering + HJB equation).

However, many real-world phenomena (e.g., financial markets, turbulent flows, biological systems) exhibit non-Markovian behavior (memory effects) and non-semimartingale characteristics (low Hölder regularity). A prime example is fractional Brownian motion (fBm) with a Hurst index $H < 0.5$ . In such cases, the noise cannot be decomposed into a local martingale and a finite variation process, rendering classical Itô calculus inapplicable. The authors define this as the generalized Linear Quadratic (gLQ) problem.

2. Methodology

To overcome the mathematical breakdown of Itô calculus, the authors employ Rough Path Theory. This framework generalizes stochastic calculus by considering not just the path of a process, but also its "lift"—higher-order iterated integrals (such as Lévy areas) that capture the fine-grained structure of the noise.

Key methodological components include:

Rough Differential Equations (RDEs): The system dynamics are modeled using RDEs, which provide a well-posed way to define solutions for drivers with low Hölder regularity ( $H \in (1/3, 1]$ ).
Controlled Rough Paths: The authors use the concept of controlled paths to define rough integrals via the Sewing Lemma, ensuring the mathematical rigor of the state evolution.
Rough Hamilton-Jacobi-Bellman (HJB) Equation: The authors extend the HJB framework to the rough setting, utilizing the continuity properties of rough path lifts to ensure that the value function is well-defined and solvable.
Signature Representations: The framework leverages the algebraic structure of iterated integrals to handle the non-semimartingale nature of the noise.

3. Key Contributions

The paper provides a rigorous mathematical foundation for the gLQ problem through three primary theoretical results:

Optimal Controller with Full State Observation (Theorem 2): The authors derive an optimal control law $u^*(t) = -R^{-1}B^\top P(x(t) + V(t))$ . Unlike classical LQR, this includes a predictive correction term $V(t)$ , which uses the rough path lift to estimate and offset the expected impact of future noise increments on the state trajectory.
Optimal Observer for Correlated Noise (Theorem 4): The authors extend the separation principle to the gLQ setting. They derive an optimal observer gain $L$ that accounts for the cross-correlation between process noise and measurement noise, even when both are non-semimartingales.
Pathwise Optimality and Continuity (Theorem 5): A significant theoretical contribution is proving that while Itô solution maps are generally discontinuous, the rough path solution maps are locally uniformly continuous. This ensures that the control strategy is robust to small perturbations in the observed noise paths.

4. Results and Numerical Validation

The authors validate their theory using an inverted pendulum model subjected to two types of highly irregular noise:

Fractional Brownian Motion (fBm) with $H = 0.35$ : The classical LQ controller fails completely, leading to exponential divergence of the state and saturated control effort. The proposed gLQ controller successfully stabilizes the system, maintaining bounded states and efficient control.
Lévy Jump Processes (Stable Distribution with $\alpha = 1.5$ ): The heavy-tailed nature of the noise causes the classical controller to fail. The gLQ controller demonstrates robustness, effectively managing the jumps and regulating the system to equilibrium.

5. Significance

This work is significant because it bridges the gap between advanced stochastic analysis and practical control engineering. By providing a mathematically rigorous way to handle "rough" and "memory-heavy" noise, it:

Expands the scope of LQ control to include complex physical and financial phenomena that were previously considered "unmodelable" by standard LQG.
Provides a robust alternative to classical methods in environments where Brownian assumptions lead to system failure.
Sets a foundation for future research in integrating rough path theory with Reinforcement Learning (RL), specifically through the use of path signatures to represent long-term dependencies in state spaces.