⚡ electrical engineering

Optimal Modified Feedback Strategies in LQ Games under Control Imperfections

This paper proposes an optimal modified feedback strategy for two-player finite-horizon linear quadratic nonzero-sum games that compensates for implementation imperfections by augmenting the nominal game with measurable deviation dynamics, thereby locally outperforming standard equilibrium-derived feedback in terms of state trajectory and cost.

Original authors: Mahdis Rabbani, Navid Mojahed, Shima Nazari

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mahdis Rabbani, Navid Mojahed, Shima Nazari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: When Perfect Plans Meet Messy Reality

Imagine two expert dancers, Player 1 and Player 2, rehearsing a complex routine. They have spent months calculating the perfect steps (the Nash Equilibrium) so that neither of them can improve their performance by changing their own moves alone. They know exactly where to step, when to spin, and how to hold hands to minimize their "effort" and stay in sync.

However, in the real world, things rarely go exactly according to the script. Maybe Player 2 has a stiff ankle, or their shoes are slippery. When they try to execute a perfect spin, they actually end up sliding a little bit instead. This is what the paper calls "Control Imperfections."

The paper asks a crucial question: If Player 2 messes up their step, what happens to Player 1? And more importantly, can Player 1 adjust their own moves to save the dance, even if Player 2 keeps stumbling?

The Problem: The "Ripple Effect"

In the world of engineering (like self-driving cars or robots), players often rely on mathematical formulas to control their machines. These formulas assume the machine does exactly what it's told.

But in reality:

Motors are slow: They don't start or stop instantly.
Signals lag: There's a tiny delay between sending a command and the machine moving.
Errors happen: The machine might not move the exact distance calculated.

The authors found that when Player 2 has these "stiff ankles" (delays or errors), it creates a ripple effect. Because the two players are connected (like the dancers holding hands), Player 1's path gets thrown off, and Player 1 ends up paying a higher "cost" (using more energy or ending up in the wrong spot) than planned.

The Solution: The "Compensated Feedback" (The Smart Partner)

Most people would say, "Well, Player 1 just has to deal with it, or they should both try to re-calculate a new perfect dance."

But this paper proposes a clever middle ground. Player 1 doesn't need to change the whole dance routine. Instead, Player 1 can become a super-observant partner.

Here is the strategy:

Watch the Partner: Player 1 watches Player 2's actual movements in real-time.
Predict the Slip: Player 1 notices, "Oh, Player 2 is sliding a bit to the left because their shoe is slippery."
Adjust the Step: Instead of just following the original script, Player 1 adds a tiny "correction" to their own move. If Player 2 is sliding left, Player 1 steps slightly right to counterbalance it.

The paper calls this a Compensated Feedback Law. It's like Player 1 is wearing a pair of glasses that lets them see the "error" in Player 2's movement and instantly adjust their own body to keep the dance smooth.

The Math Metaphor: The "Augmented System"

To make this work, the authors used some heavy math (Riccati equations), but think of it like this:

Usually, a dancer only thinks about their own position.

Old Way: "I am at position X. I need to move to Y."

The new way (The Augmented System) is like the dancer thinking about two things at once:

New Way: "I am at position X. AND my partner is currently sliding 2 inches to the left. Therefore, I need to move to Y plus an extra 2 inches to the right to cancel out their slide."

By treating the "slip" as a known variable that can be measured, the math proves that Player 1 can always do better (or at least no worse) than just blindly following the original plan.

The Numerical Example: The Two Carts

To prove this works, the authors simulated a scenario with two carts connected by a spring and a damper (like two shopping carts tied together with a bungee cord).

Goal: Both carts want to stop exactly at the center line.
The Glitch: Player 2's cart has a "lazy motor" (a lag). When told to stop, it keeps rolling a bit too far.
The Result:
- Scenario A (Perfect World): Both carts stop perfectly at the center.
- Scenario B (The Problem): Player 2's lazy motor makes it overshoot. Because they are tied together, Player 1 gets dragged off course too. Both end up far from the target.
- Scenario C (The Fix): Player 1 sees Player 2's lazy motor dragging them. Player 1 pushes harder in the opposite direction to pull Player 2 back.
- Outcome: Player 1 ends up very close to the target (saving themselves), even though Player 2 is still a bit off.

Why This Matters

This research is a game-changer for robotics and autonomous systems (like self-driving cars or drone swarms).

In the past, if one robot in a team had a glitchy motor, the whole team's plan might fail, or everyone would have to stop and re-calculate a new plan, which takes time.

This paper shows that one smart agent can adapt on the fly. It doesn't need to wait for a new plan. It can just "feel" the other agent's mistake and adjust its own behavior instantly to keep the mission successful.

Summary in One Sentence

When your partner in a team effort makes a mistake due to physical limitations, you don't have to fail with them; by watching their error in real-time and adjusting your own moves to cancel it out, you can save the performance.

1. Problem Formulation

The paper addresses a critical gap in Game-Theoretic control: the discrepancy between theoretical Nash Equilibrium (NE) strategies and their practical implementation in engineering systems.

Context: The study focuses on a finite-horizon, discrete-time, two-player Linear Quadratic (LQ) nonzero-sum game.
The Challenge: In real-world applications (e.g., robotics, autonomous vehicles), control inputs are subject to imperfections such as actuator lags, delays, and tracking errors. Consequently, a player's implemented input ( $u_{i,k}$ ) often deviates from the nominal Nash command ( $u^*_{i,k}$ ).
The Specific Problem: The authors investigate how small, measurable deviations by Player 2 affect the state trajectory and, crucially, the cost function of Player 1. They ask: If Player 2 cannot perfectly execute their optimal strategy due to hardware limitations, how should Player 1 modify their own strategy to mitigate the resulting performance degradation?

2. Methodology

The authors employ a three-pronged methodological approach: sensitivity analysis, augmented system modeling, and optimal control design.

A. First-Order Sensitivity Analysis

The authors first derive closed-form expressions to quantify the impact of Player 2's deviations ( $\Delta u_2$ ) on Player 1's cost ( $J_1$ ).

They define the state perturbation $\Delta x_k$ caused by $\Delta u_2$ using the closed-loop state transition matrix $\Phi(k, j)$ .
They derive a first-order expansion of the cost deviation $\Delta J_1$ . This reveals that the cost change is linearly proportional to the deviation sequence, weighted by a sensitivity matrix $\Lambda_j$ that depends on the game's Riccati solutions and time-varying dynamics.
Key Insight: Without compensation, Player 1's standard Feedback Nash Equilibrium (FNE) policy becomes sub-optimal when Player 2's execution is imperfect.

B. Augmented System Modeling

To design a compensator, the authors model the execution error as a dynamic disturbance.

Disturbance Model: They assume Player 2's actuator follows a discrete-time first-order lag (tracking error dynamics). The disturbance $w_k$ is defined as the difference between the implemented and nominal inputs, scaled by the input matrix.
Augmented State: They construct an augmented state vector $z_k = [x_k^\top, w_k^\top]^\top$ , combining the system state and the measurable (or estimable) disturbance.
Policy Class: They restrict the search for a new policy to the causal affine class: $u_{1,k} = -K_k x_k - L_k w_k$ . This structure allows Player 1 to use state feedback ( $K_k$ ) and a feedforward term ( $L_k$ ) based on the observed deviation.

C. Optimal Compensation Design

The core contribution is the derivation of an optimal compensated feedback (CF) law.

By treating the augmented system as a standard LQ control problem, they apply the Bellman principle of optimality.
They derive a coupled finite-horizon Riccati recursion for the augmented system matrices ( $\bar{A}_k, \bar{B}_k, \bar{Q}$ ).
This yields unique optimal gains $\{K_k, L_k\}$ that minimize Player 1's cost given that Player 2 remains fixed at their nominal (but imperfectly executed) strategy.
Theoretical Guarantee: The paper proves that this CF policy is uniquely optimal within the affine class and strictly outperforms the uncompensated reference feedback (REF) for sufficiently small deviations (proven via a second-order cost gap analysis).

3. Key Contributions

Sensitivity Analysis: Derivation of explicit, closed-form expressions characterizing how one player's control deviations propagate through discrete-time LQ game dynamics to affect the opponent's cost.
Compensation Law: Development of a structured compensation policy that augments the nominal game dynamics with measurable deviation dynamics. This policy is proven to be optimal within a causal affine class.
Local Performance Improvement: Rigorous proof showing that the proposed compensated feedback reduces the cost for the influenced player to the second order of the deviation magnitude compared to the standard Nash feedback.
Distinction from Nash Equilibrium: The authors clarify that the proposed policy is not a new Nash Equilibrium for the modified game (since Player 2 does not adapt). Instead, it is a best-response compensation strategy for Player 1 against a fixed, imperfect opponent.

4. Results (Numerical Example)

The authors validate their theory using a two-cart system coupled by a spring-damper.

Setup: Two carts (Player 1 and Player 2) must regulate to the origin. Player 2's actuator is subjected to a first-order lag ( $\tau=0.8s$ ).
Scenarios:
- Case I (FNE): Ideal execution (no lag). Baseline performance.
- Case II (REF): Player 2 has lag; Player 1 uses standard FNE (uncompensated).
- Case III (CF): Player 2 has lag; Player 1 uses the proposed Compensated Feedback.
Findings:
- Case II vs. Case I: The lag caused Player 1's cost to increase by 51.36%.
- Case III vs. Case II: The proposed CF reduced Player 1's cost by 10.71% relative to the uncompensated lag case.
- Recovery: The CF strategy recovered approximately 31.56% of the performance loss induced by the lag.
- Trade-off: While Player 1 improved significantly, Player 2's cost increased further in Case III. This highlights the asymmetric nature of the benefit in a general-sum game; the design is player-specific and not socially optimal (Pareto optimal).

5. Significance and Implications

Robustness to Implementation Errors: The paper moves beyond theoretical equilibrium concepts to address the "reality gap" in engineering systems where actuators are imperfect.
Practical Applicability: The method relies on measurable or estimable deviations. In many modern systems (e.g., networked control, autonomous vehicles), telemetry or state estimation can provide the necessary data ( $w_k$ ) to implement this compensation without requiring Player 2 to change their strategy.
Theoretical Extension: It extends classical disturbance accommodation and preview control principles to the domain of multi-agent game theory, showing how a rational agent can adapt to an opponent's physical limitations.
Future Directions: The work suggests future research into online estimation of opponent deviations and adaptive laws that could dynamically adjust to time-varying or correlated error patterns.

In summary, this paper provides a mathematically rigorous framework for a player in a game-theoretic setting to optimally adapt to an opponent's implementation errors, significantly recovering performance losses that would otherwise occur if standard Nash strategies were blindly applied.