💻 computer science

Reinforcement Learning-Based Dynamic Event-Triggered Nonlinear Games Under Incomplete and Asymmetric Information

This paper proposes a reinforcement learning-based dynamic event-triggered interception strategy for unknown maneuvering targets under incomplete and asymmetric information that adaptively switches between optimal and game-based policies to reduce conservatism while ensuring system stability and excluding Zeno behavior.

Original authors: Tao Ren, Lei Shao, Shu Li

Published 2026-11-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tao Ren, Lei Shao, Shu Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the high-stakes arena of aerospace defense, the challenge of intercepting a maneuvering target is a race against time and physics. Imagine two objects moving through the sky: one is a defender trying to catch the other, while the other is an evader trying to escape. For decades, engineers have treated this interaction as a mathematical game of perfect information, assuming both sides know exactly what the other is thinking and planning. They also assumed the defender must always prepare for the worst possible move the enemy could make. While this "worst-case" approach guarantees safety, it often forces the defender to waste energy and overreact to harmless movements, much like a person jumping out of the way of a falling leaf. In the real world, however, information is rarely perfect. The defender often cannot see the enemy's true intentions or know exactly how much energy the enemy is willing to spend to escape. Furthermore, the computers on board these vehicles have limited power and cannot constantly process massive amounts of data. This creates a difficult puzzle: how can a defender make smart, efficient decisions when it does not know the enemy's full plan, and how can it do so without exhausting its computer's resources?

To solve this, researchers at the Air Force Engineering University and Liaoning University of Technology have developed a new method that combines artificial intelligence with a smarter way of making decisions. Instead of assuming the enemy is always playing a perfect, aggressive game, their system watches the enemy's actual movements in real time. It asks a simple but crucial question: is this specific maneuver actually threatening enough to require a complex, energy-intensive counter-strategy? If the enemy is moving weakly or predictably, the system ignores the complex game theory and uses a simpler, more efficient path to intercept. However, if the enemy suddenly makes a sharp, evasive move that genuinely endangers the mission, the system instantly switches to a sophisticated "game" mode, treating the target as a rational opponent and calculating the best possible counter-move. This adaptive switching allows the interceptor to remain robust against dangerous threats while avoiding the waste of energy on non-threatening ones.

The researchers also tackled the problem of not knowing the enemy's hidden preferences. In a real engagement, the defender cannot see the enemy's internal computer to know how much it values survival versus fuel consumption. To bridge this gap, the team built a digital tool that acts like a detective, reconstructing the enemy's hidden intentions by observing its physical movements. By analyzing how the target maneuvers, the system can estimate what the target values most and adjust its own strategy accordingly. This happens alongside a new way of running the computer's brain. Instead of updating its calculations every single fraction of a second, which would overwhelm the system, the new method only updates when something truly important changes. This "event-triggered" approach means the computer sleeps through unimportant moments and wakes up only when necessary, saving power and reducing the risk of errors caused by too much data.

In their tests, the team simulated a three-dimensional interception scenario where a defender tried to catch a target moving at high speeds. They compared their new method against older strategies that either ignored the enemy's intelligence or assumed the enemy was always playing the most aggressive game possible. The results showed that the old "always prepare for the worst" method worked, but it was inefficient, often using more fuel than necessary. The new adaptive method, however, achieved the same successful interception but with significantly less energy. In one specific test, the new method improved the accuracy of the final hit, reducing the distance between the interceptor and the target from a miss of 0.70 units to a precise hit of 0.44 units in a game-like scenario, and from 0.25 to 0.14 units in a scenario with erratic, square-wave movements. The simulations confirmed that the system could successfully estimate the enemy's hidden intentions and switch between simple and complex modes without ever getting stuck in a loop of constant, useless updates.

The study proves that it is possible to build an intelligent interception system that is both tough and efficient. By refusing to treat every enemy move as a maximum threat and by only doing heavy calculations when the situation demands it, the system finds a better balance between safety and resource conservation. The researchers verified that their mathematical proofs hold up, ensuring the system remains stable and does not behave erratically. While these findings were demonstrated through computer simulations rather than live flight tests, they offer a clear path forward for future aerospace defense. The work suggests that the next generation of defense systems will not just be faster or stronger, but smarter, capable of reading the room and adapting its strategy to the reality of the moment rather than a rigid, worst-case prediction.

Technical Summary: Reinforcement Learning-Based Dynamic Event-Triggered Nonlinear Games Under Incomplete and Asymmetric Information

Problem Formulation
The paper addresses the challenge of intercepting an unknown, maneuvering target in a three-dimensional nonlinear dynamic environment. The problem is characterized by three primary constraints:

Incomplete and Asymmetric Information: The interceptor lacks knowledge of the target's cost function parameters, specifically the weighting matrix ( $R_2$ ) that defines the target's trade-off between evasion performance and maneuvering energy consumption. The target's intention is private information, creating an asymmetric information structure.
Maneuverability Constraints: The interceptor operates under input saturation limits, requiring non-quadratic utility functions to handle control constraints.
Computational and Communication Burden: Traditional time-triggered reinforcement learning (RL) requires frequent updates and data transmission, which is inefficient for high-speed aerospace systems with limited onboard resources.

The interaction is modeled as a two-player zero-sum nonlinear game. However, the authors argue that treating the target as a worst-case adversary throughout the entire engagement leads to excessive policy conservatism and unnecessary energy consumption when the target's maneuvers are weak or intermittent.

Methodology
The proposed solution is a Dynamic Event-Triggered (DET) Composite Game Interception Method that integrates Reinforcement Learning (RL), disturbance observation, and adaptive switching. The framework consists of the following core components:

Composite Game Architecture with Adaptive Switching:
- An adaptive switching function evaluates the real-time influence of the target's maneuvers on the interception situation.
- If the target's maneuver significantly degrades the interception situation, the system activates a game-based mode, solving for a Nash Equilibrium (NE) strategy.
- If the maneuver influence is below a designed threshold, the system switches to a non-game optimal mode, treating the target as an estimable external disturbance. This reduces conservatism and control effort when the target is not acting as a rational adversary.
Target Intention Estimation Mechanism:
- To address the unknown target cost function, a disturbance observer is designed to reconstruct the target's actual maneuvering input from engagement measurements.
- By inverting the optimality mapping between the target's optimal strategy and its cost weights, the unknown intention weighting matrix ( $R_2$ ) is inferred online.
- This estimated intention is fed into a Virtual Actor Neural Network (NN), allowing the interceptor to approximate the true target policy and solve the game equations more accurately under incomplete information.
Nested Dual Dynamic Event-Triggered Mechanism:
- State-Triggered Mechanism: Determines when to update the neural network weights and transmit state information. It uses an internal dynamic variable to ensure the inter-event time is lower-bounded, excluding Zeno behavior.
- Situation-Triggered Mechanism: Determines when to switch between the game-based and non-game modes. Crucially, this mechanism is only evaluated at the triggering instants of the state-triggered mechanism. This nested design prevents frequent mode switching caused by small-amplitude fluctuations in the situation function.
Neural Network Approximation:
- Critic NN: Approximates the value function (cost-to-go).
- Actor NN: Approximates the interceptor's optimal policy.
- Virtual Actor NN: Approximates the virtual target's policy based on the estimated intention.
- The weights of these networks are updated online using gradient descent methods derived from the Hamilton-Jacobi-Isaacs (HJI) equation.

Key Contributions
The paper claims three main contributions:

Situation-Aware Composite Interception: Unlike existing methods that assume a worst-case target throughout the engagement, this work proposes an adaptive switching architecture. It activates game-based strategies only when necessary, preserving robustness against strong evasion while avoiding the energy waste of persistent worst-case assumptions during weak or non-adversarial target phases.
Online Intention Estimation in Adversarial Games: The paper introduces a mechanism to estimate the target's private intention weighting matrix online. By combining a disturbance observer with the inversion of the target's optimality condition, the interceptor recovers the unknown cost parameters, improving the accuracy of equilibrium strategy learning under asymmetric information.
Nested Dual Dynamic Event-Triggered Mechanism: The authors propose a dual-triggering structure where state updates and mode switching are decoupled but synchronized. This design reduces computational and transmission burdens while explicitly excluding Zeno behavior by construction, as both mechanisms are verified only at periodic sampling instants.

Results
Numerical simulations were conducted in a 3D interception scenario with a terminal time of 20 seconds. The results demonstrate:

Interception Success: The proposed composite strategy successfully intercepts targets performing both game-driven (rational) maneuvers and square-wave (non-game) maneuvers. In contrast, a pure non-game optimal strategy failed against game-driven targets, and a pure game strategy exhibited higher control effort.
Convergence: The weights of the Critic, Actor, and Virtual Actor neural networks converged to bounded steady values. The estimated target intention weighting matrix ( $\hat{R}_2$ ) converged toward the true target values despite initial estimation errors.
Efficiency: The dynamic event-triggered mechanism significantly reduced the number of policy updates and state transmissions compared to time-triggered schemes.
Performance Metrics: In the game-driven scenario, the interception accuracy (miss distance) improved from 0.70 to 0.44. In the square-wave scenario, accuracy improved from 0.25 to 0.14. The authors attribute this to the suppression of redundant updates and the smoothing of input commands.
Stability: Theoretical analysis proved that the closed-loop system states, neural network weight estimation errors, and disturbance observation errors are Uniformly Ultimately Bounded (UUB).

Significance and Claims
The paper positions this work as a solution to the trade-off between robustness and efficiency in nonlinear game-based interception. The authors claim that by dynamically switching between game and non-game modes, the method avoids the "excessive policy conservatism" inherent in traditional worst-case designs. Furthermore, by estimating the target's intention online, the method overcomes the limitation of assuming known opponent parameters, which is often unrealistic in real-world adversarial scenarios. The integration of dynamic event-triggering is highlighted as a practical necessity for reducing the computational load on aerospace systems with limited resources. The authors conclude that the method effectively balances interception performance, control effort, and policy conservatism, making it suitable for complex aerospace adversarial environments. Future work is suggested to extend the framework to cooperative multi-interceptor and multi-target scenarios.

Technical Summary: Reinforcement Learning-Based Dynamic Event-Triggered Nonlinear Games Under Incomplete and Asymmetric Information

More like this