Reach-avoid games for players with damped double integrator dynamics

1. Problem Formulation

The paper addresses a reach-avoid differential game involving two players: an Attacker (A) and a Defender (D).

• Dynamics: Both players follow damped double integrator dynamics, which is a more realistic model than simple motion (constant velocity) or standard double integrators. The state includes position and velocity, and the control input is acceleration with a magnitude limit and a damping factor ($\mu > 0$).
  $$\dot{x}_i = v_i, \quad \dot{v}_i = -\mu v_i + u_i \mathbf{d}(\theta_i)$$
  where $u_i \in [0, u_{im}]$ is the control magnitude and $\theta_i$ is the direction. The defender has a higher maximum acceleration capability ($u_{Dm} > u_{Am}$).
• Objectives:
  • Attacker: Attempts to reach a target point (origin) without being captured.
  • Defender: Attempts to capture the attacker (co-locate positions) as far as possible from the target.
• Game Type: A zero-sum game where the payoff is the distance of the capture point from the target. The solution seeks Nash equilibrium strategies.

2. Methodology

The authors employ a combination of Differential Game Theory (Hamilton-Jacobi-Isaacs framework) and Geometric Analysis.

A. Optimal Control and Trajectory Analysis

• Using the Hamiltonian, the authors derive that the optimal control strategy for both players involves constant acceleration magnitude ($u_i = u_{im}$) and a constant direction ($\theta_i$). This is termed the "Normal Strategy."
• The trajectories under Normal Strategy are analytically derived, showing that position and velocity evolve with exponential decay terms due to damping.

B. Multiple Reachable Region (MRR)

A critical geometric insight is introduced regarding the Isochrone (the set of points reachable at time $t$).

• Unlike simple motion models where isochrones are concentric circles, damped double integrator isochrones are circles with time-varying centers and radii.
• Key Finding: Due to the damping and initial velocity, a single point in the state space can be reached by a player via multiple distinct Normal strategies (specifically, 1, 2, or 3 different strategies).
• MRR Definition: The Multiple Reachable Region ($M_i$) is defined as the set of points reachable by more than one Normal strategy.
• Reaching Times: For a point $x \in M_i$, there are three distinct reaching times: $t_{i1} < t_{i2} < t_{i3}$.
  • $t_{i1}$ and $t_{i3}$ correspond to "local minimal" reaching times.
  • $t_{i2}$ corresponds to a "local maximal" reaching time (the player enters the region, then exits, then re-enters).
• The boundary of the MRR is formed by self-intersection points of the isochrones.

C. Attacker's Dominance Region (ADR) Redefinition

The paper redefines the Attacker's Dominance Region (points the attacker can reach before the defender) to account for the MRR properties.

1. $R_I$ (Standard): Points where the attacker can reach via a Normal strategy before the defender, ensuring no capture along the path.
2. $R_{II}$ (Vulnerable): Points where the attacker arrives earlier than the defender but is captured en route because the optimal trajectory is not a straight line.
3. $R_{III}$ (New Contribution): A novel region introduced specifically for damped dynamics. These are points inside the defender's MRR ($M_D$) where the attacker can reach safely by reducing its acceleration magnitude ($u_A < u_{Am}$). By slowing down, the attacker alters its isochrone shape to avoid the defender's capture zone, effectively reaching points that would be unreachable under maximum acceleration.

3. Key Contributions

1. Introduction of MRR: The paper characterizes the Multiple Reachable Region for damped double integrator dynamics, proving that points can be reached via 1 to 3 distinct Normal strategies with specific temporal ordering ($t_1 < t_2 < t_3$).
2. Novel Dominance Region ($R_{III}$): The authors identify and mathematically define a third type of dominance region ($R_{III}$). They prove that in this region, the attacker can achieve a better outcome by intentionally reducing acceleration to synchronize arrival with the defender at a specific time, thereby avoiding capture.
3. Optimal Strategy Derivation:
   • ADR Strategy: Used when the optimal terminal point lies on the boundary of the standard dominance region ($L$). Both players use maximum acceleration.
   • MRR Strategy: Used when the optimal terminal point lies in $R_{III}$. The defender uses maximum acceleration, while the attacker uses a reduced acceleration ($\hat{u}_A$) calculated to arrive exactly when the defender does, satisfying the necessary optimality conditions.
4. Necessary Condition Proof: The paper proves that the proposed strategies satisfy the Hamilton-Jacobi-Isaacs (HJI) equation (specifically $H=0$), confirming they are optimal under the Nash equilibrium sense.

4. Results and Simulations

Numerical simulations validate the theoretical findings across three scenarios:

• Case 1 (Standard ADR): When the optimal point is in $R_I$, the ADR strategy (max acceleration for both) is optimal. The trajectories overlap with open-loop predictions, and unilateral deviation worsens the payoff.
• Case 2 (MRR Strategy in $R_{III}$): When the optimal point is in $R_{III}$, the standard ADR strategy fails (the optimal point shifts dynamically). The attacker successfully reaches the target by adopting the MRR strategy (reducing acceleration), while the defender maintains max acceleration. This results in a simultaneous arrival at the target point, which is the saddle point.
• Case 3 (Crossing $R_{II}$): Demonstrates that if the attacker attempts to cross the vulnerable region $R_{II}$ without adjusting strategy, the defender can intercept the attacker before it reaches the target.

5. Significance and Conclusion

• Realism: This work bridges the gap between simplified geometric game models and real-world dynamics where damping (e.g., air resistance, friction) is significant.
• Strategic Depth: It reveals that "faster is not always better." In damped systems, slowing down (reducing control input) can be an optimal strategy to manipulate the geometry of reachable regions and avoid capture.
• Future Work: The authors note that the exact shape of $R_{III}$ is not fully determined in all cases, and scenarios involving non-convex dominance regions or multiple local minima require further investigation.

In summary, this paper provides a rigorous analytical framework for reach-avoid games with damped dynamics, introducing the concept of Multiple Reachable Regions and a new class of dominance regions that fundamentally change optimal strategies by allowing the attacker to trade speed for safety.