Online Intention Prediction via Control-Informed Learning

1. Problem Definition

The paper addresses the challenge of online intention prediction for autonomous systems operating in uncertain and dynamic environments. Specifically, the goal is to estimate the system's goal state ($x^*_g$) in real-time based on observed state trajectories.

The problem formulation introduces three critical complexities that existing methods often fail to handle simultaneously:

• Time-Varying Goals: The autonomous agent may switch its goal state at unknown times during execution (e.g., adversarial behavior or dynamic mission changes).
• Unknown System Parameters: The system dynamics ($f$) and the control objective function (weights $\omega$) contain unknown constant parameters.
• Continuous Goal Space: Unlike methods that select from a predefined discrete set of goals, this approach estimates the goal in a continuous state space.

The core objective is to develop an algorithm that updates the goal estimation $\hat{x}_{g,t}$ such that the estimation error converges to zero ($||\hat{x}_{g,t} - x^*_{g,t}|| \to 0$) despite noisy measurements and potential goal switches.

2. Methodology

The proposed solution, termed Online Control-Informed Learning (OCIL) with a Shifting Horizon strategy, integrates inverse optimal control (IOC) with online state estimation.

A. Shifting Horizon Strategy

To handle time-varying goals, the algorithm does not optimize over the entire history of observations. Instead, it employs a shifting horizon (sliding window) of length $T_m$.

• Mechanism: At any time $t$, the predictor only considers observations from $\hat{t} = \max(t - T_m, 0)$ to $t$.
• Benefit: This "forgets" outdated information corresponding to previous goals, allowing the estimator to rapidly adapt when the agent switches objectives. It also mitigates the accumulation of noise and parameter drift.

B. Problem Formulation as IOC/IRL

The intention prediction is cast as an Inverse Optimal Control (IOC) or Inverse Reinforcement Learning (IRL) problem.

• The goal state $x^*_g$ is treated as an unknown parameter within the objective function, alongside unknown dynamics parameters ($p$) and cost weights ($\omega_r, \omega_f$).
• The system solves a forward optimal control problem over the shifting horizon using current parameter estimates to generate a predicted trajectory.
• The discrepancy between the predicted trajectory and the actual noisy observation is used to update the parameters.

C. Online Parameter Estimation via PDP

The estimation of the combined parameter vector $\theta_t = \text{col}\{p, \omega_r, \omega_f, x_g\}$ is performed using a recursive update law derived from Pontryagin Differentiable Programming (PDP).

• Gradient Computation: The algorithm computes the sensitivity of the state trajectory with respect to the parameters ($\frac{\partial x_t}{\partial \theta}$) using the Hamiltonian and costate equations (Lemma 2). This allows for efficient gradient calculation without backpropagation through time in the traditional deep learning sense.
• Update Law: A Kalman-filter-like update rule (Lemma 1) is applied. The gain matrix $K_t$ is computed using the Jacobian of the residual function. This ensures local asymptotic convergence of the estimated parameters to the true values, even as they change over time.

3. Key Contributions

1. Novel Problem Formulation: The paper formulates intention prediction for systems with time-varying goals and unknown parameters (both in dynamics and objectives), moving beyond the static goal assumption of prior work.
2. Shifting Horizon Strategy: A specific mechanism to discount outdated trajectory data, enabling the system to track goal switches effectively without being biased by historical data.
3. Control-Informed Online Learning: The integration of PDP to compute gradients for online parameter updates, allowing the system to simultaneously learn the goal, dynamics, and cost weights in real-time.
4. Continuous Goal Estimation: The ability to estimate goals in a continuous space rather than relying on a finite, predefined set of candidates.

4. Experimental Results

The authors validated the approach through simulations and hardware experiments on a quadrotor drone.

• Simulation (Noise Robustness): Under varying levels of Gaussian noise, the algorithm demonstrated fast convergence and accurate goal prediction. The average prediction time was 60 ms per update on a standard CPU, confirming real-time feasibility.
• Simulation (Goal Switching):
  • In scenarios with 2 and 3 goal switches, the proposed method rapidly reduced prediction loss after each switch.
  • Comparison: Compared to standard OCIL (without the shifting horizon), the proposed method significantly outperformed the baseline. Standard OCIL failed to converge when goals changed because it was heavily influenced by outdated observations.
  • The shifting horizon strategy consistently yielded lower prediction losses across different switching intervals.
• Hardware Experiments:
  • Real-world tests on a quadrotor in a motion-capture environment confirmed the algorithm's robustness.
  • The average prediction time was 92.3 ms, well within the 100 ms measurement cycle of the motion capture system.
  • The system successfully predicted the drone's intention to avoid a "no-fly zone" and reach a target, even as the drone executed complex maneuvers.

5. Significance

This work represents a significant advancement in human-robot interaction and autonomous collaborative systems.

• Adaptability: By handling time-varying goals and unknown dynamics, the system can react to adversarial agents or dynamic environments where pre-programmed models fail.
• Real-Time Performance: The computational efficiency makes it suitable for safety-critical applications where immediate reaction to an agent's changing intent is required.
• Generalization: The ability to estimate goals in continuous space without predefined candidates makes the framework applicable to a wider range of real-world scenarios where the set of possible goals is unknown or infinite.

In summary, the paper provides a robust, mathematically grounded framework for predicting the intentions of autonomous agents in complex, changing environments, bridging the gap between theoretical optimal control and practical, real-time application.