TPK: Trustworthy Trajectory Prediction Integrating Prior Knowledge For Interpretability and Kinematic Feasibility

This paper proposes TPK, a trustworthy trajectory prediction framework that integrates class-specific interaction and kinematic priors to ensure physically feasible and interpretable predictions for mixed traffic agents, demonstrating improved reliability over state-of-the-art baselines on the Argoverse 2 dataset despite a minor trade-off in raw accuracy.

The Gist

Imagine you are teaching a robot to drive a car. You show it millions of hours of video footage so it can learn how people, cars, and bikes move. Eventually, the robot gets really good at guessing where everyone will be in the next few seconds.

But here's the problem: The robot is a genius at math, but it's terrible at common sense.

Sometimes, the robot might predict that a car will drive through a solid wall because the math said it was possible, or it might think a parked car is a huge threat while ignoring a speeding truck coming right at it. It's like a chess player who knows all the rules but doesn't understand why a move is good or bad.

This paper introduces a new way to make self-driving cars trustworthy. The authors, Marius Baden and his team, built a system called TPK (Trustworthy Trajectory Prediction). They didn't just make the robot smarter; they gave it a "gut feeling" and a "physics check" to ensure its predictions make sense to humans and obey the laws of nature.

Here is how they did it, using some simple analogies:

1. The "Gut Feeling" (Interaction Prior)

The Problem:
Imagine you are walking down a busy street. You naturally pay attention to the person running toward you, but you ignore the person standing still three blocks away.
Current AI models often get this wrong. They might stare at the person three blocks away and ignore the runner, simply because the math in their "brain" got confused.

The Solution:
The team gave the AI a Rulebook (called DG-SFM). Think of this like a human instinct.

• The Analogy: Imagine the AI has an invisible "personal space bubble" around the car. If someone is running fast toward that bubble, the Rulebook screams, "Pay attention! Danger!" If someone is parked far away, the Rulebook whispers, "Ignore them."
• The Result: Instead of the AI guessing randomly, it is guided by these rules. The researchers found that when the AI's "attention" matched this Rulebook, it made fewer mistakes. If the AI started ignoring the Rulebook, it was a sign that it was about to make a bad prediction. It's like a teacher raising their hand to say, "Wait, that doesn't make sense!"

2. The "Physics Check" (Kinematic Feasibility)

The Problem:
Even if the AI guesses who to look at, it might guess how they move in impossible ways.

• The Analogy: Imagine the AI predicts a pedestrian will instantly teleport 10 feet to the left, or a car will turn 90 degrees in a split second without slowing down. In the real world, humans and cars can't do that. They have weight, momentum, and limits.
• The Issue: The data the AI learns from (the videos) is messy. Sometimes the cameras are shaky, making a car look like it teleported. The AI learns these "glitches" as if they are real moves.

The Solution:
The team built a Physics Filter (Kinematic Layers) at the end of the AI's brain.

• The Analogy: Think of the AI as a creative writer who writes a story about a car driving. The Physics Filter is the editor who says, "Whoa, cars can't turn that fast. Rewrite that sentence."
• The Innovation: They created a special version of this filter just for pedestrians. Most previous filters were designed for cars (which turn like bicycles). But people walk differently; they can stop, start, and wiggle. The team invented a new "Double Integrator" model that understands how humans actually move—allowing them to change direction smoothly but not instantly teleport.

3. The Trade-Off: Accuracy vs. Trust

You might ask: "Does adding these rules make the AI worse at guessing?"

The Answer: Yes, slightly.

• The Analogy: Imagine a race car driver who is so fast they sometimes drive off the track because they are pushing the limits. If you put guardrails on the track (the Physics Filter), they might be a tiny bit slower because they can't drive on the grass anymore.
• The Reality: The AI's raw accuracy dropped a tiny bit because it stopped "cheating" by predicting impossible moves. But, it stopped predicting impossible moves entirely.
• Why this matters: In self-driving, a prediction that is 99% accurate but includes a car flying through the air is useless. A prediction that is 98% accurate but obeys the laws of physics is safe.

The Big Picture

The authors compared their new system to the current "best" AI (called HPTR).

• Old AI: "I think that car will turn left because the math says so," (even if it's physically impossible).
• New AI (TPK): "I think that car will turn left. I know this because the Rulebook says it's paying attention to the right things, and the Physics Filter says it's actually possible for a car to do that."

In summary: This paper isn't just about making the robot faster at guessing. It's about making the robot explainable and safe. By giving the AI a set of rules to follow and a physics check to pass, the team created a system that doesn't just predict the future; it predicts a future that humans can trust.

Technical Summary

Here is a detailed technical summary of the paper "TPK: Trustworthy Trajectory Prediction Integrating Prior Knowledge for Interpretability and Kinematic Feasibility."

1. Problem Statement

While deep learning models have advanced trajectory prediction for autonomous driving, they suffer from a lack of trustworthiness. Current state-of-the-art (SOTA) models often produce:

• Physically Infeasible Predictions: Trajectories that violate kinematic constraints (e.g., impossible accelerations or sharp turns).
• Illogical Reasoning: Attention mechanisms that assign high importance to irrelevant agents, contradicting human intuition and safety logic.
• Lack of Generalization: Existing "prior knowledge" approaches (like Social Force Models) often focus exclusively on pedestrians or vehicles, failing to generalize to mixed traffic scenarios involving cars, cyclists, and pedestrians simultaneously.

The authors argue that improving interpretability and physical feasibility is more critical for real-world deployment than marginal gains in raw accuracy, especially given the presence of noise and infeasible data in ground-truth datasets.

2. Methodology

The proposed framework, TPK, builds upon the SOTA transformer-based model HPTR but introduces two major architectural modifications to enhance trustworthiness: a Class-Specific Interpretable Encoder and a Feasible Decoder.

A. Interpretable Encoder (Interaction Priors)

To address illogical agent interactions, the authors replace the standard attention mechanism with a Class-Specific Agent-to-Agent Layer guided by a novel prior.

• Class-Specific Layers: Instead of a single interaction layer, TPK uses separate instances for vehicles, pedestrians, and cyclists. This allows the model to learn distinct behavioral patterns for each class and mitigates data imbalance issues.
• DG-SFM (Directed-Gradient Social Force Model): A novel rule-based prior is introduced to calculate interaction importance scores ($\beta$). Unlike standard SFM, DG-SFM accounts for:
  • Directionality: It stretches the repulsive potential into an "egg shape" along the agent's velocity vector.
  • Gradient: It calculates the rate of change of the potential as agents move, effectively capturing how quickly a neighbor is approaching the focal agent.
• Integration Strategies: The paper compares two methods for integrating the prior into the attention mechanism:
  1. Multiply-and-Renormalize (MnR): Multiplies predicted attention by the prior.
  2. Gating-and-Loss (GnL): Uses a learnable gating MLP to balance the predicted attention and the prior, supplemented by an auxiliary Kullback-Leibler (KL) divergence loss to prevent the network from ignoring the prior.
  • Result: The GnL method with DG-SFM was found to yield the strongest correlation between attention deviations and prediction errors, enhancing interpretability.

B. Feasible Decoder (Kinematic Priors)

To ensure physical realism, the decoder predicts control inputs (action-space) rather than direct Cartesian trajectories, which are then passed through class-specific kinematic layers.

• Vehicles & Cyclists: Utilize the Unicycle Model, chosen for computational efficiency and lack of need for online parameter estimation.
• Pedestrians: The authors introduce and evaluate a Double Integrator Model (predicting x/y accelerations).
  • Comparison: The Single Integrator is too flexible (ignores acceleration limits); the Unicycle is too restrictive (unnatural turning constraints). The Double Integrator offers the best trade-off, allowing flexible heading changes while constraining velocity via acceleration limits.
• Constraint Enforcement: Feasibility limits (e.g., max acceleration $\pm 8 m/s^2$, max curvature) are enforced via tanh functions and clipping on the control inputs before trajectory generation.

3. Key Contributions

1. Heterogeneous Interaction Prior: A novel DG-SFM prior applicable to all agent classes (vehicles, pedestrians, cyclists) that aligns interaction scoring with human intuition regarding risk and approach speed.
2. Enhanced Interpretability: Integration of the prior into the attention mechanism (via GnL) creates a measurable correlation between attention deviations and prediction errors, allowing for potential monitoring systems to flag unreasonable predictions.
3. Guaranteed Kinematic Feasibility: The introduction of class-specific kinematic layers, including a novel Double Integrator model for pedestrians, ensures all output trajectories adhere to physical laws, eliminating infeasible predictions found in datasets and baseline models.

4. Experimental Results

The model was benchmarked on the Argoverse 2 (AV2) dataset against HPTR and other baselines.

• Interpretability:
  • A strong positive correlation ($\rho = 0.23$) was observed between the difference in attention scores (Prior vs. Network) and prediction error (minADE).
  • This confirms that when the model's attention deviates from the DG-SFM prior, the prediction is likely incorrect, validating the prior's utility for interpretability.
• Feasibility:
  • Ground Truth Noise: 27% of ground-truth trajectories in AV2 were found to be kinematically infeasible.
  • Baseline Failure: The standard HPTR model produced 88% infeasible trajectories (due to learning dataset noise).
  • TPK Success: TPK achieved 0% infeasible trajectories, successfully filtering out dataset noise and ensuring physical realism.
• Accuracy Trade-off:
  • TPK showed a slight decrease in raw accuracy metrics (e.g., minFDE) compared to the official HPTR.
  • However, the authors argue this is a necessary trade-off. The accuracy drop is attributed to the model refusing to learn the "noise" (infeasible paths) present in the ground truth, thereby producing more reliable and deployable predictions.
  • The Double Integrator model for pedestrians achieved the best balance of reproduction error and physical realism.

5. Significance

This paper shifts the paradigm in trajectory prediction from purely optimizing accuracy metrics to prioritizing trustworthiness.

• Safety Critical: By guaranteeing kinematic feasibility, the system prevents autonomous vehicles from planning impossible maneuvers.
• Explainability: The correlation between attention scores and the DG-SFM prior provides a mechanism to "audit" the model's reasoning, identifying scenarios where the model is "hallucinating" interactions.
• Mixed Traffic Robustness: The framework successfully handles the complexity of mixed traffic (pedestrians, cyclists, cars) without relying on separate models for each class, addressing a major gap in current literature.

In conclusion, TPK demonstrates that integrating domain knowledge (priors) into deep learning architectures is essential for creating autonomous driving systems that are not only accurate but also safe, logical, and physically grounded.