🤖 AI

A Unified Framework for Trajectory Prediction with Explicit Planning and Reaction Decomposition

The paper proposes INTraJ, a unified trajectory prediction framework that decomposes social influence into a planning stage for generating reference trajectories and a reaction stage for local adjustments, achieving state-of-the-art performance across multiple benchmarks by validating the critical role of staged social modeling.

Original authors: Jiaheng Chen, Jiaxing Li, Tinghe Zhang, Chaopeng Guo

Published 2026-08-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiaheng Chen, Jiaxing Li, Tinghe Zhang, Chaopeng Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict where a friend will be in five minutes. If you just look at where they are walking right now, you might guess they'll keep going straight. But if you see a dog run toward them, or a group of friends waving, your guess changes. You anticipate the dog might make them stop, or the friends might make them turn. This is the heart of trajectory prediction, a field of computer science that tries to guess the future paths of moving things, like cars on a highway or people in a busy plaza. It's not just about math; it's about understanding how one moving object reacts to others. For self-driving cars to be safe, they can't just drive like robots ignoring the world; they need to "think" about how other drivers and pedestrians will move, and how those movements will change their own path. The big challenge has always been figuring out how to teach computers to do this kind of social thinking without getting confused by too many details at once.

Enter INTraJ, a new framework proposed by researchers that suggests we've been looking at this problem the wrong way. Instead of trying to predict the whole future path in one giant, messy brain-burst, INTraJ suggests we break the process down into two distinct steps: Planning and Reaction. Think of it like planning a road trip. First, you look at the map and decide your general route (the "Plan"). You know you need to go from City A to City B, and you've checked the traffic reports to see where the big jams might be. This is your "socialized plan." But once you're actually driving, you can't just stick to the map blindly. If a squirrel darts in front of your car, you swerve. If a construction crew blocks a lane, you merge. These are your "Reactions."

The paper argues that most current computer models try to do both the map-reading and the swerving at the exact same time, which makes them messy and prone to errors. INTraJ fixes this by separating the two. It first builds a "Socialized Intent"—a smooth, ideal path that the agent (like a car or a person) would take if they thought ahead about everyone else's future moves. Then, it adds a "Reaction" layer on top, which handles the quick, local adjustments needed when things don't go exactly as planned.

The researchers tested this idea on four different datasets, including real-world driving data (Argoverse 2) and crowd movement data (like people walking in a city). They found that by splitting the process into "Plan then React," the computer made much better predictions. Specifically, the final predictions were more accurate, especially for long-term guesses (like where someone will be in 5 seconds rather than 1 second). In the crowded city datasets, the new method reduced the final prediction error significantly, dropping the average error to 9.87 pixels on one dataset and improving long-term accuracy by over 12% on another. Even more impressively, this "two-step" approach worked just as well whether the computer was trying to predict the paths of many cars at once or just a single person in a crowd.

The paper explicitly rules out the idea that social influence is just a single, tangled web of information that needs to be processed all at once. Instead, it suggests that social influence plays different roles at different times: first, it shapes the big picture plan, and second, it triggers quick local fixes. The authors are confident in these results because they tested the method on multiple different computer "backbones" (the underlying engines that do the math) and saw consistent improvements every time. They even showed that the method is robust; even if the computer's guess about what other people will do is slightly wrong (like if it thinks a car will turn left when it actually goes straight), the INTraJ system doesn't crash. It handles the mistake gracefully, adjusting its own plan without falling apart.

In short, INTraJ teaches computers to be better at "socializing" by teaching them to think ahead before they act. It's a bit like telling a student, "Don't just react to the test questions as they come; read the whole exam first, make a plan, and then tackle the tricky parts." By separating the big picture from the small details, the researchers found a way to make trajectory prediction smoother, safer, and more reliable for the future of autonomous driving and crowd analysis.

Technical Summary: A Unified Framework for Trajectory Prediction with Explicit Planning and Reaction Decomposition

Problem Statement
Trajectory prediction aims to estimate the future motions of agents within a scene, a task critical for autonomous driving and crowd analysis. While recent methods have improved predictive capacity through coupled formulations that encode individual tendencies, scene constraints, and social interactions into a shared latent space, these approaches often entangle social effects from different sources and time scales. This entanglement increases model complexity and vulnerability to spuriously correlated neighbor patterns. Furthermore, existing structured methods typically treat social influence as a unified component of the prediction process, failing to distinguish its specific functional roles in trajectory formation. Specifically, current models rarely explicitly represent the "socially shaped planning" that occurs when an agent anticipates others' future behaviors before executing local reactive adjustments.

Methodology: The INTraJ Framework
The authors propose INTraJ, a unified framework that reformulates trajectory prediction as an intent-driven, two-stage process: Planning followed by Reaction. This approach explicitly decomposes social influence into these distinct stages rather than modeling them jointly.

Core Decomposition (Plan-React Module):
- Planning Stage: The model extracts a "pure intent" ( $I_{pure}$ ) from the target agent's history and local context. This seed is then updated using a "future social carrier" ( $z$ ) derived from surrounding agents to generate a "socialized intent" ( $I_{social}$ ). A decoder translates this into a planning-level reference trajectory ( $T_{ideal}$ ), which represents the agent's overall passing plan after anticipating future social behaviors.
- Reaction Stage: The model retains the backbone's full-context prediction ( $T_{full}$ ) to capture immediate interactions. The difference between the full prediction and the planning reference ( $S_{high} = T_{full} - T_{ideal}$ ) approximates local reactive adjustments (e.g., collision avoidance, speed changes).
- Fusion: A learnable gating mechanism ( $\alpha$ ) controls the injection of these reactive adjustments into the final trajectory: $T_{final} = T_{ideal} + \alpha \cdot S_{high}$ . This allows the model to dynamically balance global planning trends with local interaction responses.
Unified Implementation Across Paradigms:
INTraJ is instantiated under two distinct prediction paradigms, differing only in how the pure intent and social carrier are constructed:
- Multi-Target Paradigm: Used with backbones like QCNet. The pure intent is extracted from the focal agent's history. The future social carrier is constructed from the explicit predicted futures of non-focal agents ( $P_{others}$ ), which are encoded and fused via cross-attention to update the planning seed.
- Single-Target Paradigm: Used with backbones like DeMo. Since explicit futures of non-focal agents are not directly available, the framework constructs the pure intent via a "mask others" branch (ego-only) and the social carrier via a "static self" branch. In the static self branch, the ego's motion is set to static, allowing the model to encode the latent social response of surrounding agents under a counterfactual condition. This latent response is then injected into the planning representation.
Training Objectives:
The framework is trained with a joint loss function comprising:
- Prediction Loss ( $L_{pred}$ ): Standard loss for the final output trajectory.
- Planning Reference Loss ( $L_{ref}$ ): Supervises the $T_{ideal}$ branch using a smoothed version of the ground-truth trajectory to enforce smooth, low-frequency motion plans.
- Auxiliary Regularization ( $L_{aux}$ ): A variance-based regularizer applied to the gating weight $\alpha$ to prevent the model from collapsing to a uniform high value (effectively ignoring the decomposition) during early training.

Key Contributions

Hierarchical Social Modeling: The paper introduces a view that decomposes social influence into a planning stage (socially shaped global path) and a reaction stage (local adjustments), formulating trajectory formation as a sequential process.
Unified Framework: INTraJ provides a single formulation that supports both multi-target and single-target prediction paradigms, adapting the construction of social carriers to the specific backbone architecture.
Generalizability: The method demonstrates consistent performance gains across autonomous driving (Argoverse 2) and pure crowd forecasting (ETH/UCY, SDD) scenarios, indicating that hierarchical social interaction modeling generalizes across different interaction environments.

Experimental Results
Extensive experiments were conducted on four benchmarks: Argoverse 2 (AV2), Argoverse 2-ped, ETH/UCY, and SDD.

Crowd Benchmarks: On ETH/UCY and SDD, INTraJ integrated with the Resonance backbone achieved state-of-the-art (SOTA) performance. Notably, it reduced the Final Displacement Error (FDE) on SDD to 9.87 (from >10 in existing methods) and improved FDE by 7.2% overall on ETH/UCY, with a 12.2% gain on long-horizon subsets.
Autonomous Driving: On AV2, INTraJ improved both QCNet (multi-target) and DeMo (single-target) backbones. For instance, on DeMo, it achieved a minFDE6 of 1.17 and minADE6 of 0.61.
Ablation Studies: Results confirmed that both the planning and reaction stages are necessary; removing either stage or the gating mechanism degraded performance. The gains were attributed to the structural decomposition rather than parameter stacking.
Robustness: The framework demonstrated tolerance to imperfect predictions of surrounding agents, maintaining performance even when Gaussian noise was injected into the future social carrier.
Efficiency: Compared to the baseline QCNet, INTraJ introduced moderate computational overhead (parameters increased from 7.66M to 8.46M; latency from 37.94ms to 44.64ms) while outperforming other plug-in refinement methods like SmartRefine.

Significance and Claims
The paper claims that INTraJ reframes trajectory prediction from a monolithic regression task to a planning-driven two-stage process. By explicitly separating the construction of a socially shaped ideal trajectory from the recovery of local reactive adjustments, the framework addresses the limitations of coupled formulations that entangle social effects. The authors posit that this staged social modeling is critical for producing stable, coherent, and long-horizon consistent predictions. The results validate that decomposing social influence into planning and reaction provides an effective and generalizable paradigm for trajectory prediction across diverse multi-agent scenarios.

More like this