💻 computer science

Trajectory Learning with Graph Representations for Social Robot Navigation

This paper proposes an imitation learning framework for socially compliant robot navigation that utilizes a graph-based auxiliary network to encode crowd interactions and a trajectory-level learning module to capture temporal dynamics, thereby outperforming existing data-driven baselines in both simulation and real-world scenarios.

Original authors: Berke Kartal, Burcu Kilic, Yigit Yildirim, Emre Ugur

Published 2026-07-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Berke Kartal, Burcu Kilic, Yigit Yildirim, Emre Ugur

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot trying to walk through a busy airport or a crowded warehouse. Its goal is to get from Point A to Point B without bumping into anyone or making people feel uncomfortable. This is called "social navigation."

The problem is that most robots are either too rigid (following strict rules that fail in chaos) or too clumsy (learning by trial and error, which leads to mistakes piling up).

This paper introduces a new way to teach robots how to walk politely among humans. Think of it as giving the robot two superpowers: a "social radar" and a "time machine."

1. The "Social Radar": Seeing the Crowd as a Web

Most robots look at people as a list of separate dots: "Person A is here, Person B is there." They miss the bigger picture. They don't realize that if Person A stops to talk to Person B, Person C might have to step around them.

The authors created a tool called the Graph Feature Autoencoder (GFAE).

The Analogy: Imagine the crowd isn't a list of people, but a giant spiderweb. Every person is a node on the web, and the invisible strings connecting them are their relationships (who is walking with whom, who is blocking whom).
How it works: This tool uses a special type of math (Graph Neural Networks) to "feel" the tension in the web. It pays extra attention to people who are close together or moving in groups. Instead of just seeing "10 people," the robot sees "three groups of friends and two people walking alone."
The Result: The robot gets a single, compact "summary" of the crowd's mood and structure, rather than a messy list of coordinates.

2. The "Time Machine": Predicting the Future

Older learning methods are like a driver who only looks at the road directly in front of their bumper. If they see a gap, they drive into it. But if a pedestrian is 10 feet away and walking toward that gap, the robot doesn't realize it until it's too late. This leads to "error accumulation"—one small mistake makes the next one worse, and the robot gets stuck or crashes.

The authors' second tool is a Navigation Module that learns from human demonstrations.

The Analogy: Instead of just copying a human's step-by-step movements, this robot watches a video of a human walking and learns the entire story of the walk at once. It learns that "when I see that group of people, I should start curving left now, even though they are far away."
How it works: The robot doesn't just guess where it will be next; it guesses where the people will be next, too. It predicts the future positions of the crowd and its own path simultaneously.
The Result: The robot can make "preemptive" moves. It steps aside before a collision is even possible, just like a skilled human dancer who anticipates their partner's next move.

How They Tested It

The team tested this in two ways:

In a Video Game (Simulation): They built a virtual warehouse with 160 different scenarios (people crossing, groups chatting, people walking in different directions).
In the Real World: They used a real dataset recorded outdoors with actual people and a robot.

The Results

When they compared their new robot to older methods:

Old Robots (Behavior Cloning): Often bumped into people, got stuck, or took a long time to navigate because they reacted too slowly.
The New Robot: It was much smoother. It bumped into people far less often, reached its destination more successfully, and moved in a way that felt natural and polite.

The Bottom Line

The paper claims that by teaching a robot to understand the social structure of a crowd (using the "spiderweb" radar) and to predict the future of both itself and the people around it (using the "time machine" learning), it can navigate human spaces much more safely and politely than current methods.

The authors note that while this works great for social maneuvering, they didn't test it with heavy obstacles like walls or furniture, and they didn't include a "hard stop" safety brake in this specific experiment. But for the specific job of walking politely among people, their method is a significant upgrade.

Technical Summary: Trajectory Learning with Graph Representations for Social Robot Navigation

Problem Statement

Autonomous mobile robots operating in human-populated environments must navigate while minimizing disturbance to pedestrians. While capturing social interactions and incorporating pedestrian motion estimations are crucial for socially compliant behavior, existing methods face significant limitations:

Reinforcement Learning (RL): Requires hand-crafted reward functions that often reduce complex social behaviors to static criteria, limiting the ability to reproduce real-world pedestrian patterns.
Imitation Learning (Behavior Cloning - BC): Offers direct training from real-world data but suffers from error accumulation due to single-step decision-making. Small distribution shifts between training and testing lead to compounded errors.
State Representation: Many approaches rely on fixed numbers of pedestrians or independent pairwise interactions, failing to model the relational structure and dynamic social interactions within a crowd.
Foresight: Navigating based solely on current or past observations limits the robot's ability to anticipate pedestrian intentions, leading to reactive rather than preemptive maneuvers.

Methodology

The authors propose a two-stage imitation learning framework that decouples crowd representation learning from navigation policy learning.

1. Graph Feature Autoencoder (GFAE)

To address the limitation of modeling social structure, the authors introduce a novel auxiliary network, the Graph Feature Autoencoder (GFAE).

Architecture: It utilizes a Graph Transformer Layer (GTL) as an encoder. This layer employs a multi-head attention mechanism to dynamically weight interactions between agents (pedestrians and the robot) based on their proximity and influence, rather than treating all interactions equally.
Function: The GFAE processes a graph-based state where nodes represent agents and edges represent interactions. It aggregates node embeddings via max pooling to produce a fixed-size crowd embedding ( $z$ ).
Training Objective: To ensure the pooled embedding preserves per-node information, the network includes a decoder that reconstructs individual node features from the global embedding. The model is trained to minimize the Mean Squared Error (MSE) between original and reconstructed node features.
Outcome: This produces transferable crowd embeddings that capture human-human and human-robot interactions independent of the downstream navigation task.

2. Conditional Robot Trajectory Generation (Navigation Module)

The navigation module builds upon the Conditional Neural Movement Primitives (CNMP) framework to address error accumulation and lack of foresight.

Input: The module takes a shared latent representation ( $r$ ) derived from sampled observations, which include time, robot position, goal, and the GFAE-generated graph embeddings.
Dual-Head Prediction: Unlike standard BC, this module employs two decoder heads operating from the same latent vector:
1. Trajectory Head: Predicts the robot's future position.
2. State Head: Predicts future crowd embeddings (pedestrian movement estimations).
Training Objective: The network is trained with a trajectory-level learning objective. It samples observation and target timesteps uniformly from demonstrations and minimizes a linear combination of:
- Trajectory loss (Gaussian negative log-likelihood).
- State prediction loss (MSE between predicted and actual future crowd embeddings).
Mechanism: By jointly predicting the robot's trajectory and future crowd states, the shared encoder is regularized to encode information about both the robot's path and pedestrian dynamics. This allows the robot to incorporate anticipated pedestrian movements into its decision-making, mitigating the error accumulation typical of single-step BC.

Key Contributions

Spatiotemporal Framework: Integration of learned spatial crowd representations (via GNNs) with temporal trajectory learning.
Graph Feature Autoencoder (GFAE): A novel method to encode the social structure of a crowd into a fixed-size embedding by leveraging attention mechanisms to model human-human and human-robot interactions.
Trajectory-Level Navigation Module: A navigation module trained with a trajectory-level objective that mitigates error accumulation. It jointly predicts pedestrian movement estimations and the robot trajectory from a shared latent representation, enabling preemptive maneuvers.

Experimental Results

The framework was evaluated on both a simulated warehouse environment (SEAN 2.0) and a real-world dataset (SCAND).

Simulation Results (SEAN 2.0)

Metrics: Evaluated on Intimate Zone Intrusion Count (IZIC), Personal Zone Intrusion Count (PZIC), Minimum Distance to Pedestrians (MDTP), Collisions, Navigation Time, and Success Rate.
Performance: The proposed method (GE+CNMP) outperformed Behavior Cloning (BC), Diffusion Policy, and a baseline using Graph Embeddings with BC (GE+BC).
- It achieved the highest success rate (98.12%) and the lowest intrusion counts.
- It significantly outperformed the Diffusion Policy, which the authors attribute to the lack of a trajectory-level objective in the diffusion approach and the smaller dataset size.
- Ablation Study: Confirmed that using learned graph embeddings (GE+BC) significantly improves performance over raw state representations (BC). The TransformerConv layer with max pooling yielded the best reconstruction loss in the GFAE.
Visualization: t-SNE visualizations showed that the GFAE latent space successfully clustered states by social scenario, whereas raw state space did not.

Real-World Results (SCAND Dataset)

Setup: Tested on noisy, real-world LiDAR data from outdoor environments.
Performance: GE+CNMP significantly outperformed BC, achieving a success rate of 98.33% compared to 53.33% for BC.
Observation: BC frequently failed by overshooting goals or colliding with pedestrians, while the proposed method maintained social compliance and reached goals effectively.

Significance and Claims

The paper claims that its framework successfully addresses three primary limitations of prior social navigation methods:

Error Accumulation: By utilizing a trajectory-level learning objective rather than single-step imitation, the method reduces the compounding of errors during execution.
Anticipated Movements: By jointly predicting future crowd states alongside the robot trajectory, the system enables flexible, preemptive maneuvers rather than reactive ones.
Social Structure Modeling: The GFAE captures the relational structure of the crowd (human-human interactions) rather than treating pedestrians as independent entities or a fixed set.

The authors conclude that their method exhibits socially compliant maneuvers in both simulated and real-world settings, demonstrating that learning temporal relations between sensorimotor data and crowd embeddings yields superior social performance compared to common reactive methods.

Limitations

The authors acknowledge specific limitations:

Scalability: The pooling operation used to create graph embeddings may face scalability challenges in highly crowded settings.
Collision Avoidance: The framework does not integrate explicit collision avoidance guarantees; experiments were conducted in environments without static obstacles to focus on social maneuvers. Future work plans to integrate this framework into a navigation stack where low-level controllers handle collision avoidance.