Technical Summary: HiCrowd - Hierarchical Crowd Flow Alignment for Dense Human Environments

Problem Statement

Mobile robots face significant challenges navigating dense human environments, primarily due to the Freezing Robot Problem (FRP). In this scenario, a robot struggles to find safe motions amidst complex human interactions and becomes stuck, unable to reach its goal. Traditional reactive planners (e.g., Social Force, ORCA) often fail to capture intricate human-human interactions in dense settings, leading to FRP. Conversely, recent Reinforcement Learning (RL) approaches, while learning socially compliant behaviors, often struggle to map complex crowd states directly to low-level control actions while guaranteeing safety. The core challenge is balancing navigation efficiency, safety, and social compliance without causing the robot to freeze or collide.

Methodology: HiCrowd Framework

The authors propose HiCrowd, a hierarchical framework that integrates Reinforcement Learning (RL) with Model Predictive Control (MPC) to enable efficient and safe navigation. The core hypothesis is that in dense environments, robots should leverage human motion as guidance (aligning with compatible flows) rather than treating humans solely as dynamic obstacles to be avoided.

1. Hierarchical Architecture

The system separates long-term strategic decision-making from short-term motion control:

• High-Level (RL Policy): The RL agent acts as a strategic planner. Instead of outputting direct control commands, it predicts a follow point $(f_x, f_y)$ in the robot's local frame. This point guides the robot to align with a suitable pedestrian group (local crowd flow).
• Low-Level (MPC Controller): The MPC module takes the RL-generated follow point as a reference. It optimizes a short-horizon trajectory to track this guidance while ensuring dynamic feasibility and safety (collision avoidance) with respect to nearby humans.

2. Reinforcement Learning Module

The RL policy is trained using the Soft Actor-Critic (SAC) algorithm. The observation space includes the robot's state, goal, and the states (position, velocity) of $n$ nearby humans within a sensing radius.

• Reward Function: The reward $r_t$ is a weighted sum of three components:
  1. Goal Reaching ($r_{goal}$): Sparse reward for reaching the destination.
  2. Goal Proximity ($r_{dist}$): Dense reward encouraging movement toward the goal.
  3. Crowd Following ($r_{follow}$): A novel reward term that encourages alignment with crowd flows.
     • Humans are clustered into groups using DBSCAN based on spatial proximity and motion similarity.
     • The reward is calculated based on the maximum score across all groups, measuring the match between the robot's motion and the group's average speed and heading.
     • This term explicitly incentivizes the robot to join compatible flows, accelerating learning and improving performance.

3. Model Predictive Control Module

The MPC minimizes a cost function $J = J_{follow} + J_{safe}$ over a planning horizon $H$:

• Follow Cost ($J_{follow}$): Minimizes the distance between the predicted trajectory and the RL-provided follow point.
• Safety Cost ($J_{safe}$): Penalizes proximity to humans predicted via a Constant Velocity Model (CVM), using an exponential cost function to enforce a safety radius.

Key Contributions

1. Principle of Crowd Flow Guidance: The paper demonstrates that leveraging human motion as guidance, rather than solely as obstacles, provides a powerful principle for safe and efficient navigation.
2. Hierarchical RL-MPC Framework: The authors introduce a specific architecture where a high-level RL agent, trained with a crowd-following reward, generates socially aware follow points, while a low-level MPC ensures safe execution.
3. Real-World Validation: The method is shown to be effective in real-world deployments without retraining, bridging the sim-to-real gap.

Experimental Results

The authors evaluated HiCrowd against baselines including ORCA (reactive), SARL (flat RL), CrowdAttn (attention-based RL), and an MPC-only ablation. Evaluations were conducted on the ETH-UCY dataset (real-world recordings) and a Synthetic dataset (denser, opposing flows) in both offline (humans do not react) and online (humans react via ORCA) settings.

• Performance Metrics: HiCrowd achieved the highest Success Rate (SR) (100% in online ETH-UCY and Synthetic settings) and the lowest Navigation Time (NT) and Path Length (PL).
• Freezing Robot Problem: HiCrowd significantly reduced the Freezing Frequency (FF), achieving near-zero freezing in most settings, whereas baselines frequently got stuck.
• Safety: While HiCrowd did not always maximize the minimum pedestrian distance compared to some baselines, it consistently maintained a social distance above 1.2 meters.
• Ablation Study: Removing the crowd-following reward ($\lambda_f = 0$) resulted in slower learning convergence and lower performance, confirming that explicitly rewarding flow alignment is critical.
• Real-World Deployment: The robot was deployed in a public museum and at Expo 2025 Osaka. Without retraining, it successfully navigated dense pedestrian flows, merging with groups, navigating against flows, and crossing flows without collisions or freezing.

Significance and Claims

The paper claims that HiCrowd offers a robust solution to the freezing robot problem by shifting the paradigm from "avoiding obstacles" to "aligning with flows." The authors argue that this hierarchical approach effectively decouples long-term social reasoning from short-term safety constraints. The results suggest that treating humans as dynamic guides rather than static barriers allows robots to maintain consistent progress in dense crowds. The successful real-world deployment in unstructured, crowded environments without retraining highlights the method's potential for practical application in human-centric robotics.

Limitations noted by the authors:

• The fixed ratio between high-level and low-level actions may cause delayed reactions in rapidly changing conditions.
• The current framework does not explicitly model the spatial geometry of groups, relying primarily on motion features.
• Static obstacles (e.g., walls) were not the primary focus of the current evaluation.