Technical Summary: ACME Dataset for Social Navigation

Problem Statement

Social robot navigation requires autonomous agents to move efficiently and naturally within dynamic human environments while adhering to implicit, context-dependent cultural norms. Unlike autonomous driving, which benefits from structured rules (e.g., lane markings, traffic lights), mobile service robots must navigate unstructured spaces where social norms vary significantly across cultures, geographical locations, and robot embodiments.

Existing datasets for social navigation and pedestrian trajectory prediction suffer from critical limitations:

• Lack of Diversity: Most datasets are collected in single locations or countries, failing to capture the variance in pedestrian behavior driven by cultural context (e.g., preferred walking sides, personal space).
• Limited Embodiment: Prior works often focus on a single robot form factor, ignoring how robot morphology influences human interaction and navigation expectations.
• Insufficient Complexity: Many datasets lack challenging "in-the-wild" scenarios, such as high-density crowds, blind corners, and narrow corridors, or rely on simplified human models that do not generalize to real-world social compliance.
• Data Gaps: There is a scarcity of datasets that combine goal-driven robot navigation with explicit human-robot interaction (e.g., speech) and verified, metric-grounded pedestrian trajectories across diverse cultural settings.

Methodology

The authors introduce ACME (A Cross-cultural, Multi-Embodiment dataset for social navigation), a large-scale data collection effort spanning 8 sites in 5 countries (Singapore, Japan, USA, Spain, Germany) using 7 distinct robot embodiments.

Data Collection Strategy

• Robot Platforms: The dataset utilizes 5 visually distinct robot types: 2 quadrupeds (Unitree GO1/GO2), 2 suitcase-style differential drive robots (human-operated), 1 rover-style robot, and 2 mobile service robots (e.g., Stretch 2, Shadow Robot).
• Teleoperation Guidelines: To ensure natural interactions, teleoperators were instructed to stay as far from the robot as safely possible (Wizard of Oz setup) to avoid influencing pedestrian behavior. They were also guided to fix navigation goals apriori and mark trajectory segments (Begin/End/Discard).
• Robot Speech: A subset of the data (NUS, UEx, Miraikan, Keio) includes explicit robot speech interactions. Teleoperators triggered specific utterances ("Excuse Me," "Attention, Robot Here," "Please Give Way") via an interface to study crowd interaction policies.
• Sensors: All robots were equipped with onboard sensors including RGB cameras, LiDAR, and odometry. A subset of locations also featured overhead Bird's-Eye View (BEV) cameras for pedestrian tracking.

Data Processing and Annotation

• Anonymization: To comply with Institutional Review Board (IRB) requirements across different countries, data underwent varying levels of anonymization (from face blurring to full-body segmentation). To compensate for information loss, the authors provide bounding boxes, segmentation masks, and pose keypoints for anonymized pedestrians.
• Trajectory Verification: Pedestrian trajectories from BEV cameras were initially tracked using automated tools (ByteTrack) and then human-verified at 10Hz. The authors developed a custom annotation tool to correct tracking errors (e.g., ID switches, missing detections, trajectory breaks).
• Coordinate Transformation: 2D pixel coordinates from BEV cameras were transformed into metric ground-plane coordinates using homography estimation derived from AprilTags or stationary keypoints.
• Scenario Tagging: Robot trajectories were manually tagged with scenario labels (e.g., narrow corridors, intersections, blind corners, passing conversational groups) to facilitate scenario-specific filtering.

Key Contributions

1. Scale and Diversity: ACME is the largest (by duration) and most diverse social navigation dataset to date, featuring 29.35 hours of onboard robot data and 43.5 hours of overhead pedestrian tracking data across 5 countries and 7 robot embodiments.
2. Human-Labeled BEV Trajectories: It provides the largest human-labeled Bird's-Eye View pedestrian trajectory prediction dataset with metric coordinate transformations, addressing the lack of verified, metric-grounded data in prior works.
3. Cross-Cultural and Multi-Embodiment Analysis: The dataset explicitly captures variations in navigation behavior across different cultures and robot morphologies, enabling the study of context-aware social norms.
4. Robot Speech Integration: It includes a unique subset of data capturing explicit robot-crowd interactions through speech, allowing for the study of verbal communication policies in navigation.
5. Comprehensive Benchmarks: The authors provide extensive benchmarks of State-of-the-Art (SoTA) vision navigation models (ViNT, NoMAD) and trajectory prediction models (SocialGAN, AgentFormer, MoFlow, etc.) on the dataset, demonstrating the challenges posed by ACME compared to existing benchmarks.

Results and Analysis

Dataset Characteristics

• Scenario Complexity: Compared to prior datasets (SCAND, MuSoHu), ACME captures a broader distribution of challenging scenarios. It features higher pedestrian density near the robot and more constrained traversability (narrow corridors, blind corners).
• Planner Failure Rates: When benchmarked against a geometric planner (TEB), ACME scenes resulted in 2x more planner failures per minute and 4x more failures per 5 meters compared to SCAND, indicating that ACME captures scenarios where standard geometric planners struggle significantly.
• Navigation Model Performance: SoTA foundational navigation models (ViNT, NoMAD) trained on existing datasets (SCAND) showed degraded performance on ACME, with higher Final Destination Errors (FDE) and orientation errors, highlighting the generalization gap.
• Trajectory Prediction: SoTA trajectory prediction models performed worse on ACME than on the ETH/UCY datasets. When filtering for dynamic pedestrians only, the performance drop was even more significant (ADE/FDE drops of ~0.38m/0.72m), suggesting ACME captures complex human motion dynamics not present in smaller, older datasets.

Behavioral Insights

• Cultural Differences: Analysis of passing side preferences revealed that pedestrians in the US and Germany predominantly pass on the right, while those in Japan and Singapore showed a tendency to pass on the left, though this was less pronounced in narrow corridors.
• Speed and Density: Walking speeds varied significantly by location; outdoor locations (Germany, Singapore) showed higher speeds, while museum environments (Miraikan) and high-density areas showed slower speeds.
• Personal Space: Lateral personal space was found to be narrower in corridor environments compared to open spaces, suggesting environmental constraints override cultural norms in tight spaces.

Significance and Claims

The paper claims that ACME addresses a fundamental gap in social robotics research by providing a data-driven foundation for learning navigation policies that are robust to cultural and embodiment variations. The authors assert that:

• ACME is the first dataset to systematically capture social navigation across multiple cultural contexts and robot embodiments.
• The dataset's diversity and complexity make it a critical benchmark for evaluating the generalization capabilities of current navigation and trajectory prediction models.
• The inclusion of verified metric trajectories and scenario tags enables researchers to develop and test context-aware and embodiment-aware social navigation policies, moving beyond the limitations of single-location or single-robot studies.
• The dataset facilitates the study of robot speech as a tool for proactive crowd management, a feature largely absent in previous datasets.

The authors conclude that ACME will serve as a resource for advancing social navigation research, particularly in developing models that can adapt to the nuanced, implicit, and culturally dependent norms of human-shared spaces.