Technical Summary: Robot Squid Game (SQUID)

Problem Statement

Quadruped robots have demonstrated significant potential in unstructured outdoor terrains, yet autonomous traversal of confined 3D environments—such as tunnels, caves, and collapsed structures—remains a critical capability gap. Existing navigation strategies face three primary limitations:

1. Oversimplified Training: Many methods rely on static or simplified environmental models that fail to capture the geometric diversity of real-world confined spaces.
2. Over-Specialization: Highly specialized policies often require substantial retuning when encountering new tunnel shapes or geometries.
3. Sensor Sensitivity: Approaches are often sensitive to sensory noise, leading to performance degradation during real-world deployment.

Current methods struggle to coordinate body posture, limb articulation, and environmental awareness simultaneously to avoid ceiling collisions and lateral obstructions in 360° constrained spaces.

Methodology: SQUID Framework

The paper introduces SQUID (Skill-fused Quadrupedal locomotion Using Imitation and Distillation), a Reinforcement Learning (RL) framework that combines procedural environment generation with a teacher-student policy distillation paradigm. The system operates in three stages:

1. Procedural Tunnel Generation

To address the lack of geometric diversity in training, the authors developed a physics-aware procedural generator that creates 3D tunnel environments with parameterized variations. The system defines four distinct tunnel classes, each with tunable difficulty parameters:

• Triangle (△): Equilateral cross-sections with sharp angles requiring body rotation.
• Circle (⃝): Uniformly enclosed structures where radius reduction increases difficulty.
• Half-Circle (T): Semicircular arcs with randomized Z-normal flips to simulate inverted terrain.
• Gap (⊢⊣): Central voids with elevated side shelves, challenging stable foothold selection.

These environments are sequentially connected with increasing difficulty, ensuring the robot encounters a broad distribution of spatial constraints rather than memorizing fixed paths.

2. Privileged Teacher Training

Four specialized "expert" teacher policies are trained independently using Proximal Policy Optimization (PPO) within the RL framework.

• Privileged Information: Teachers utilize full state observability, including ground-truth environmental details (e.g., exact terrain heightmaps, ceiling constraints) unavailable to the final deployed robot.
• Observation Space: Includes proprioceptive data (joint positions/velocities, base linear/angular velocities) and exteroceptive terrain measurements.
• Reward Function: A weighted sum of terms encouraging velocity tracking, stability (penalizing vertical/horizontal fluctuations), smooth actuation, collision avoidance, and structured step timing.

3. Policy Distillation to Student

A unified "student" policy is trained via Dataset Aggregation (DAgger) to distill knowledge from the four expert teachers.

• Input Constraints: Unlike the teachers, the student relies solely on onboard sensing: depth images (64×48 pixels) and historical proprioception.
• Tunnel-Conditioned Distillation: During training, the student is supervised by the specific expert policy matching the current tunnel class label.
• Class-Balanced Aggregation: A replay buffer ensures balanced sampling across all four tunnel types, preventing bias toward any single geometry and preserving specialized behaviors within a single model.

Key Contributions

The paper outlines three primary contributions:

1. Tunnel Simulation Pipeline: A procedural generation system creating 3D tunnel environments with parameterized geometric variations (cross-section asymmetry, slope transitions) that exceed the diversity of existing confined 3D training datasets.
2. Tunnel-Conditioned Policy Distillation: A multi-expert teacher-student scheme that uses class-balanced aggregation to retain locomotion behaviors across heterogeneous tunnel geometries within a single deployable policy.
3. Real-World Depth-Based Deployment: Experimental validation demonstrating that the distilled policy successfully transfers to real hardware using only depth observations and proprioceptive inputs, without privileged simulator information.

Experimental Results

The authors evaluated SQUID through simulation and real-world deployment on a Unitree Go2 quadruped.

Simulation Performance

Compared to a Vanilla RL (V-RL) baseline and a Hierarchical Planner (HP), SQUID demonstrated superior performance:

• Success Rate: SQUID achieved the highest success rates across all tunnel classes and difficulty levels, particularly in challenging triangular and gap tunnels where V-RL failed frequently and HP struggled with adaptive motion.
• Efficiency: SQUID maintained faster traversal times than the overly cautious HP and exhibited lower collision frequencies than the erratic V-RL.
• Energy: The method achieved better energy efficiency by ensuring smoother gait transitions and reducing unnecessary corrective movements.

Ablation Studies

The authors conducted ablation studies to verify the necessity of each component:

• Multi-Expert vs. Single Teacher: A student trained with only one teacher (circular tunnels) failed to generalize to asymmetric tunnels (triangular/gap), confirming the need for multi-expert knowledge.
• Procedural Generation: Training in fixed, non-procedural environments resulted in rigid policies that failed under unexpected variations.
• Elevation Mapping: Using a two-layer elevation map (floor + ceiling) for teachers caused convergence issues; ground-only elevation maps proved more effective for learning stable strategies.
• Reward Shaping: Removing collision or vertical velocity penalties led to unstable, risky behaviors.

Real-World Deployment

In a controlled tunnel testbed with modular segments (circular, triangular with 40° slope, and gap tunnels), the SQUID policy achieved:

• 70% success rate in circular tunnels.
• 80% success rate in gap tunnels.
• 60% success rate in the most challenging sloped triangular tunnels.
  These results confirm the policy's ability to transfer to real hardware despite depth noise and missing returns.

Significance and Claims

The paper claims that SQUID effectively bridges the gap between structured simulation training and unpredictable, confined 3D spaces. By decoupling perception-handling strategies from core locomotion and leveraging a teacher-student architecture, the method eliminates the need for complex reward shaping in end-to-end training and reduces the reliance on policy retuning for new geometries.

The authors note that their evaluation focuses on structured tunnel-like geometries and does not explicitly model unstructured clutter (e.g., debris, rocks) or tight-turn junction networks. They identify extending the procedural generator to include such unstructured confined passages as a direction for future work. The work demonstrates that robust quadrupedal locomotion in confined environments is achievable through the synthesis of diverse training data and the distillation of specialized expert skills into a generalizable, vision-based policy.