SmoothTurn: Learning to Turn Smoothly for Agile Navigation with Quadrupedal Robots

1. Problem Statement

Quadrupedal robots are increasingly used in time-sensitive applications like fire rescue and industrial inspection, where agile navigation at high speeds is critical. While existing reinforcement learning (RL) methods have achieved robust straight-line locomotion and single-goal reaching, they struggle with sequential local navigation involving frequent direction changes.

Key Challenges Identified:

• Stop-and-Go Behavior: Existing single-goal policies are trained to reach a target and stay there to maximize reward. When deployed for sequential goals, this causes the robot to decelerate significantly or stop completely at each waypoint before turning, destroying momentum and increasing traversal time.
• Lack of Anticipation: Standard policies condition actions only on the current goal. They fail to anticipate upcoming turns, leading to unstable, aggressive maneuvers or falls when switching directions at high speeds.
• Brittle Goal Switching: Current methods rely on strict thresholds to switch goals. If thresholds are too loose, the robot switches while moving too fast and falls; if too strict, it stops unnecessarily.

2. Methodology: SmoothTurn Framework

The authors propose SmoothTurn, a learning-based control framework designed to enable smooth, high-speed turning across a sequence of goals. The framework consists of three core innovations:

A. Sequential Local Navigation Formulation

The task is redefined from reaching a single terminal goal to traversing an ordered sequence of goals $G = \{g_1, ..., g_N\}$.

• Dual Reaching Conditions: To handle the transition between goals robustly, the system uses two conditions:
  1. Direct Switch: Allows switching goals if the robot is close to the target and moving fast (preventing unnecessary stops).
  2. Stop Switch: A relaxed condition that only triggers if the robot is nearly stationary, acting as a safety fallback.

B. Novel Reward Function

The authors replace the standard single-goal reward (which incentivizes stopping) with a Sequential Goal-Reaching Reward ($r_t^{seq}$):

• Continuous Progress: The reward is a smooth function of the number of goals reached ($k_t$) and the distance to the current goal.
• No Discontinuities: Unlike previous methods where the reward drops sharply when switching goals, this reward increases smoothly as the robot approaches the next target, encouraging continuous motion without pausing.
• Dynamic Heading Penalty: The reward weights heading errors dynamically, prioritizing distance closure first and fine-tuning orientation only when close to the target.

C. Lookahead Observation & Curriculum

• Lookahead Window: The observation space is expanded to include a window of future goals ($g_{k+1}, ..., g_{k+n}$). This allows the policy to anticipate upcoming turns, enabling it to adjust momentum and orientation before reaching the current goal.
• Automatic Goal Curriculum: A curriculum learning mechanism progressively increases the difficulty of goal sequences (turning angles and distances) based on the robot's success rate. It starts with straight, short paths and evolves to complex zigzag patterns, ensuring stable training.

3. Key Contributions

1. Task Formulation: Defined the Sequential Local Navigation task, explicitly targeting the challenge of smooth direction changes during high-speed locomotion.
2. RL Framework Design: Introduced a novel reward structure, a lookahead observation space, and an automatic curriculum that collectively enable the robot to maintain momentum and execute smooth turns.
3. Emergent Behaviors: The system autonomously learned complex behaviors not explicitly programmed, such as:
   • Reorienting toward the next goal while still approaching the current one.
   • Cutting corners to take shorter paths when tolerances allow.
   • Managing momentum to avoid falls during sharp turns.
4. Real-World Deployment: Successfully deployed the policy on a Unitree Go2 quadrupedal robot with onboard sensors and computation, demonstrating stability in both simulation and the real world.

4. Experimental Results

The authors evaluated SmoothTurn against a baseline single-goal policy in simulation and on a physical robot.

• Simulation Performance:

  • Success Rate & Stability: SmoothTurn achieved near-perfect success rates (99%+) across various turning patterns (60°, 90°, 120°, 150° zigzags) and threshold settings. The baseline policy suffered high fall rates (up to 93%) under loose thresholds due to instability.
  • Traversal Time: SmoothTurn was significantly faster. For example, in a 90° turn sequence, it completed the task in 4.31s compared to the baseline's 5.69s (under strict thresholds) because it did not stop at intermediate goals.
  • Relaxed Conditions: When trained with relaxed goal-reaching conditions (ignoring heading error or increasing distance tolerance), the robot learned to take even more efficient paths, reducing traversal time further (e.g., down to 3.40s for a 60° turn).

• Ablation Studies:

  • Lookahead: Removing the lookahead window ($n=1$) caused performance to degrade significantly, confirming that anticipating future goals is essential for smooth turning.
  • Curriculum: Training without the curriculum (starting with hard tasks) resulted in training failure (near-zero success), proving the necessity of progressive difficulty scaling.

• Real-World Results:

  • Deployed on a Unitree Go2, SmoothTurn outperformed the baseline in all four real-world turning tasks, reducing mean traversal time by approximately 1.5 to 2.0 seconds per sequence.
  • The robot successfully maintained high speeds and executed smooth turns without falling, validating the sim-to-real transfer.

5. Significance

SmoothTurn addresses a critical bottleneck in legged robot deployment: maneuverability. While straight-line speed is often the focus, real-world navigation requires frequent turning. This work demonstrates that by shifting the reward structure from "stop-and-stay" to "continuous progress" and providing the agent with foresight (lookahead), robots can learn to orchestrate their momentum autonomously.

The ability to navigate sequences of goals smoothly without stopping makes quadrupedal robots viable for time-critical real-world applications (e.g., search and rescue, rapid logistics) where every second counts. The framework's success in learning emergent behaviors like "cutting corners" and "pre-turning" suggests a path toward more intelligent, adaptive, and efficient autonomous navigation systems.