SENSE-STEP: Learning Sim-to-Real Locomotion for a Sensory-Enabled Soft Quadruped Robot

1. Problem Statement

Robust closed-loop locomotion for soft quadruped robots remains a significant challenge due to:

• High-dimensional dynamics: Soft structures deform in complex, non-linear ways that are difficult to model analytically.
• Actuator limitations: Pneumatic actuators exhibit strong hysteresis and non-linearities.
• Sensing gaps: Conventional proprioception (e.g., joint encoders) is insufficient for soft robots because rigid kinematic models cannot accurately represent large deformations.
• Contact uncertainty: Interactions with the ground (especially on inclines or uneven terrain) are difficult to predict without rich tactile feedback.

Existing solutions are often limited to simulation or rely on hardware with lower morphological complexity. There is a lack of experimental demonstrations combining pneumatic actuation, high-dimensional deformation, and active tactile feedback for closed-loop control.

2. Methodology

A. Hardware Platform: TActile Soft Quadruped (TASQ)

The authors developed a novel robot platform designed for rich tactile interaction:

• Actuation: Four legs driven by Pneumatic Single-Structure Omnidirectional Actuators (PSSOA). These consist of a single elastomeric body with three internal chambers, enabling omnidirectional bending and axial extension via differential pressurization.
• Tactile Suction Feet: Each leg terminates in a custom suction cup that serves two functions:
  1. Active Anchoring: A controllable vacuum generator allows feet to stick to the ground, enabling stable locomotion on inclines.
  2. Force Sensing: The cups embed neodymium magnets and Hall-effect sensors. As the cup deforms under load, the magnetic field changes. A learned Multilayer Perceptron (MLP) converts these magnetic readings into Normal Ground Reaction Force (GRF) estimates.
• State Estimation: The robot uses an onboard Inertial Measurement Unit (IMU) for body orientation/velocity and the tactile sensors for contact forces. This enables blind locomotion (no external cameras or motion capture).

B. Control Framework: Staged Learning

The authors propose a three-stage learning pipeline to bridge the Sim-to-Real gap:

1. Imitation Learning (Warm-Start):
   • A reference gait (inspired by slow, stable elephant walking) was executed on the real robot.
   • A Multilayer Perceptron (MLP) was trained in simulation via Behavior Cloning (BC) to imitate this gait. This provides a stable initial policy, avoiding the instability of training from scratch.
2. Reinforcement Learning (Fine-Tuning):
   • The policy is further refined using Soft Actor-Critic (SAC) in simulation.
   • Domain Randomization: Key physical parameters (mass, stiffness, damping, hysteresis, suction force) and sensor noise are randomized during training to ensure robustness.
   • Reward Function: Balances forward progress, body orientation stability (pitch/roll/yaw), height maintenance, and actuation smoothness.
3. Deployment:
   • The trained policy maps onboard observations (IMU, tactile forces, pressure states, CPG signals) to continuous pressure increments and suction commands.
   • The action space is 16-dimensional (12 pressure increments + 4 suction gradients).

3. Key Contributions

• Novel Sensing Integration: Development of a compact, deformable tactile suction cup that simultaneously anchors the robot and estimates ground reaction forces using magnetic field variations, avoiding bulky load cells.
• Sim-to-Real Transfer for Soft Robots: A successful demonstration of simulation-trained, closed-loop locomotion on a physical, high-dimensional soft quadruped without external sensing.
• Staged Learning Strategy: A hybrid approach combining Behavior Cloning (to establish a stable gait prior) and Reinforcement Learning (to optimize for robustness and adaptability) specifically tailored for the challenges of soft robotics.
• Active Anchoring Control: The ability to coordinate leg deformation with active suction anchoring to maintain stability on inclined surfaces.

4. Results

Simulation Performance

• The RL-trained policy improved locomotion speed by approximately 175% over the behavior-cloned reference gait on flat terrain.
• The policy trained on a 2.5° incline generalized effectively to flat terrain, demonstrating robustness across different slopes.

Experimental Validation (Physical Robot)

• Flat Terrain: The closed-loop policy increased forward speed by 41% compared to an open-loop baseline, while significantly improving body stability and step consistency.
• Inclined Terrain (5°): The closed-loop policy increased speed by 91% compared to the open-loop baseline (which struggled significantly). The robot actively adjusted its body posture to remain horizontal.
• Ablation Studies:
  • Replacing sensor data with noise drastically reduced performance, confirming the necessity of tactile and inertial feedback.
  • Enabling closed-loop control (with sensors) improved speed by 17% on flat terrain and 56% on inclines compared to configurations without sensory feedback.
  • Disabling suction entirely prevented forward motion, proving that active anchoring is physically necessary for locomotion on these surfaces, but sensors are critical for coordinating the timing of that anchoring.

5. Significance

This work represents a major step forward in soft robotics by proving that learning-based control can overcome the modeling difficulties of soft materials.

• Overcoming Hysteresis: By relying on feedback rather than precise open-loop models, the system compensates for the inherent hysteresis of pneumatic actuators.
• Safe Interaction: The combination of soft compliance and active suction anchoring allows for safe, stable mobility in contact-rich environments where rigid robots might slip or fall.
• Blind Locomotion: The system demonstrates that complex, adaptive locomotion is possible using only proprioceptive and tactile feedback, removing the dependency on vision or motion capture systems.
• Foundation for Future Work: The framework provides a testbed for tackling more complex challenges like uneven terrain, partial foot failure, and dynamic disturbances.

In conclusion, R SENSE-STEP validates that integrating tactile sensing with staged imitation and reinforcement learning enables soft quadrupeds to achieve robust, high-performance closed-loop locomotion in the real world.