Rapid co-design of Buoyancy-assisted robots for Challenging Locomotion using Gaussian Evolutionary Specialists

Technical Summary: Rapid Co-design of Buoyancy-assisted Robots using Gaussian Evolutionary Specialists

Problem Statement

The paper addresses the challenge of robot co-design, which involves the joint optimization of robot morphology (physical structure) and control policies to achieve task performance unattainable by either component in isolation. Specifically, the authors focus on the Buoyancy-Assisted Light Legged Unit (BALLU), a bipedal robot that uses helium balloons to offset weight.

The core difficulty lies in the computational cost of the co-design process. It is a bi-level optimization problem:

1. Outer Loop: Searches the morphology design space.
2. Inner Loop: For every candidate morphology, a robust controller must be trained (typically via Reinforcement Learning, RL).

Existing approaches face significant bottlenecks:

• Repeated RL Training: Training a new policy from scratch for every morphology candidate is computationally prohibitive.
• Universal Policies: Pre-training a single "universal" policy conditioned on morphology parameters to allow zero-shot evaluation fails due to behavioral diversity collapse. The policy converges to a single, suboptimal locomotion strategy because conflicting gradients from diverse morphologies average out during training.
• End-to-End Mixture-of-Experts (MoE): While MoE architectures theoretically allow for specialized experts, they suffer from representation collapse in this context. The router fails to differentiate between experts early in training, causing all experts to converge to similar behaviors, rendering the mixture ineffective.

Methodology: Gaussian Evolutionary Specialists (GES)

The authors propose Gaussian Evolutionary Specialists (GES), a framework that decouples the partitioning of the design space from the learning of control policies to explicitly capture diverse behaviors without the failure modes of universal or end-to-end MoE policies.

Core Mechanism:
GES maintains a set of $K$ specialist policies, each associated with a Gaussian territory (defined by a mean $\mu_k$ and diagonal covariance $\Sigma_k$) over the design parameter space. The framework operates in an iterative loop consisting of three phases until the design space is sufficiently covered (95% Monte Carlo coverage):

1. Initialization:

   • $K$ Gaussian centers are initialized in the normalized design space using Farthest Point Sampling (FPS) followed by Lloyd's Relaxation to ensure well-dispersed, space-filling coverage.
   • Each specialist is initialized as a random MLP.

2. Iterative Territory Evolution:

   • Train: Each specialist $\pi_k$ samples designs from its current Gaussian territory and trains via Proximal Policy Optimization (PPO), resuming from previous checkpoints to retain learned behaviors.
   • Probe: Candidate designs are sampled near the boundaries ($2\sigma$) of all Gaussian territories. These candidates are evaluated by all $K$ specialists. Each candidate is competitively assigned to the specialist that achieves the highest task performance score ($J$).
   • Refit: The Gaussian parameters ($\mu_k, \Sigma_k$) for each specialist are updated to reflect the mean and covariance of the designs it "won" in the probe phase. This allows territories to expand and adapt based on empirical performance.

3. Co-Design Integration:

   • Once training is complete, the specialists serve as zero-shot evaluators for the outer design optimization loop (e.g., Bayesian Optimization).
   • For a new candidate design, a geometric router selects the nearest specialist based on Gaussian likelihood, and the specialist's performance is evaluated directly, eliminating the need for re-training.

Key Contributions

1. Identification of Failure Modes: The paper identifies behavioral diversity collapse as a previously unrecognized failure mode of universal policies in robot co-design, where policies converge to a single strategy regardless of morphology. It also highlights how representation collapse undermines end-to-end MoE training in this setting.
2. GES Framework: The proposal of a two-stage framework that uses iterative Gaussian territory evolution to resolve both collapse issues. By decoupling partitioning from specialization, GES ensures each specialist trains on a stable, well-defined region of the design space.
3. Hardware Validation: Successful deployment of a GES-optimized design on physical hardware, demonstrating a significant improvement in obstacle traversal capabilities.

Results

The method was validated on the BALLU platform in simulation (Isaac Lab) and on hardware.

• Simulation Performance:

  • Obstacle Traversal (2D Space): GES achieved a mean performance improvement of 25.5% over a monolithic universal policy (e2e MLP) and 8.62% over an end-to-end MoE. GES designs cleared obstacles up to 54 cm, compared to 38 cm for the universal policy.
  • Obstacle Traversal (3D Space): GES showed a 31.7% peak gain over the universal policy, with a mean improvement of 18.8%.
  • Ramp Climbing: GES improved mean performance by approximately 13% over baselines in 2D space.
  • Efficiency: GES reduced the total co-design wall-clock time by 37% (from ~100 hours to ~63 hours) by replacing costly RL re-training with direct evaluation.

• Hardware Results:

  • A co-designed BALLU (optimized for $GCR=0.86, SPCF=0.008$) successfully cleared a 24 cm obstacle.
  • This represents a 3x improvement over the baseline BALLU design, which could only clear an 8 cm obstacle.
  • The results confirm that morphology improvements discovered in simulation transfer effectively to the physical platform, despite sim-to-real gaps.

• Behavioral Analysis:

  • Unlike the baselines, GES specialists maintained behavioral diversity. Visualizations showed that different specialists controlling the same robot design produced qualitatively distinct foot trajectories and strategies to clear obstacles.
  • The end-to-end MoE baseline suffered from representation collapse, distributing designs equally among experts, whereas GES established hard partitions where each specialist dominated its specific subspace.

Significance and Claims

The paper claims that GES provides a scalable and efficient solution for robot co-design by overcoming the fundamental limitations of universal policies and standard MoE architectures. By explicitly capturing diverse behaviors through iterative territory evolution, GES enables the discovery of high-performance robot morphologies that would otherwise be missed by collapsed evaluators. The framework is presented as platform-agnostic, though the current validation is specific to buoyancy-assisted legged robots. The authors note that the method assumes a fixed number of specialists and a continuous, bounded design space, identifying these as areas for future adaptation.