Stability-Guided Exploration for Diverse Motion Generation

This paper proposes a novel black-box simulation method that combines RRT-style search, sampling-based MPC, and a stability-guided sampling scheme to efficiently discover diverse, long-horizon robot manipulation strategies without relying on labor-intensive human demonstrations or task-specific guidance.

The Gist

Imagine you are trying to teach a robot how to move objects around a room. You have two main options:

1. The "Human Tutor" Method: You spend years filming yourself pushing boxes, throwing balls, and using tools, then feed that footage to the robot. The problem? Humans are limited. We only know a few ways to do things, it's expensive to film, and we might miss the clever, weird, or super-efficient ways a robot could do it.
2. The "Robot Dreamer" Method: You let the robot simulate millions of scenarios in a computer. The problem? Robots are bad at dreaming. They often get stuck in a loop, trying the same small movement over and over, or they fall into a "local trap" (like a ball rolling into a corner where it can't get out) and give up.

This paper introduces a new method called StaGE (Stability-Guided Exploration) that acts like a smart tour guide for the robot's imagination. It helps the robot discover a huge variety of creative ways to move things without needing a human to show it how.

The Core Idea: The "Safe Harbor" Map

Think of the robot's world as a vast, foggy ocean. Most of the ocean is dangerous (unstable states where things fall over or break). However, there are scattered islands of calm water where everything is balanced and stable (like a ball sitting still on a table, or a cup resting on a hook).

Old methods tried to sail randomly across the whole ocean, hoping to find a path. They often got lost in the stormy parts.

StaGE's strategy is different:

1. Draw the Map: First, the robot quickly generates a map of all the "Safe Harbors" (stable states) it can imagine.
2. The Tour Guide: Instead of sailing randomly, the robot uses these Safe Harbors as destination markers. It asks, "How can I get from my current spot to that stable island?"
3. The Wild Ride: Here is the magic trick. The robot is not forced to stay on the calm water. To get from one island to another, it is allowed to sail through the stormy, unstable waves. It can throw a ball, catch it, or slide a box across a ramp. It just needs to make sure that at the end of the journey, it lands safely on a new island.

The Three Secret Ingredients

To make this work, the authors added three "superpowers" to the robot's brain:

• The "K-Nearest Neighbor" Trick (The Friendly Neighbor):
  Usually, a robot looks for the single closest stable island to aim for. But what if that island is blocked? StaGE tells the robot: "Don't just look at the closest one; look at the top 16 closest islands and pick one at random." This prevents the robot from getting stuck trying to reach a blocked target and encourages it to explore different directions.

• The "Top N Actions" Strategy (The Multi-Path Explorer):
  When the robot decides to move, it usually picks the one best move. StaGE says: "No, pick the top 16 best moves and try all of them!" This creates a branching tree of possibilities, like a choose-your-own-adventure book, ensuring the robot finds many different ways to solve the problem, not just the first one it sees.

• The "Dead-End Detector" (The Smart Quitter):
  If the robot tries to move and realizes it's heading into a place where it can never recover (like a ball falling off a cliff), it immediately stops trying to expand that path. It saves its energy for paths that actually lead somewhere.

What Did They Find?

The team tested this in four different "playgrounds":

1. The Ramp: A ball rolling down a slope. The robot learned to push it just right so it didn't fall off.
2. The Cube Push: Two robots pushing a box. They learned to throw the box, catch it, and spin it.
3. The Tool User: A robot arm with a hook. It learned to use the hook to grab a box it couldn't reach with its fingers.
4. The Handoff: Two robot arms working together, tossing a block back and forth like a game of catch.

The Result:
StaGE didn't just find one way to do these tasks. It found hundreds of unique, diverse strategies. It discovered that sometimes the best way to move an object is to throw it, sometimes to slide it, and sometimes to use a tool.

Why This Matters

In the past, we had to manually program robots to "throw" or "use a tool." With StaGE, we don't need to teach the robot the rules. We just give it the goal of "find stable states" and let it explore.

It's like giving a child a box of LEGOs and saying, "Build something stable," instead of giving them a specific instruction manual. The child (the robot) will build towers, bridges, and weird sculptures that the adult never thought of.

This method allows robots to learn complex, long-term skills (like juggling or using tools) purely by exploring, making them much more adaptable to the real world where things don't always go according to plan.

Technical Summary

1. Problem Statement

The paper addresses the bottleneck of data collection in robot learning. While scaling datasets improves deep learning performance, collecting human demonstrations is labor-intensive, narrow, and fails to explore the full space of feasible robot states. Conversely, existing synthetic data generation methods often rely on local trajectory optimization (e.g., Model Predictive Control or MPC), which are prone to getting stuck in local minima and fail to discover diverse, long-horizon solutions.

The core challenge is to generate diverse, dynamic, and contact-rich manipulation strategies (including non-prehensile actions like pushing, throwing, and tool use) across complex environments without relying on task-specific guidance, hand-crafted motion primitives, or analytical constraints.

2. Methodology: StaGE

The authors propose StaGE (Stability-Guided Exploration), a novel algorithm that combines Rapidly-exploring Random Trees (RRT) with sampling-based Model Predictive Control (MPC). The method operates in two main stages:

A. Sampling Physically Stable States (The Manifold)

Instead of sampling uniformly from the entire configuration space (which is inefficient due to the low probability of finding valid contact states), StaGE first samples from a manifold of stable configurations ($C_{stable}$).

• Process: It formulates a non-linear program to find states where objects are in quasi-static equilibrium (forces and moments sum to zero) and satisfy contact/friction constraints.
• Purpose: These stable states serve as "guideposts" or targets for the search tree. Crucially, the planner is not restricted to staying on this manifold; it uses these points to guide the search but allows the tree to grow through unstable, dynamic regions to enable complex maneuvers.

B. Kinodynamic RRT with Stability Guidance

The second stage builds a search tree rooted at a starting state to find paths between stable configurations.

• Black-Box Simulation: The planner interacts directly with a physics simulator (black-box) without needing gradient information.
• Key Extensions for Diversity:
  1. $K$-Nearest Neighbors (K-NN): Instead of extending the single closest node to a target, the algorithm randomly selects one of the $k$-nearest neighbors. This prevents the tree from getting stuck and encourages branching.
  2. $N$-Best Actions: When extending a node, the algorithm selects the top $n$ actions that reduce the distance to the target, rather than just the single best action. This promotes path diversity.
  3. Node Rejection: If a node fails to expand toward any target stable state, it is marked as a "dead-end" and disabled. This filters out unrecoverable states (e.g., an object falling off a ramp) early in the process.
  4. Path Extraction & Filtering: Once the tree is grown, paths are extracted from nodes near stable states. Redundant paths are removed using the Hausdorff distance to ensure the final dataset contains truly diverse trajectories.

3. Key Contributions

1. Novel Algorithm (StaGE): A task-agnostic method that finds complex, long-horizon manipulations without motion priors. It uniquely combines RRT-style global exploration with stability-guided sampling.
2. Stability-Guidance Scheme: A mechanism that guides the search toward stable configurations while permitting exploration through unstable, dynamic states. This enables the discovery of non-prehensile skills (throwing, pivoting) that pure stable-state planners would miss.
3. Task-Agnostic Diversity: The method generates diverse behaviors (pushing, grasping, tool use, handovers) purely through exploration, without manually tuned cost functions or specific task definitions.
4. First of its Kind: To the authors' knowledge, this is the first generic method applying RRT with black-box simulation to non-prehensile manipulation without relying on hand-crafted primitives or analytical constraints.

4. Experimental Results

The method was evaluated in four challenging environments with different robot morphologies (single/multi-robot, translational joints, and 7-DOF arms):

• SpheresRamp: A ball on a ramp (tests non-recoverable states).
• SpheresCube: Two robots manipulating a cube (tests orientation changes and multi-contact).
• PandaHook: A robotic arm using a hook to manipulate a cube (tests tool use).
• PandasCube: Two Panda arms collaborating (tests bimanual cooperation).

Performance Metrics:

• Coverage: StaGE significantly outperformed baselines (RRT-sim and Predictive Sampling) in the percentage of stable states reached.
• Path Count & Diversity: StaGE generated orders of magnitude more diverse paths. For example, in the SpheresCube environment, StaGE found 134.2 diverse paths compared to 0.1 for the RRT-sim baseline.
• Entropy: The method achieved higher state entropy, indicating a broader exploration of the state space.
• Ablation Studies: Removing the "N-best actions" or "K-nearest neighbors" extensions drastically reduced performance, confirming their necessity for diversity.

Notable Discoveries: The system autonomously discovered complex skills such as:

• Using a hook to pull a cube.
• Throwing and catching a cube between two robots.
• Pivoting objects against walls.

5. Significance and Conclusion

This work demonstrates that pure exploration, when guided by physical stability constraints, is sufficient to discover highly complex robotic skills without human demonstration or task-specific reward engineering.

• Impact on Robot Learning: By providing a scalable way to generate diverse synthetic data, StaGE addresses the data bottleneck in robot learning, potentially enabling better training for foundation models in robotics.
• Generalization: The method generalizes across different robot morphologies and environments without re-tuning, suggesting a robust approach to kinodynamic planning in contact-rich scenarios.
• Future Directions: The authors suggest that while stable states are a good starting point, future work could incorporate other informative states (e.g., impact moments) and improve trajectory smoothness.

In summary, StaGE bridges the gap between global sampling-based planning and local dynamic control, offering a powerful tool for generating the diverse, high-quality datasets required for the next generation of robot learning.