Technical Summary: Safe Stochastic Explorer (S.S. Explorer)

Problem Statement

Autonomous robots operating in unstructured, safety-critical environments (e.g., planetary exploration, warehouses, homes) must navigate and interact with surroundings where system dynamics and safety characteristics are not fully known. Existing safe control methods, such as Hamilton-Jacobi Reachability (HJR) and Control Barrier Functions (CBFs), typically assume known system dynamics. Conversely, existing safe exploration techniques often fail to account for the unavoidable stochasticity inherent in real-world interactions, such as a rover skidding on unknown terrain or a robot pushing unmapped objects.

The core problem addressed is safe, goal-driven exploration in unknown environments under stochastic dynamics. The robot must balance two competing objectives:

1. Exploration: Gathering information to reduce uncertainty about the environment's safety properties.
2. Safety: Ensuring with high confidence that the robot remains within a safe set throughout its operation, despite stochastic state transitions.

The paper defines the safety function $q(x)$ as unknown, where a state is unsafe if $q(x) > q_T$. The agent operates under a stochastic transition model $x_{t+1} \sim p(x_{t+1}|x_t, a_t)$ and aims to reach a goal maximizing a performance function $f(x)$ without violating safety constraints.

Methodology

The authors propose Safe Stochastic Explorer (S.S. Explorer), a framework that utilizes Gaussian Processes (GPs) to learn the unknown safety function online. The method leverages the predictive uncertainty of GPs to guide information-gathering actions while providing probabilistic bounds on safety violations. The framework is presented in three variations:

1. Discrete State Space Formulation

For finite, discrete state spaces, the method models the agent's motion as a Markov chain. It defines "completely safe" states through three iterative criteria:

• Immediate Safety: The state's safety cost is below the threshold with high probability.
• Safe Stochastic Return: There exists a guaranteed trajectory back to the initial safe set without exiting the set of immediately safe states, ensuring recursive feasibility.
• Safe Stochastic Arrival: The agent can transition to the state from the current safe set while ensuring the entire stochastic arrival distribution remains within safe bounds.

The algorithm employs set-expansion and refinement operators:

• Immediate Safe Expansion ($\bar{I}$): Uses GP upper/lower confidence bounds ($u_t(x), l_t(x)$) and Lipschitz continuity assumptions to expand the set of states believed to be safe.
• Safe Stochastic Return Refinement ($\bar{R}$): Iteratively restricts the safe set to states from which the agent can return to the safe core despite stochastic deviations.
• Safe Stochastic Arrival Refinement ($\bar{m}$): Ensures that the distribution of states reachable in one step from the safe set remains safe.

The agent selects actions by identifying "candidate expanders"—uncertain but pessimistically safe states near the frontier of the safe set that provide information about target regions.

2. Continuous State Space Formulation

To address scalability in continuous domains, the authors introduce a relaxation. Instead of tracking complex sets, they use indicator functions derived from GP posterior distributions.

• Relaxation: The strict "Safe Stochastic Return" guarantee is relaxed to ensure only "Safe Stochastic Arrival" and immediate safety. This removes the guarantee of task completion (avoiding getting stuck) but ensures instantaneous safety.
• Analytical Computation: To handle the stochastic arrival of a Gaussian transition distribution $T(x, a) \sim \mathcal{N}(\mu, \Sigma)$, the method uses moment matching to analytically compute the safety value distribution of the arrival state. This allows for the direct evaluation of safety indicators without sampling.
• Path Safety: Lipschitz continuity is used to adjust the safety threshold ($q_T' = q_T - L \cdot d_{max}/2$) to guarantee safety along the continuous trajectory between discrete planning steps.

3. Object-Centric Formulation

The framework is extended to scenarios involving physical interaction with multiple unknown objects (e.g., pushing items on a shelf).

• Object-Centric Representation: To handle moving objects, safety is modeled in the relative state space of the robot with respect to each object. This allows the GP to reuse past interaction data even as object positions change.
• Multi-Object Safety: The global safety function is defined as the maximum safety cost across all objects. The overall safe set is computed as the intersection of the individual safe sets for each object.
• Implementation: In the continuous case, this is implemented by training separate GPs for each object and aggregating their worst-case probabilistic bounds during prediction, avoiding the computational cost of intersecting high-dimensional sets explicitly.

Key Contributions

1. Discrete Safe Exploration: A novel method for safe exploration in discrete state spaces with stochastic transitions, utilizing GP-based uncertainty to bound safety violations and guarantee safe stochastic return.
2. Continuous Relaxation: A scalable relaxation for continuous state spaces that retains safety guarantees (immediate safety and safe arrival) while improving computational efficiency for real-world domains, albeit with a relaxed guarantee on task completion.
3. Safe Object Interaction: An application-driven extension demonstrating how the framework ensures safe physical interaction with multiple unknown objects by modeling safety in relative coordinates and intersecting safe sets.

Experimental Results

The authors validated S.S. Explorer through extensive simulations and hardware experiments:

• Ground Robot (Discrete): In 80 trials across 8 environments with high stochasticity ($p=0.35$ for intended moves), S.S. Explorer achieved a 100% success rate with 0% safety violations. In contrast, a baseline method (which ignored stochasticity) achieved only 35% success and 65% violation rates.
• Ground Robot (Continuous): Across 425 trajectories, S.S. Explorer achieved a 70.35% success rate with a 6.82% violation rate, significantly outperforming the baseline (41.8% success, 49.8% violation). A variant with dynamic $\beta$-scaling (adjusting conservatism based on progress) achieved an 88.47% success rate with a 9.88% violation rate.
• Safe Object (Simulation): In 55 trials involving a robotic arm pushing unknown objects, the stochastic formulation achieved up to 81.8% success with 0% violations (with specific variance settings), whereas the baseline failed 50.9% of the time.
• Hardware: Real-world experiments using a Franka Emika Panda arm successfully navigated a table with Pringles cans of varying masses. The method reached the goal without toppling objects, while a naive baseline toppled the objects, violating safety.

Significance and Claims

The paper claims that S.S. Explorer represents a step forward toward enabling reliable, widespread robot autonomy in complex, uncertain environments. Its significance lies in:

• Bridging the Gap: Explicitly accounting for stochastic dynamics in safe exploration, a critical gap left by prior methods that assume known dynamics or neglect transition uncertainty.
• Handling Unknown Interactions: Providing a principled framework for safe physical interaction with unknown objects, moving beyond simple collision avoidance to managing interaction dynamics (e.g., tipping, friction).
• Balancing Safety and Performance: Demonstrating that by leveraging uncertainty (via GPs) and stochastic modeling, robots can explore more effectively and reach goals more often than deterministic baselines, which tend to fail or violate safety when faced with real-world noise.

The authors acknowledge limitations, including the computational scalability of GPs ($O(N^3)$), the reliance on prior knowledge of system bounds (e.g., Lipschitz constants), and the trade-off between safety and performance. They note that the continuous and object-centric variants relax the "safe stochastic return" guarantee, meaning the robot could theoretically get stuck, though it remains instantaneously safe. Future work aims to reintroduce return guarantees and handle complex inter-object interactions.