Technical Summary: Safe Exploration for Nonlinear Processes Using Online Gaussian Process Learning

Problem Statement

The paper addresses the challenge of controlling nonlinear systems with partially known dynamics in safety-critical applications (e.g., robotics, aerospace, process industries). While data-driven methods like reinforcement learning offer adaptability, they often lack hard guarantees on stability and constraint satisfaction during the exploration phase. Existing safe learning approaches frequently rely on idealized assumptions, such as full knowledge of nonlinear dynamics or conservative Lipschitz over-approximations of uncertainty, which limit their practicality.

The specific problem tackled is designing a control framework that ensures safe exploration for nonlinear systems where:

1. Only a stabilizable linear approximation of the process is known.
2. Unmodeled nonlinear dynamics (residuals) must be learned online.
3. The system must satisfy state and input constraints with high probability throughout the learning process.

Methodology

The proposed framework integrates Gaussian Process (GP) regression, Lyapunov stability theory, and convex optimization to achieve safe, informative exploration.

1. System Decomposition and GP Modeling

The system dynamics are modeled as a control-affine nonlinear system:
$$\dot{x} = Ax + Bu + g(x)$$
where $(A, B)$ is a known, stabilizable linearization, and $g(x)$ is an unknown smooth residual. The residual $g(x)$ is modeled online using a Gaussian Process. As data is collected, the GP provides a posterior mean $\mu_t(x)$ and standard deviation $\sigma_t(x)$. A high-probability confidence envelope is constructed such that:
$$\|g(x) - \mu_t(x)\|_2 \leq \beta_t \sigma_t(x)$$
with probability $1-\delta$, where $\beta_t$ is a time-varying confidence parameter derived from finite-sample GP-UCB schedules.

2. Probabilistic Control-Invariant Sets (PCIS)

Safety is enforced through a Probabilistic Control-Invariant Set (PCIS) derived from a Control Lyapunov Function (CLF), $V(x) = x^\top P x$. The PCIS is defined as the set of states where there exists a control input that ensures the Lyapunov function decreases, even under the worst-case GP uncertainty within the confidence envelope.
The condition for the set $S_t$ is:
$$\nabla V \cdot (Ax + Bu + \mu_t(x)) + \beta_t \sigma_t(x) \|\nabla V\| + \lambda V(x) \leq 0$$
This construction guarantees that if the system starts within the PCIS, it remains there with high probability ($1-\delta$), ensuring recursive feasibility of constraints.

3. Optimization-Based Controller

At each time step, the controller solves a Convex Quadratic Program (QP) to determine the control input $u$. The objective is to maximize information gain (by encouraging excitation in directions of high uncertainty) while strictly adhering to safety constraints.
The QP minimizes the deviation from a nominal stabilizing controller $u_{lin}$:
$$\min_{u, s \geq 0} \|u - u_{lin}\|_2^2 + \rho s$$
subject to the Lyapunov decrease condition (incorporating the GP uncertainty bound) and input constraints. A slack variable $s$ ensures feasibility, penalized by $\rho$ to prioritize safety.

4. Adaptive Expansion

As the GP learns the residual dynamics, the uncertainty $\sigma_t(x)$ decreases. This reduction allows the PCIS to expand adaptively, unlocking progressively richer exploration regions without violating safety guarantees.

Key Contributions

The paper outlines three primary contributions:

1. Nominal-Residual Decomposition: A framework combining a known linear model with an online GP residual, enabling safe learning of unknown nonlinearities without requiring full model knowledge.
2. Lyapunov-Based PCIS Construction: A method to construct probabilistic invariant sets that incorporate GP uncertainty, providing finite-sample probabilistic guarantees for stability and constraint satisfaction.
3. Optimization-Based Controller: A real-time controller that solves a convex QP at each step to balance information gain (exploration) with strict safety guarantees (CLF decrease), avoiding the conservatism of multi-step uncertainty propagation found in other Safe MPC formulations.

Experimental Results

The framework was validated on two benchmarks: a 2D polynomial system and a MIMO three-tank water-level process.

2D Polynomial System

• Learning Performance: The GP model achieved significant accuracy improvements (RMSE reduced from ~1.11 to ~0.03) over 12 iterations.
• Uncertainty Calibration: Empirical coverage of the 95% confidence intervals improved from 37% to 100% after calibration, ensuring the safety certificates were reliable.
• Safety: The proposed GP-PCIS controller maintained all trajectories within the certified safe set with zero constraint violations. In contrast, an unsafe baseline (LQR without safety layers) violated state constraints.
• Set Expansion: The certified safe set volume expanded by approximately 27.7% as uncertainty decreased, demonstrating the adaptive nature of the framework.

Three-Tank Water-Level Process

• Complexity: This MIMO, coupled system tested the framework's scalability and real-time feasibility.
• Sparse vs. Exact GP: Sparse GP approximations (using inducing points) were shown to be viable, offering >1000x faster inference times compared to exact GPs while maintaining near-nominal coverage (~94.5%) and acceptable RMSE.
• Safety: Over a 2000-second identification horizon, the framework achieved zero violations of water level constraints (45%–87%), whereas an unsafe baseline briefly dipped below the lower limit.
• Real-Time Performance: Median per-step QP solve times were in the millisecond range, confirming feasibility for real-time implementation.

Significance and Claims

The paper claims to provide a unified and implementable framework for safe exploration in partially known nonlinear systems. Its significance lies in:

• Balancing Tractability and Conservatism: Unlike multi-step Safe MPC formulations that can become overly conservative, this approach enforces a one-step Lyapunov decrease with GP confidence bounds, resulting in a convex QP without multi-step uncertainty blow-up.
• Practicality: It requires only a stabilizable linear model (common in practice) rather than exact nonlinear dynamics, making it applicable to real-world systems where unmodeled effects must be learned online.
• Theoretical Guarantees: It offers finite-sample probabilistic safety guarantees, ensuring that the system remains stable and within constraints with high probability, even during the active learning phase.

The authors acknowledge limitations, including the conservatism of one-step worst-case bounds and the cubic scaling of exact GP training, suggesting future work on multi-step chance constraints and sparse GP updates for higher-dimensional tasks. However, the results demonstrate that the proposed method effectively enables informative exploration while strictly adhering to safety constraints.