Curvature-Guided Safety Filters: State-Dependent Hessian-Weighted Projection with Provable Performance Bounds

1. Problem Statement

Safety filters are essential for ensuring that learning-enabled control systems (e.g., Reinforcement Learning) operate within safe boundaries without compromising performance.

• The Limitation of Current Methods: Most existing safety filters rely on Euclidean projection, which minimizes the geometric distance between a nominal (potentially unsafe) action and the safe set. While computationally efficient and safe, this approach is often suboptimal because it ignores the long-term value (performance) of the action. It treats all directions of deviation from the optimal action equally, potentially penalizing directions that are critical for long-term reward.
• The Optimization Dilemma: Directly optimizing the action-value function ($Q$-function) within the safe set to find the true optimal safe action is theoretically ideal but computationally prohibitive. The resulting optimization problem is typically nonconvex due to the complex shape of the $Q$-function and the safe set, making real-time implementation difficult.
• The Goal: Develop a safety filter that maintains strict safety guarantees and real-time computational efficiency (like Euclidean projection) but significantly improves long-term performance by incorporating the curvature of the value function.

2. Methodology

The authors propose a Curvature-Guided Safety Filter that replaces the standard Euclidean metric with a state-dependent, Hessian-weighted metric.

A. Hessian-Weighted Projection

Instead of minimizing the Euclidean distance $\|u - u_{ref}\|_2$, the filter solves a weighted quadratic program:
$$u_W = \arg \min_{u} \|u - u_{ref}\|^2_{W(x)} \quad \text{s.t.} \quad u \in \mathcal{U}(x)$$
Where:

• $u_{ref}$ is the unconstrained optimal action derived from the $Q$-function.
• $\mathcal{U}(x)$ is the state-dependent admissible safe set.
• The Weighting Matrix: $W(x) = -\nabla^2_u Q(x, u_{ref})$.
  • This matrix is derived from the Hessian (second derivative) of the action-value function with respect to the action.
  • By using the negative Hessian, the projection penalizes deviations in directions where the $Q$-function is highly sensitive (high curvature), effectively biasing the correction toward action directions that preserve long-term value.

B. Theoretical Guarantees

The paper establishes rigorous performance bounds under mild regularity assumptions (Lipschitz continuity of the Hessian and local negative definiteness):

1. Safety Preservation: The method guarantees forward invariance of the safe set. Since the optimization is constrained to the safe set $\mathcal{U}(x)$, safety is preserved regardless of the specific objective function used.
2. Near-Optimality Bound: The authors prove a uniform bound on the performance gap between the weighted projection ($u_W$) and the true safe optimal action ($u^*$). The error is bounded by a constant $C$ dependent on the Hessian Lipschitz constant ($L_2$) and the curvature lower bound ($\mu$).
3. Superiority Condition: A sufficient condition is derived showing that the Hessian-weighted projection outperforms Euclidean projection when the local curvature of the $Q$-function is strong relative to higher-order variations.

C. Data-Driven Construction for Black-Box Controllers

Since the true $Q$-function is often unknown in learning-based control, the authors propose a data-driven scheme to construct $W(x)$:

• Structured Function Approximation: They use a linear function approximator for $Q_\theta(x, u)$ with specific feature blocks:
  • Constant and linear terms.
  • Quadratic terms: Explicitly designed to capture curvature ($\psi_s(x) \otimes \text{vech}(uu^\top)$).
  • Nonlinear residual terms.
• Regularized Fitted Q-Iteration: During offline training, they employ a least-squares regression with two specific regularization terms:
  1. Curvature Dominance ($R_\mu$): Ensures the quadratic coefficients form a positive definite matrix (enforcing the negative definiteness of the Hessian).
  2. Lipschitz Control ($R_{L2}$): Bounds the third-order derivatives of nonlinear features to control the Hessian's Lipschitz constant.
• Online Execution: Once trained, the matrix $W(x)$ is computed analytically. The online step involves solving a convex quadratic program, maintaining real-time efficiency comparable to Euclidean projection.

3. Key Contributions

1. Novel Projection Metric: Introduction of a state-dependent, Hessian-weighted projection that utilizes the curvature of the action-value function to improve performance without sacrificing convexity.
2. Provable Performance Bounds: Derivation of theoretical bounds proving that the proposed filter is near-optimal and establishing specific conditions under which it strictly outperforms standard Euclidean projection.
3. Data-Driven Implementation: A practical framework for learning the weighting matrix from data using structured function approximation and regularization, making the method applicable to black-box controllers.
4. Error Analysis: Quantification of how approximation errors in the learned $Q$-function propagate to the final control action, showing that the performance advantage is preserved even with learning errors if the curvature is sufficiently strong.

4. Results

The method was validated on a nonlinear quadrotor tracking-and-avoidance task involving obstacles.

• Safety: Both Euclidean and Hessian-weighted filters successfully maintained safety (obstacle avoidance).
• Performance: The Hessian-weighted filter achieved tighter tracking near obstacles and demonstrated a higher action-value (Q-value) compared to the Euclidean baseline, indicating better long-term performance.
• Efficiency: The computational overhead of the proposed method was comparable to Euclidean projection, confirming its suitability for real-time operation.

5. Significance

This work bridges the gap between safety and performance in learning-enabled control.

• It moves beyond the "conservative" nature of standard safety filters by acknowledging that not all deviations from the optimal action are equally costly.
• It provides a theoretically grounded alternative to nonconvex optimization, offering a convex formulation that is provably close to the global optimum.
• The data-driven approach makes it applicable to complex, real-world systems where analytical models of the value function are unavailable, offering a path toward safer and more efficient autonomous systems.