⚡ electrical engineering

Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports

This paper proposes a human-centered safety filter (HCSF) for shared autonomy that uses a learned neural safety value function to provide smooth, minimal interventions via a model-free control barrier function, enhancing safety and user satisfaction in complex environments like AI-assisted motorsports without compromising human agency.

Original authors: Donggeon David Oh, Justin Lidard, Haimin Hu, Himani Sinhmar, Elle Lazarski, Deepak Gopinath, Emily S. Sumner, Jonathan A. DeCastro, Guy Rosman, Naomi Ehrich Leonard, Jaime Fernández Fisac

Published 2026-02-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Donggeon David Oh, Justin Lidard, Haimin Hu, Himani Sinhmar, Elle Lazarski, Deepak Gopinath, Emily S. Sumner, Jonathan A. DeCastro, Guy Rosman, Naomi Ehrich Leonard, Jaime Fernández Fisac

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are learning to ride a bicycle. You want to feel the wind in your face and the thrill of steering yourself, but you’re also terrified of crashing into a tree.

If you had a "Last-Resort" Safety System, it would be like a robot that lets you ride however you want, but the very millisecond you are about to hit that tree, it grabs the handlebars with a violent jerk and yanks you away. You’re safe, but you’re startled, you might fall over from the sudden movement, and you feel like you’ve lost control of your own bike.

This paper introduces a much smarter way to help: the Human-Centered Safety Filter (HCSF).

The Core Idea: The "Gentle Nudge" vs. The "Panic Grab"

The researchers wanted to solve a problem in high-stakes environments—specifically, professional car racing. In racing, a driver needs to feel "in the zone." If an AI co-pilot suddenly takes over the steering wheel to prevent a crash, it ruins the driver's rhythm, causes "automation surprise" (that "Hey! What just happened?!" feeling), and makes the driver stop trusting the machine.

The Old Way (LRSF): Think of this like a strict parent who ignores you while you're playing, but the moment you step one inch toward a hot stove, they scream and physically pull you back. It works, but it’s jarring and scary.

The New Way (HCSF): Think of this like a professional driving instructor sitting next to you. They don't grab the wheel. Instead, as you approach a sharp turn too fast, they gently guide your hands or apply a tiny bit of extra pressure to the brake. You still feel like you are driving, but the car stays on the track. It’s a "nudge" rather than a "yank."

How does it work? (The "Magic" behind the scenes)

Usually, to make a safety system, engineers need a perfect mathematical map of how a car works (how much the tires grip, how the engine reacts, etc.). But in complex simulators or real-world racing, that "map" is too complicated to write down.

The researchers used Artificial Intelligence to learn the "rules of safety" through trial and error, much like a human learns by practicing.

Learning the Danger Zones: The AI "drives" millions of miles in a simulator, learning exactly which positions and speeds lead to a crash.
The Safety Math (Q-CBF): Instead of needing a map of the car's engine, the AI creates a "safety bubble" around the car. It constantly asks: "If the human does this action, will we stay inside the bubble?"
The Minimal Change: If the human's action would pop the bubble, the AI calculates the smallest possible correction needed to keep the bubble intact. It’s like a GPS that doesn't tell you to "Turn Left Now!" with a loud beep, but instead subtly adjusts your route so you naturally end up in the right lane.

Does it actually work?

The researchers tested this with 83 real people using a high-end racing simulator (Assetto Corsa). They compared three groups:

The Solo Drivers: No help at all (lots of crashes).
The "Panic Grab" Group: The old-school, abrupt safety system.
The "Gentle Nudge" Group: The new HCSF system.

The Results:

Safety: Both safety systems kept people from crashing, but the "Gentle Nudge" felt much more natural.
Agency (The "I'm in Charge" feeling): People in the "Gentle Nudge" group felt like they were still the ones driving. People in the "Panic Grab" group felt like the car was fighting them.
Comfort: The "Gentle Nudge" was smooth. The "Panic Grab" was jerky and uncomfortable.
Happiness: Surprisingly, the people with the "Gentle Nudge" were actually happier and more satisfied with the experience than those driving alone!

Why does this matter?

This isn't just about racing. This technology could be used in:

Self-driving cars: Where the car doesn't "fight" you, but gently corrects your lane position.
Robotic surgery: Where a robot helps a surgeon avoid hitting a vital organ without taking over the entire procedure.
Industrial robots: Where machines work alongside humans and need to stay safe without being unpredictable.

In short: This paper proves that we don't have to choose between being safe and being in control. We can have both, as long as the AI learns to be a polite partner rather than a sudden intruder.

Technical Summary: Safety with Agency: Human-Centered Safety Filter (HCSF)

1. Problem Statement

In human-AI shared autonomy—particularly in high-stakes, high-performance environments like motorsports—traditional safety filters face a critical trade-off. While existing methods (such as Last-Resort Safety Filters or LRSF) effectively prevent catastrophic failures, they often do so through abrupt, discontinuous overrides that disregard the human operator's intent. This leads to:

Loss of Agency: The driver feels they have lost control.
Automation Surprise: Unpredictable interventions cause confusion and distrust.
Reduced Comfort: Jerky, "last-moment" corrections create a poor user experience.
Scalability Issues: Most traditional model-based safety filters (like Hamilton-Jacobi reachability) suffer from the "curse of dimensionality," making them difficult to apply to complex, high-dimensional systems with unknown dynamics.

2. Methodology

The authors propose the Human-Centered Safety Filter (HCSF), a framework designed to provide smooth, minimal interventions that prioritize human agency while guaranteeing safety.

A. Mathematical Framework: The Q-CBF
The core innovation is the introduction of a model-free state–action Control Barrier Function (Q-CBF).

Neural Synthesis: Instead of requiring a mathematical model of the system dynamics (which is often unknown in complex environments), the authors use model-free Reinforcement Learning (RL)—specifically a variant of Soft Actor-Critic (SAC)—to learn a neural safety value function $Q(x, u)$ . This function represents the expected future safety margin for a given state $x$ and action $u$ .
The Constraint: The HCSF solves an optimization problem at each timestep to find an action $u$ that is as close as possible to the human's intended action $u_{human}$ while satisfying the safety constraint:
$Q(x, u) \geq \gamma V(x)$
where $V(x)$ is the state-only safety value function and $\gamma$ is a design parameter. This ensures the system stays within the maximal safe set without needing to know the system's underlying physics (control-affine models, etc.).

B. Implementation & Human-Machine Interface (HMI)

Environment: The system was deployed in Assetto Corsa, a high-fidelity racing simulator, using a professional racing rig (steering wheel, pedals, etc.).
Transparency: To prevent "automation surprise," the HCSF includes visual cues (horizontal and vertical arrows on the screen) that communicate the direction and magnitude of the AI's corrections in real-time.

3. Key Contributions

First Fully Model-Free CBF: A safety filter that requires zero knowledge of system dynamics for either synthesis or runtime monitoring, making it applicable to "black-box" systems.
Agency-Preserving Optimization: A novel formulation that treats safety as a "nudge" (minimal deviation) rather than a "hard override."
High-Dimensional Scalability: By leveraging neural approximations via RL, the method scales to high-dimensional state spaces (e.g., 133-dimensional observations in racing) that are intractable for traditional grid-based methods.
Empirical Validation: A large-scale, in-person user study (83 participants) evaluating both quantitative trajectory data and qualitative human perception.

4. Experimental Results

The study compared the HCSF against LRSF (conventional last-resort filter) and Unassisted Driving.

Robustness: HCSF achieved near-zero failures and significantly reduced out-of-track incidents compared to unassisted driving. It maintained a level of robustness at least equal to LRSF.
Human Agency: HCSF significantly outperformed LRSF in agency. While LRSF caused a drop in the driver's sense of control, HCSF maintained an agency level comparable to unassisted driving by keeping input modifications small and smooth.
Comfort: HCSF produced significantly lower jerk and input difference (I.D.) magnitudes than LRSF, resulting in a much smoother perceived driving experience.
Satisfaction: HCSF users reported significantly higher overall satisfaction and higher levels of trustworthiness and competence in the AI compared to both other groups.

5. Significance

This work shifts the paradigm of robot safety from "safety at all costs" to "safety with agency." By proving that a safety filter can be both mathematically principled (based on HJ reachability and CBF theory) and human-centric (smooth and transparent), the authors provide a blueprint for future shared autonomy systems. This has profound implications for Advanced Driver Assistance Systems (ADAS), collaborative industrial robotics, and any high-stakes environment where human-AI teamwork is essential for both safety and performance.