⚡ electrical engineering

Safe Human-to-Humanoid Motion Imitation Using Control Barrier Functions

This paper presents a vision-based framework that enables safe, real-time human-to-humanoid motion imitation by capturing skeletal keypoints and enforcing collision avoidance through a Control Barrier Function layer formulated as a Quadratic Program.

Original authors: Wenqi Cai, John Abanes, Nikolaos Evangeliou, Anthony Tzes

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenqi Cai, John Abanes, Nikolaos Evangeliou, Anthony Tzes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot that can dance, wave, or high-five just by watching you do it. That's the dream of Human-to-Humanoid Motion Imitation. But there's a big catch: if the robot tries to copy you too literally, it might twist its own arm off, or worse, accidentally punch you in the face.

This paper presents a "smart safety guard" that lets a robot copy your moves safely, without needing expensive suits or sensors. Here is how it works, broken down into simple concepts:

1. The Eyes: "The Ghost in the Machine"

Usually, to make a robot move like a human, you need special suits with sensors everywhere. This paper uses a single camera instead.

The Analogy: Think of the camera as a pair of eyes that sees you not as a person, but as a stick-figure skeleton (like the ones you see in video games).
The Magic: The computer instantly translates your stick-figure pose into the robot's joint angles. It's like a translator that turns "Human English" into "Robot Spanish" in real-time.

2. The Problem: The "Clumsy Dancer"

If you tell a robot to copy you perfectly, it might try to do something impossible.

Self-Collision: Imagine trying to hug yourself so tightly that your elbow hits your own nose. The robot might try this too.
Human-Robot Collision: If you wave your hand near the robot, it might try to wave back and accidentally hit your hand.
The Challenge: How do you let the robot be responsive and fun, but stop it from hurting itself or you?

3. The Solution: The "Invisible Force Field" (Control Barrier Functions)

This is the core of the paper. They use a mathematical tool called a Control Barrier Function (CBF).

The Analogy: Imagine the robot is driving a car, and you are the passenger giving it directions (the "imitation command").
- Normally, the car follows your directions perfectly.
- But, there is an invisible force field around the car and around you.
- If your directions would drive the car into a wall (a self-collision) or into you (a human collision), the force field gently pushes the steering wheel back.
- The car still tries to go where you want, but it takes a slightly different path to stay safe. It's like a very polite, invisible bodyguard that says, "I can't let you do that, but here is the closest safe way to do it."

4. The Shape: "The Hot Dog Bun Model"

To make this force field work fast, the computer needs to know the shape of the robot and the human.

The Analogy: Instead of trying to calculate the complex shape of a human body with muscles and clothes, the system simplifies everything into capsules (like hot dog buns or pill shapes).
The robot's torso, arms, and legs are all just a collection of these hot dog buns. The human is too.
The computer just checks: "Are these two hot dog buns touching?" If they are getting too close, the safety guard kicks in. This is much faster and easier for the computer to calculate than complex 3D models.

5. The Brain: The "Traffic Cop" (The QP Solver)

All these calculations happen in a split second using a method called a Quadratic Program (QP).

The Analogy: Think of this as a super-fast traffic cop at a busy intersection.
- Input: "I want to go North!" (The human's move).
- Constraint: "But there is a red light and a pedestrian crossing!" (The safety rules).
- Output: The cop instantly calculates the best path: "Go North, but slow down and turn slightly left to avoid the pedestrian."
The robot does this hundreds of times per second. If no danger is near, it copies you perfectly. If danger is near, it tweaks the move just enough to be safe.

Why Does This Matter?

No Costly Gear: You don't need to wear a $10,000 motion-capture suit. A regular webcam works.
Real-Time Safety: It doesn't just plan a safe path ahead of time; it reacts instantly if you suddenly move your hand toward the robot.
Future of Interaction: This is a huge step toward having robots that can work alongside us, learn from us, and play with us without us needing to be afraid of getting hurt.

In a nutshell: This paper teaches a robot how to be a good dance partner. It watches you, tries to copy your moves, but has a built-in "don't touch" instinct that gently nudges it away from crashing into itself or you, all while keeping the dance flowing smoothly.

1. Problem Statement

The paper addresses the critical challenge of ensuring operational safety in human-to-humanoid motion imitation. While camera-based human pose estimation offers a low-cost, accessible alternative to wearable motion capture systems for teleoperation and demonstration, it introduces significant safety risks.

The Core Issue: Real-time imitation often leads to self-collisions (e.g., arms crossing the torso) and human-robot collisions (e.g., the robot's arm striking the human demonstrator) because standard retargeting algorithms do not inherently account for geometric constraints.
The Gap: Existing methods often lack a unified, online safety filtering mechanism that can process visual inputs and enforce collision avoidance without compromising the responsiveness of the imitation.

2. Methodology

The authors propose a vision-based framework that integrates pose estimation, motion retargeting, and a safety layer based on Control Barrier Functions (CBF) formulated as a Quadratic Program (QP). The system operates in three main stages:

A. Vision-Based Pose Estimation & Retargeting

Input: A single camera captures human skeletal keypoints.
Filtering: Raw skeleton data is processed using an Exponential Moving Average (EMA) combined with jump rejection to ensure robustness.
Pose Reduction: Instead of solving a full inverse kinematics problem, the system estimates a reduced 8-DoF human pose vector ( $\theta$ ) representing torso roll/pitch, shoulder pitch/roll, and elbow pitch.
Retargeting: An affine mapping transforms the human pose into the robot's joint space ( $q_{nom}$ ). This includes scaling, offsetting, and clipping to joint limits. This nominal command is treated as "unsafe" and passed to the safety layer.

B. Capsule-Based Collision Modeling

To represent geometry efficiently, both the robot and the human are approximated using a 7-capsule model:

Components: Torso, left/right upper arms, left/right forearms, and left/right thighs.
Parameterization: Each capsule is defined by two endpoints and a radius.
- Robot: Endpoints are updated via Forward Kinematics (FK) from the current joint state.
- Human: Endpoints are extracted from denoised skeleton points and transformed into the robot's reference frame.

C. CBF-QP Safety Control Layer

This is the core contribution, acting as a filter between the nominal command and the robot controller.

Safety Function: For every pair of capsules (robot-robot or robot-human), a safety function $h_{AB}$ $h_{A B}$ is defined based on the distance between the closest points of the capsules minus the sum of their radii and a barrier margin ( $\phi$ $ϕ$ ).
- $h > 0$ : Safe.
- $h = 0$ : Boundary.
- $h < 0$ : Collision.
Constraint Formulation: The system enforces the standard first-order CBF condition ( $\dot{h} + \gamma h \geq 0$ ). Using the chain rule, this is linearized into a constraint on joint velocities ( $u$ ):
$A_i(q)u \geq -\gamma h_i(q)$
Optimization: The system solves a Quadratic Program (QP) in real-time:
- Objective: Minimize the deviation between the actual joint velocity ( $u$ ) and a reference velocity ( $u_{ref}$ ) derived from the nominal command.
- Constraints: All active CBF constraints (self-collision and human-robot collision) and joint velocity limits.
- Output: A safe joint velocity command that modifies the nominal motion only when necessary to prevent collisions.

3. Key Contributions

Unified Safety Framework: Established a vision-based pipeline that enables online, collision-aware motion generation directly from camera observations, bridging the gap between perception and safe execution.
Capsule-Based CBF-QP: Developed a unified control layer using capsule geometry to handle both self-collision and human-robot collision simultaneously. This approach balances geometric accuracy with computational efficiency.
Computational Benchmarking: Provided a comparative analysis of different collision primitives (spheres, bounding boxes, capsules) and hardware backends (CPU vs. GPU), demonstrating the superiority of the capsule model on GPU hardware for real-time performance.

4. Experimental Results

The framework was evaluated in a high-fidelity simulator using two primary scenarios:

Self-Collision (Cross-arm reach): The nominal retargeting caused the robot's arms to overlap. The CBF-QP successfully intervened, modifying the trajectory to maintain a safety margin while preserving the qualitative intent of the movement.
Human-Robot Collision (Side-by-side arm raise): When the human demonstrator moved their arm toward the robot, the system prevented contact by adjusting the robot's joint velocities, maintaining the safety boundary without stopping the motion entirely.

Performance Benchmarks (Table I):

Geometry Comparison: Capsules were compared against spheres (with varying subsampling) and bounding boxes.
- Constraint Count: Capsules and boxes had significantly fewer constraints (72) compared to sphere decompositions (705–1988), leading to faster solve times.
- GPU Performance: The capsule model on an NVIDIA RTX 5080 GPU achieved the highest control rate (33.084 Hz) with the lowest solve time (0.030s), outperforming other geometries and CPU implementations.
Conclusion: The capsule model offers the best trade-off between geometric fidelity and computational load for real-time CBF applications.

5. Significance and Future Work

Significance: This work demonstrates that safe, responsive human-robot imitation is achievable without expensive wearable sensors. The use of CBFs provides a mathematical guarantee of safety, while the capsule-based modeling ensures the system runs fast enough for real-time control.
Future Directions: The authors plan to transfer the framework to physical hardware, conduct quantitative evaluations of imitation fidelity, and explore more complex collision pair sets for richer interaction scenarios.

In summary, this paper presents a robust, mathematically grounded solution for safe humanoid teleoperation, proving that vision-based control can be made safe through efficient geometric modeling and optimization-based filtering.