Technical Summary: BEAVR

Problem Statement

Robotic teleoperation systems currently face significant fragmentation, often being tied to specific hardware, proprietary data formats, or custom frameworks. This lack of standardization hinders reproducibility and slows progress in human-robot interaction (HRI) and policy learning. While recent Virtual Reality (VR)-based teleoperation systems (e.g., OpenTeach, Bunny VisionPro) offer immersive interfaces, they frequently suffer from steep learning curves, inaccessible documentation, non-standard data logging, and a lack of standardized bridges to learning-ready datasets. Furthermore, existing solutions often struggle to scale across heterogeneous robotic platforms (from 7-DoF arms to full-body humanoids) without incurring latency penalties or requiring specialized networking infrastructure.

Methodology

BEAVR (Bimanual, multi-Embodiment, Accessible, Virtual Reality Teleoperation System for Robots) is an open-source framework designed to unify real-time control, data recording, and policy learning. The system architecture is organized into three primary modules:

1. Teleoperation Module:

   • Input: Utilizes commodity Meta Quest 3S VR headsets to capture 24 OpenXR-compliant hand keypoints at 90 Hz.
   • Processing: Employs a modular, component-based design with three distinct processes communicating via ZMQ channels:
     • Detector: Captures raw keypoint data and session commands.
     • Operator: Transforms input data into robot-relevant commands. This includes constructing a stable hand frame using Gram-Schmidt orthogonalization, performing coordinate transformations (Y-up VR to Z-up robot), and applying scale adjustments based on the specific embodiment (e.g., hand vs. gripper).
     • Interface: Sends high-level actions to the robot's control system.
   • Control Solvers: Implements a multi-target Inverse Kinematics (IK) solver using Damped Least Squares (DLS) to optimize eight fingertip positions. It incorporates anti-collision routines and temporal smoothing (moving averages and quaternion SLERP blending) to ensure stable, low-jitter control.

2. Data Collection Module:

   • Integrates directly with the LeRobot framework to record synchronized streams of observations, actions, and metadata.
   • Exports data in the standardized LeRobot dataset schema (using Arrow/Parquet for tables and MP4 for video), enabling immediate downstream use for imitation learning.
   • Supports real-time RGB and RGB-D video streaming from commodity cameras.

3. Model Training Module:

   • Leverages LeRobot's training pipeline with an enhanced asynchronous "think–act" control loop.
   • A dedicated control loop streams actions at a fixed rate, while a separate inference thread computes policy outputs as resources allow, decoupling computation from real-time execution to maintain performance under high loads.

4. Communication Architecture:

   • Utilizes a zero-copy, thread-safe network API built on ZMQ to manage publisher-subscriber and request-reply patterns.
   • Designed to handle multiple robots and high-frequency command streams without data races or system crashes.

Key Contributions

The paper presents four primary contributions:

1. Hardware-Agnostic Interface: A VR teleoperation interface capable of scaling from 7-DoF tabletop arms (e.g., xArm7) to full-body humanoids (e.g., RX-1) and dexterous hands (e.g., LEAP Hand) using a single interface.
2. Low-Latency Streaming Architecture: A zero-copy design achieving end-to-end delays of ≤35 ms over commodity networks, with sub-millisecond jitter.
3. Dataset-Native Logger: A system that exports demonstrations directly into the LeRobot schema, facilitating immediate integration with modern imitation-learning frameworks.
4. Open Benchmark Suite: A release comprising six manipulation tasks, 50 expert demonstrations, and pretrained baseline policies (ACT, DiffusionPolicy, SmolVLA).

Experimental Results

The authors evaluated BEAVR across three dimensions: task performance, policy learning, and system scalability.

• Task Performance: The system was tested on six tasks (e.g., stacking blocks, flipping a cube, pouring, manipulating tape). In a 10-trial setup, BEAVR achieved success rates ranging from 60% (Tape task) to 100% (Flip cube, Pick and place). When compared to baselines (OpenTeach, Holo-Dex, Any-Teleop) on specific tasks like "Flip cube" and "Pick and place," BEAVR demonstrated competitive or superior success rates, though completion times varied based on hardware differences (e.g., using a LEAP hand vs. a standard gripper).
• Policy Learning: Using 50 expert demonstrations collected via BEAVR, the authors trained Action-Chunking Transformers (ACT), Action Diffusion, and SmolVLA. On a "Pickup Box" task, ACT achieved a 100% success rate (avg. time 9.16s), Diffusion achieved 80% (23.88s), and SmolVLA achieved 70% (33.84s). Human operators achieved 100% success (12.08s).
• System Performance & Scalability:
  • Latency: At 90 Hz, BEAVR achieved ~10 ms one-way latency with sub-millisecond jitter. At 30 Hz, latency was ~33 ms. These metrics outperform typical Wi-Fi-based teleoperation systems and are competitive with VR-based interfaces like Vicarios.
  • Frequency Stability: The system maintained control frequencies within 99.2%–99.4% of target rates across single-arm, bimanual, and multi-robot (up to 4 effectors) configurations.
  • Jitter: Jitter remained consistently low (<0.90 ms for arms, <0.22 ms for hands) even when scaling to multiple robots or increasing command rates, indicating that bottlenecks are localized to local control loops rather than the BEAVR architecture.
  • Cost: A complete setup (VR headset, robot, servos, printed parts) was estimated at approximately $1,080, highlighting the system's affordability.

Significance and Claims

The paper claims that BEAVR addresses critical barriers in robotics research by providing a unified, open-source platform that bridges teleoperation and data-driven policy learning. Its significance lies in:

• Democratization: Lowering the cost and technical barrier to entry for high-quality dexterous teleoperation and policy training.
• Standardization: Providing a standardized pipeline for data collection (LeRobot) that eliminates the need for custom data formatting and facilitates community-wide benchmarking.
• Scalability: Demonstrating that a single VR interface can control heterogeneous robots (arms, hands, humanoids) without incurring significant latency penalties or requiring specialized networking infrastructure.
• Modularity: Offering a flexible architecture where new embodiments can be integrated by updating transformation matrices and solver targets without reengineering the entire pipeline.

The authors position BEAVR not merely as a teleoperation tool, but as a foundational framework for advancing research in dexterous manipulation and multi-robot coordination through accessible, reproducible, and scalable data collection.