HACTS: a Human-As-Copilot Teleoperation System for Robot Learning

1. Problem Statement

Current teleoperation systems used for robot learning (specifically for Vision-Language-Action models and Human-in-the-Loop Reinforcement Learning) suffer from a critical limitation: unilateral control.

• Lack of Feedback: In traditional systems (e.g., VR, joysticks, motion capture), the human operator sends commands to the robot but receives no real-time physical feedback regarding the robot's actual state.
• Intervention Risks: When a robot is performing an autonomous task and requires human intervention, the lack of synchronization between the robot's current state and the operator's input can lead to sudden, unintended movements or loss of control. This is analogous to a driver trying to steer an autonomous vehicle without a steering wheel that moves with the wheels.
• Cost and Complexity: Existing bilateral control systems that offer force/tactile feedback are often prohibitively expensive, require complex hardware, and are not easily scalable to common robotic platforms (like the UR5).

2. Methodology: HACTS System Design

The authors propose HACTS (Human-As-Copilot Teleoperation System), a low-cost, scalable solution that establishes bilateral, real-time joint synchronization between a robot arm and teleoperation hardware.

A. Hardware Design

• Kinematic Equivalence: The system uses a kinematically equivalent structure derived from Denavit–Hartenberg (DH) parameters, scaled to match the robot arm (e.g., UR5).
• Low-Cost Components: The system is built using 3D-printed PLA components and off-the-shelf servo motors (DYNAMIXEL XL430 and XL330), keeping the total hardware cost under $300 (specifically $247 for the UR5 setup).
• Bilateral Actuation:
  • Leader-to-Follower: The human moves the HACTS device, and the robot mimics the motion (standard teleoperation).
  • Follower-to-Leader (Reverse Sync): When the robot is acting autonomously, the robot's joint positions are fed back to the HACTS motors, physically moving the operator's device to match the robot's state. This allows the human to feel the robot's position and intervene seamlessly.
• Control Interface: An external foot pedal allows the operator to switch between autonomous mode and human intervention mode.

B. Software Architecture

• Bidirectional Data Flow: The software stack uses the DYNAMIXEL API for direct motor monitoring.
  • Forward: Processed joint values are sent to the robot controller.
  • Reverse: Robot position data undergoes offset compensation and is relayed back to the motor units to maintain synchronization.
• Integration: The system integrates with standard policy learning frameworks (e.g., ACT, Diffusion Policy, RLPD) to collect both successful demonstrations and action-correction data (trajectories where the human intervened to fix a failing autonomous attempt).

3. Key Contributions

1. Novel Bilateral System: Introduction of HACTS, a low-cost teleoperation system enabling real-time, bilateral joint synchronization, allowing humans to act as "copilots" rather than just remote controllers.
2. Enhanced Imitation Learning (IL): Demonstration that HACTS-collected action-correction data significantly improves the failure recovery capabilities and data efficiency of IL models (specifically Action Chunking Transformer and Diffusion Policy).
3. Facilitation of HITL RL: Validation that HACTS supports complex Human-in-the-Loop Reinforcement Learning (HITL RL), enabling seamless online learning where robots adapt to human corrections in real-time.

4. Experimental Results

The authors evaluated HACTS on a UR5 robot with a Robotiq gripper across three manipulation tasks (OpenBox, SteamBun, UprightMug) and one dynamic task (CloseBin).

A. Imitation Learning (IL) Experiments

The study compared models trained on:

• pre-ACT/DP: Trained on 50 random expert trajectories.
• full-ACT/DP: Trained on 100 expert trajectories.
• HACTS-ACT/DP: Trained on 50 expert trajectories + 50 human intervention/correction trajectories collected via HACTS.

Key Findings:

• Data Efficiency (Q1): In the Equal Amount Data Comparison (EADC), HACTS-ACT outperformed full-ACT (trained on double the data) in all tasks (e.g., 80% vs. 60% success on OpenBox). This proves intervention data is higher quality than raw demonstration data.
• Failure Correction (Q2): In In-Distribution Failure Correction (FCID), HACTS models significantly recovered from failures where baseline models failed (e.g., UprightMug: 70% success vs. 10% for baseline).
• Generalization (Q3): In Out-of-Distribution (OOD) static and dynamic scenarios, baseline models (pre/full) achieved 0% success. HACTS-trained models achieved significant success rates (e.g., 30–50% on OOD static tasks), demonstrating improved generalization to unseen object positions and dynamic disturbances.

B. Reinforcement Learning (RL) Experiments

The authors implemented RLPD-HACTS, combining HACTS with the RLPD off-policy RL algorithm.

• Setup: A "CloseBin" task where the robot must close a randomly placed trash bin.
• Process: The robot explores autonomously; when it struggles, the human intervenes via HACTS. The system uses a reward classifier to guide learning.
• Results:
  • Achieved 80% success rate after 45 minutes of online fine-tuning (following 10 mins offline pre-training).
  • Efficiency: The average human intervention length decreased from ~14 steps to <6 steps as the policy learned, indicating the robot became more autonomous and required only fine-grained corrections.
  • Compared to Behavior Cloning (BC) alone, the RL approach resulted in shorter episode lengths and better handling of sparse rewards.

5. Significance and Impact

• Accessibility: By reducing the cost of bilateral teleoperation to under $300 using 3D printing and standard servos, HACTS democratizes access to high-quality human-robot interaction data collection.
• Data Quality: It shifts the paradigm from collecting only "perfect" demonstrations to collecting "failure and correction" data, which is crucial for training robust policies that can recover from errors.
• Safety and Control: The physical synchronization (reverse sync) ensures that human intervention is safe and intuitive, preventing the "loss of control" issues common in unilateral teleoperation during autonomous failures.
• Future of Robot Learning: HACTS provides a foundational infrastructure for advancing Vision-Language-Action (VLA) models and Human-in-the-Loop RL, making it easier to train robots for complex, dynamic real-world tasks.