💻 computer science

A User Study on the Suitability of Teleoperation Interfaces for Primitive Manipulation Tasks

This user study with 23 participants demonstrates that the suitability of teleoperation interfaces for robotic arm tasks depends on the specific motion primitives involved, as VR controllers proved superior for pushing buttons while 3D mice were more effective and less demanding for rotating knobs.

Original authors: Jun Aoki, Shunki Itadera

Published 2026-03-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jun Aoki, Shunki Itadera

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot arm to do chores for you. You can't just tell it "go do it"; you have to hold a remote control and guide its hand, much like playing a video game. But here's the big question: Is there one "perfect" remote control for every job, or does the best tool depend on what you're actually trying to do?

This paper is like a taste test for robot controllers. The researchers wanted to find out if a specific type of controller is better for specific types of movements. They tested two popular "remotes" on two very different, simple jobs.

The Two Contenders (The Remotes)

The 3D Mouse: Think of this like a high-tech, tiny joystick you keep on your desk. You push it forward, pull it back, or twist it with your fingers. It's like driving a car with a steering wheel; you control the movement by how much you deflect the stick.
The VR Controller: This is the wand you hold in your hand when playing virtual reality games. It tracks exactly where your hand is in the air. If you move your hand to the left, the robot moves to the left. It's like having a "ghost hand" that the robot copies.

The Two Jobs (The Challenges)

The researchers gave the robot two simple tasks to do over and over again for three minutes:

The "Button Masher" (Pushing): Imagine a panel with a row of buttons. The robot has to reach out and push them down, one by one, in a circle. This is mostly about moving forward and backward (translational motion).
The "Knob Turner" (Rotating): Imagine a panel with a row of dials (like an old radio volume knob). The robot has to grab them and spin them 180 degrees. This is mostly about twisting and turning (rotational motion).

The Results: It's All About the Match

The study found that there is no single "best" controller. The winner changed depending on the job, just like how a hammer is great for nails but terrible for screwing in a screw.

🏆 The Button Masher: The VR Controller Wins

When the task was to push buttons, the VR controller was the clear winner.

Why? Pushing a button is like reaching out to high-five someone. With the VR controller, you just naturally move your hand forward, and the robot does the same. It feels intuitive and fast.
The 3D Mouse struggle: Using the 3D mouse for this felt like trying to high-five someone by pushing a tiny joystick. It required more mental effort to translate "push the stick" into "move the hand," making the users feel frustrated and slower.

🏆 The Knob Turner: The 3D Mouse Wins

When the task was to spin knobs, the 3D mouse took the crown.

Why? Spinning a knob requires a steady, continuous twist. With the 3D mouse, you can gently tilt the stick and hold it there, creating a smooth, endless rotation. It's like turning a steering wheel with a steady hand.
The VR controller struggle: Using the VR controller for this was exhausting. To spin a knob, you had to physically twist your entire wrist over and over again. It was like trying to unscrew a jar lid by spinning your whole arm in circles. It got tiring fast, and people often lost their balance or drifted off course.

The "Workload" Scorecard

The researchers also asked the participants how tired and stressed they felt (using a survey called NASA-TLX).

For the Buttons: The VR controller was faster, but it felt a bit more physically tiring because you were moving your arm more. However, because it was so much faster and easier to aim, the users didn't mind the extra effort.
For the Knobs: The 3D mouse was the "chill" option. It required less physical movement and felt less frustrating. The VR controller made people feel physically drained because of all the wrist twisting.

The Big Takeaway

The main lesson from this study is: Don't use a sledgehammer to crack a nut, and don't use a screwdriver to hit a nail.

If you are designing a system to control a robot:

If the robot needs to reach, grab, and place things (like picking up a cup), a VR controller (hand-tracking) is probably your best friend.
If the robot needs to turn, twist, or adjust things (like tightening a bolt or turning a dial), a 3D mouse (joystick style) is likely the better choice.

The researchers concluded that we need to stop looking for one "magic" remote control for everything. Instead, we should pick the tool that matches the specific "dance moves" the robot needs to perform.

1. Problem Statement

Teleoperation is critical for assistive manipulation, remote inspection, and robot learning from demonstration. However, the design of teleoperation interfaces often lacks systematic evidence regarding how specific interface characteristics interact with different types of manipulation tasks.

The Gap: Prior research tends to evaluate single interfaces on narrow task families or assumes a "one-size-fits-all" approach. There is limited understanding of which interface is optimal for specific motion primitives (e.g., translational pushing vs. rotational turning).
The Goal: To determine if the suitability of a teleoperation interface depends on the dominant motion primitive of the task being performed, specifically comparing 3D mice (compact, rate-controlled) against VR controllers (pose-tracked, hand-centered).

2. Methodology

Apparatus and Setup

Robot: A 6-DoF (Degrees of Freedom) Kinova Gen3 Lite robotic arm with a gripper.
Interfaces:
1. 3D Mouse (3Dconnexion): Provides 6-DoF rate control via a compact desktop device.
2. VR Controller (Meta Quest Pro): Tracks the operator's dominant hand pose for gesture-based control.
3. Baseline: A GUI interface was used only for familiarization, not in the final comparative analysis.
Task Platform: A circular 3D-printed base containing tactile push-buttons and rotary knobs arranged at equal angular intervals.
Control Mapping: Both interfaces mapped 3 translational and 3 rotational velocity components to the robot's end-effector frame. Gains were tuned to ensure comparable speeds across interfaces. Gripper open/close was controlled via keyboard for all conditions.

Participants

Sample Size: $N = 23$ (ages 20–65; 10 female).
Exclusions: One participant excluded due to scheduling; two excluded from the rotation task analysis due to protocol violations (rubbing knobs instead of rotating), resulting in $N=20$ for rotation analysis.

Experimental Design

The study utilized a within-subjects design with two sessions:

Session 1 (Familiarization): Participants practiced with all three interfaces (GUI, 3D mouse, VR) to learn the tasks.
Session 2 (Data Collection): Participants performed two specific tasks using only the 3D mouse and VR controller. The order of interfaces was counterbalanced.
- Task A (Pushing): Push buttons in a clockwise circle as many times as possible in 3 minutes.
- Task B (Rotation): Rotate knobs 180° in a clockwise circle as many times as possible in 3 minutes.

Metrics:
- Performance: Count of successful operations (fully depressed button or 180° rotation).
- Workload: Subjective workload measured via the NASA-TLX questionnaire (weighted overall score and subscales: Mental, Physical, Temporal Demand, Performance, Effort, Frustration).

3. Key Results

Performance (Success Scores)

Pushing Task: The VR controller significantly outperformed the 3D mouse.
- Stats: $t(22) = -4.58, p < .001$ , Cohen's $d = -0.96$ .
- Participants completed more button pushes with the VR controller.
Rotation Task: The 3D mouse significantly outperformed the VR controller.
- Stats: $t(19) = 3.24, p = .006$ , Cohen's $d = 0.69$ .
- Participants completed more 180° knob rotations with the 3D mouse.

Subjective Workload (NASA-TLX)

Pushing Task: No significant difference in overall workload between interfaces ( $p = .42$ $p = .42$ ).
- Nuance: The VR controller had higher Physical Demand, but the 3D mouse had higher Frustration and lower Perceived Performance. These opposing factors canceled out in the weighted total score.
Rotation Task: The 3D mouse resulted in significantly lower overall workload ( $p = .007$ $p = .007$ ).
- The VR controller imposed significantly higher Physical Demand in both tasks, attributed to the fatigue of repeated wrist rotations required for knob turning.

Subscale Analysis

Physical Demand: Consistently higher for the VR controller across both tasks.
Frustration/Performance: In the pushing task, users found the 3D mouse more frustrating and felt they performed worse compared to the VR controller.

4. Key Contributions

Task-Interface Specificity: The study provides empirical evidence that interface suitability is not universal but is highly dependent on the dominant motion primitive of the task.
- Translational/Discrete Tasks (Pushing): Pose-tracked VR controllers are superior due to natural reaching and wrist movements.
- Rotational/Sustained Tasks (Turning): Rate-controlled 3D mice are superior due to stable, continuous control without physical fatigue.
Workload Trade-off Analysis: The paper highlights that relying solely on "overall" workload scores can be misleading. In the pushing task, high physical demand (VR) was offset by high frustration (3D mouse), masking the distinct advantages of each interface.
Design Guidelines: It offers concrete recommendations for interface selection in practical teleoperation deployments based on the specific motion requirements of the operation.

5. Significance and Implications

Design Strategy: Teleoperation systems should be task-aware. Designers should select interfaces based on whether the primary motion is translational (favoring VR/pose-tracking) or rotational (favoring 3D mice/rate control).
Future Systems: The findings motivate the development of hybrid or adaptive interfaces that can switch modalities or combine inputs to minimize operator fatigue and maximize efficiency depending on the current sub-task.
Evaluation Standards: Future user studies must move beyond single-interface evaluations and include multi-dimensional workload analysis (subscale breakdown) rather than just success rates or aggregate workload scores.

Conclusion: The paper demonstrates that there is no single "best" teleoperation interface. Instead, the VR controller is optimal for tasks requiring rapid, discrete translational movements (like pushing buttons), while the 3D mouse is optimal for tasks requiring sustained, precise rotational control (like turning knobs).