Technical Summary: User Evaluation of Telepresence Through a Virtual Environment Enhanced VR Framework with Real World Constraints

Problem Statement

The integration of Virtual Reality (VR) and robotics for telepresence and teleoperation offers significant potential for applications ranging from hazardous exploration to search and rescue. However, existing solutions often rely on expensive, inaccessible hardware (e.g., high-end 360-degree cameras, advanced haptic devices) or suffer from critical usability issues. Specifically, direct mapping of camera footage to a Head-Mounted Display (HMD) can induce simulator sickness due to latency, frame rate mismatches, or lag. Furthermore, traditional control schemas using keyboards, mice, or joysticks often fail to provide natural locomotion, leading to user disorientation and reduced immersion. Users lack the vestibular and proprioceptive cues associated with physically turning to look behind them, which are essential for a strong sense of spatial presence. Additionally, limited fields of view (FOV) in traditional setups exacerbate disorientation. The challenge is to develop a telepresence solution that utilizes standard, moderately priced industry hardware while maintaining accurate, immersive controls and minimizing simulator sickness.

Methodology

This study proposed a teleoperation framework that integrates a robot, a virtual environment (VE), and VR equipment to mimic the real world while mitigating hardware limitations.

• Hardware Configuration: The system utilized a ROSBot 2R robot coupled with a virtual machine running ROS 2 (Ubuntu-Humble). The robot was equipped with a built-in camera (operating at a low resolution of 320x240 and 15 FPS due to driver constraints) and a LIDAR scanner. The user interface consisted of a standard HMD and VR controllers.
• Software Architecture: The system was developed using Unity 3D and Visual Studio, incorporating the ROS TCP Endpoint for communication. A virtual environment was generated in real-time based on the robot's LIDAR data to represent the remote location.
• Core Mechanism: To address latency and immersion issues, the system employed an intermediate digital environment. Instead of directly projecting the robot's video feed to the HMD (which would expose the user to hardware lag), the user viewed a virtual control room.
  • Visual Latency Cues: A "floating screen" within the VE displayed the robot's camera feed. This screen rotated to match the robot's actual orientation in the real world. By observing the orientation of this floating screen relative to the user's own viewpoint, users could visually perceive control latency without it affecting the refresh rate of the surrounding digital environment.
  • Natural Locomotion: The system mapped the user's physical head rotation in the VE to the robot's rotation in the real world. If a user rotated 180 degrees in the VE, the robot rotated 180 degrees remotely.
  • Environmental Aids: The VE included 3D objects representing barriers and obstacles derived from the LIDAR scan, providing spatial context and navigation aids.
• User Study: A validation study was conducted with 15 volunteers (14 valid results after one technical failure). Participants navigated a physical maze containing four hidden items within a three-minute limit using the system.
• Data Collection: Post-test, participants completed a survey including Likert scale questions regarding immersion, naturalness of controls, responsiveness, and the utility of the virtual environment. A numerical question assessed the sense of presence, and a yes/no question with free-text comments gauged interest in future enhancements.

Key Contributions

1. Latency-Mitigated Telepresence Framework: The paper introduces a method to decouple the user's visual experience from the robot's hardware latency. By using a virtual control room with a floating screen to visualize the robot's actual state, the system allows the user to perceive lag without suffering the simulator sickness associated with direct, laggy video feeds on an HMD.
2. Natural Control Schema: The study demonstrates a control mechanism where physical head rotation in VR directly translates to robot rotation, restoring the natural vestibular and proprioceptive cues often lost in keyboard/mouse-based teleoperation.
3. Low-Cost Hardware Viability: The research validates that an immersive telepresence experience can be achieved using "less-than-ideal" hardware (low-resolution, low-FPS cameras and standard consumer VR) by compensating through software-driven environmental cues and latency visualization.
4. Empirical Validation: The study provides quantitative and qualitative data from a user test involving maze navigation, categorizing user feedback into five distinct areas: immersion, virtual environment acceptability, navigation aids, hardware limitations, and user desire for future development.

Results

The user study yielded the following findings across five categories:

• User Immersion: The system successfully fostered a sense of presence. The mean acceptance rate for feeling "physically there" was 78.6%, with 28.6% of users reporting they felt physically present. Regarding the overall experience as immersive, the mean acceptance was 87.5%.
• Virtual Environment Acceptability: Users generally accepted the virtual environment, with a mean acceptance of 82.1%. Most users rated their liking of the environment as "definitely" or "probably."
• Navigation Aids: Feedback on whether the virtual environment aided navigation was mixed but positive, with a mean acceptance of 73.2%. While 42.9% of users felt it "definitely" helped, others were less certain.
• Hardware Limitations & Performance:
  • Controls: Users rated the naturalness of controls positively (mean 66.1%), with 57.1% rating them as "great."
  • Responsiveness: Perceptions of responsiveness were mixed (mean 51.8%), with 28.6% finding the system "not very responsive," reflecting the underlying hardware constraints.
  • Collisions: 78.6% of users experienced at least one collision, averaging 1.78 collisions per user.
  • Getting Lost: 71.4% of users did not get lost during the test, suggesting the system provided sufficient orientation cues despite the hardware limitations.
  • Task Completion: The average number of items found was 1.93 out of four, indicating that while the system was navigable, the low-fidelity hardware and short time limit presented challenges.
• User Sentiment: 85.7% of users expressed a desire for future enhancements to enable operations in faraway remote locations. Word cloud analysis of free-text comments highlighted positive terms ("good," "perfect," "cool") alongside specific requests for camera improvements and concerns regarding latency ("laggy," "issue").

Significance and Claims

The paper claims that its primary objective was achieved: creating a VR system that enables telepresence using hardware readily available to the average user. The study concludes that:

• Feasibility of Low-Cost Solutions: It is possible to create an immersive teleoperation experience using less-than-ideal hardware if the system design carefully accounts for its limitations through visual cues and latency visualization.
• Reduction of Simulator Sickness: The intermediate digital environment successfully reduced the risk of simulator sickness by isolating the user from the raw latency of the robot's video feed.
• User Engagement: Despite hardware constraints, users reported high levels of immersion and acceptance of the virtual environment.
• Future Potential: The study validates the concept of using VR to enhance HRI, suggesting that further refinements (such as improved connectivity and additional visual cues) could further improve performance and expand the utility of such systems for remote exploration.

The authors maintain a modest tone, acknowledging that while the system was successful, the mixed results on responsiveness and collisions highlight the ongoing trade-offs when utilizing lower-specification hardware. The work serves as a proof-of-concept that standard industry hardware, when paired with an enhanced VR framework, can provide a viable path toward accessible telepresence.