Mixed Reality Tele-Ultrasound over 750 km: A Feasibility Study

Technical Summary: Mixed Reality Tele-Ultrasound over 750 km

Problem Statement
Ultrasound (US) is a critical, non-invasive, and portable diagnostic tool, yet remote and Indigenous communities often lack access to trained sonographers. Current solutions present significant trade-offs: robotic tele-ultrasound offers precision but suffers from high costs, large footprints, and complex maintenance; tele-mentored US is cheaper but relies on local expertise and lacks precise transducer control; and autonomous robotic US faces challenges in reasoning and safety. To address the need for a low-cost, flexible, and precise solution, this study evaluates "Human Teleoperation," a mixed reality (MR) system where a remote expert guides a novice "follower" to perform scans. While previous iterations of this concept existed, their feasibility over large distances (introducing latency) and for the precise motions required in abdominal imaging remained unexplored.

Methodology
The study conducted a feasibility test involving 11 remote abdominal ultrasound scans between Skidegate, Haida Gwaii, and Vancouver, Canada, separated by 754 km.

• System Architecture:

  • Follower Side (Remote): A novice wears a Microsoft HoloLens 2 headset, which renders a dynamic virtual US transducer. The follower physically aligns a real US transducer with this hologram. The system tracks the physical transducer's pose using infrared (IR) reflective markers and measures applied forces via a differential magnetic force sensor shell.
  • Expert Side (Local): A remote expert sonographer controls the virtual transducer using a 6-DOF haptic device (Touch X) with a 3D-printed transducer handle. The expert views the follower's point-of-view video and the real-time US image feed (via Windows Phone Link to a Philips Lumify app) and provides verbal guidance.
  • Communication: The system utilizes WebRTC for peer-to-peer communication, transmitting pose, force, and video data.

• Haptic Feedback Model:
  To provide stable haptic feedback despite communication latency and follower lag, the system models the patient's abdominal surface as an ellipsoid. This model is calibrated using four specific points: the xiphoid process, the extreme left and right sides of the patient, and the bed level. The system assumes the ellipsoid axes align with the patient's anatomical axes and that the ellipsoid is tangent to the bed. Forces are rendered to the expert based on the penetration depth of the virtual transducer into this ellipsoid, calculated using a spring-damper model.

• Study Protocol:
  Two expert sonographers guided 10 novice followers (ranging from 20 to 60+ years old) to acquire five specific target images: proximal aorta (AP diameter), longitudinal IVC, left liver lobe (long axis and transverse), and right portal vein (transverse).

  • Data Collection: Image quality was scored by two radiologists on a 6-point scale. Follower alignment error (Root Mean Squared Error or RMSE) was calculated by comparing physical and virtual transducer poses. Participants completed NASA TLX task load surveys and modified System Usability Scale (SUS) questionnaires.

Key Contributions

1. Long-Distance Feasibility: The study demonstrates that human teleoperation can successfully facilitate remote abdominal ultrasound examinations over a 754 km distance, overcoming challenges related to network latency.
2. Novice Utilization: It validates that individuals with no prior US experience can act as effective "cognitive robots," following remote expert guidance to acquire diagnostic-quality images.
3. Novel Haptic Rendering: The paper introduces and tests an ellipsoid-based patient model for haptic feedback. This approach provides the remote expert with a stable, intuitive representation of the patient's surface, mitigating the instability caused by direct force feedback delays.
4. Comprehensive Evaluation: The study provides a multi-faceted assessment including radiologist-graded image quality, quantitative alignment error analysis, and subjective usability/task load metrics from both novices and experts.

Results

• Image Quality: Out of 55 target images attempted, 48 were successfully captured. Of the captured images, 91.7% received a score of ≥3 (suboptimal to good) from both radiologists, with 31.3% rated as "good" (score 5). The mean quality score for captured images was 4.28 ± 0.95.
• Task Load and Usability: Novice followers reported low mental demand (23.3/100), effort (18.9/100), and frustration (16.5/100) on the NASA TLX scale, all below reference values for healthcare and virtual reality tasks. Usability scores were uniformly positive, with all 10 followers agreeing the system was easy to use and that most people could learn it quickly.
• Alignment Error: The mean position RMSE was 31.94 ± 10.17 mm, and the mean orientation RMSE was 11.39 ± 3.55°. However, when centered (removing constant offsets), the RMSE dropped significantly to 10.06 mm and 5.81°, respectively.
• Correlation: No statistically significant correlation was found between the followers' alignment errors (position or orientation) and the resulting image quality scores. Notably, the scan with the highest position error (Scan 8) received the highest image quality scores, suggesting that patient body habitus and the expert's ability to infer true transducer position from the image are more critical than absolute follower alignment precision.

Significance and Claims
The paper claims that human teleoperation is a feasible and effective method for enabling remote abdominal ultrasound imaging, even across large distances and with novice operators. The study highlights that the system allows expert sonographers to perform remote scans with high performance, achieving diagnostic-quality images in 92% of successful acquisitions. The authors emphasize that the lack of correlation between alignment error and image quality suggests the system is robust to minor tracking inaccuracies, provided the expert can interpret the visual feedback.

The authors conclude that this approach offers a promising alternative to robotic and tele-mentored systems, particularly for remote communities where cost, portability, and the lack of local expertise are barriers. They modestly note that while the system is feasible, future work is required to compare it directly with robotic and tele-mentored approaches, expand testing to other anatomical regions (e.g., renal, full aorta exams), and explore the integration of AI for autonomous guidance using the rich data collected.