Spatiotemporal Context-dependent Personalized Movement Compensation in Delayed Telemanipulation

Technical Summary: Spatiotemporal Context-dependent Personalized Movement Compensation in Delayed Telemanipulation

Problem Statement
Communication delay remains a critical barrier in teleoperated surgery and robotics, disrupting visuomotor coordination and degrading task precision. While delays under 330 ms are generally manageable, latencies exceeding 400 ms significantly impair surgeon performance and compromise perceived safety. Existing countermeasures, such as motion scaling (sub-unity gains), often rely on uniform parameters that fail to account for individual operator variability or the specific context of the task (e.g., movement direction, distance, and delay magnitude). Current literature suggests that delay sensitivity shifts based on task geometry and operator adaptation, yet the factors shaping these variations remain understudied.

Methodology
The authors conducted a two-session study with twenty healthy participants to evaluate a human-centered, personalized motion compensation framework.

• Experimental Design:
  • Session 1 (SimOnly): Participants performed teleoperated reaching tasks in a virtual environment (CoppeliaSim) using a da Vinci Research Kit (dVRK) digital twin and a haptic interface. The study utilized a within-subjects design crossing four factors: Assistance (Personalized vs. Unassisted), Delay (100, 250, 400 ms), Distance (5, 10, 15 mm), and Direction (Lateral, Longitudinal, Vertical).
  • Session 2 (Sim2Real): Participants returned to perform a modified peg-transfer task on a physical dVRK system. This session tested the transfer of gains derived from simulation to real hardware. It included Unassisted, Generalized (population-averaged), and Personalized assistance conditions.
• Personalization Strategy: The core innovation is a method to fit personalized scaling parameters ($G$) for each participant based on specific combinations of delay ($\delta$), distance ($d$), and direction ($\theta$). The gain is calculated as the ratio of peak reach distance in a no-delay baseline to that in a delayed condition, effectively creating a "gain tensor" to minimize overshoot.
• Metrics: Performance was assessed using Initial Reaching Error (overshoot/undershoot), Endpoint Error, Movement Smoothness (Spectral Arc Length), Economy of Motion (Path Straightness Index), and a Combined Error-Time (nCET) metric. Subjective workload was measured via NASA-TLX.
• Analysis: Linear mixed-effects models were used to analyze main effects and interactions, with data transformed via Box-Cox to ensure normality.

Key Contributions
The paper establishes three primary contributions:

1. Context-Dependent Personalization: It demonstrates that personalized motion scaling, conditioned on delay, distance, and direction, significantly improves telemanipulation performance compared to unassisted trials in simulation, with transferable benefits to real hardware for specific metrics.
2. Accuracy without Time Penalty: Personalized scaling improves spatial accuracy (reducing overshoot) without a reliable increase in movement time across most conditions, thereby improving the speed-accuracy tradeoff, though accuracy gains were inconsistent for endpoint error in real-world scenarios.
3. Simulation-to-Real Transfer: The study validates that personalized gains derived from simulation data transfer to real-world manipulation tasks on physical hardware, although with reduced effectiveness and specific metric limitations compared to the simulation environment.

Results

• Simulation (SimOnly): Personalized assistance consistently reduced overshoot across all delays, distances, and directions. However, the benefit for endpoint error did not increase with delay; instead, assistance was more beneficial and consistent specifically in the inward reaching direction relative to no assistance. The benefits were most pronounced at longer delays (400 ms) for overshoot, yielding performance gains of 20–25% in key metrics. Personalization also significantly improved movement smoothness and economy. Notably, accuracy benefits were direction-dependent, with inward reaching (toward the body midline) showing greater improvement than outward reaching.
• Real-World (Sim2Real): The transfer of personalized gains to the physical robot confirmed improvements in movement smoothness and economy. However, the benefits were less consistent than in simulation. Specifically, personalized assistance did not result in a consistent reduction in endpoint error in the real task and, in some cases (e.g., at 250 ms delay), decreased accuracy compared to unassisted trials. Furthermore, while personalized assistance improved initial reaching error in simulation, this benefit was eliminated in the real-world transfer.
• Personalized vs. Generalized: While population-averaged (generic) gains provided a solid baseline, personalized gains offered distinct advantages in specific contexts, particularly for inward reaching at short distances under moderate delay. In the Sim2Real session, personalized assistance significantly reduced subjective workload compared to generic gains, even when overall workload differences between assisted and unassisted trials were negligible.

Significance and Claims
The authors claim that this work highlights the potential of personalized scaling as a foundation for more adaptive teleoperation frameworks. The results suggest that delay compensation should not be viewed as a static calibration but as a dynamic, context-aware process. By integrating population-level initialization with adaptive refinement based on user state and task context (direction and distance), teleoperation systems can achieve both reliability and flexibility. The paper concludes that while generic gains are effective starting points for new users, personalization is crucial for optimizing performance in scenarios where human feedforward control is needed but uncertain, ultimately aiming to improve the safety and precision of teleoperated procedures.