💻 computer science

Measurement and Potential Field-Based Patient Modeling for Model-Mediated Tele-ultrasound

This paper presents a model-mediated tele-ultrasound system that enhances force and torque feedback accuracy in high-delay environments by dynamically updating a patient's internal potential field model using real-time measurements, resulting in significant reductions in force magnitude and vector angle errors compared to methods relying solely on Laplace's equation.

Original authors: Ryan S. Yeung, David G. Black, Septimiu E. Salcudean

Published 2026-09-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryan S. Yeung, David G. Black, Septimiu E. Salcudean

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of remote medicine, one of the most persistent challenges is the loss of physical connection. When a specialist guides a procedure from thousands of miles away, they are often blind to the subtle tactile cues that define their craft. For ultrasound imaging, this is particularly critical. The quality of the image depends entirely on how firmly the technician presses the probe against the patient's skin. Press too lightly, and the image is grainy; press too hard, and the image distorts or causes discomfort. In a traditional setting, a sonographer feels this resistance through their hand, adjusting their pressure instinctively. But in a teleoperated system, where a robot or a human follower holds the probe at a distant site, that direct line of touch is severed. If the connection is delayed by the time it takes for data to travel across a network, sending force feedback back to the operator can cause the system to become unstable or jittery, making precise control impossible.

To solve this, engineers have turned to a concept called model-mediated teleoperation. Instead of trying to send the raw, delayed force signal from the remote site, the local computer builds a digital map of the patient's body. As the operator moves their controller, the computer calculates what the force should feel like based on this map, providing immediate feedback without waiting for a signal from the other side. However, a generic map is not enough. A patient's body is not a uniform block of rubber; it has varying stiffness, curves, and textures. If the digital model assumes the body is perfectly smooth and uniform, the feedback will be wrong, and the operator will make mistakes. The core question becomes: how can we make this digital map smart enough to learn the specific physical properties of the patient it is representing, using the very forces and movements that occur during the scan?

A team of researchers at the University of British Columbia has developed a method to answer this question, creating a system that updates its internal model of a patient in real-time using actual measurements. Their work focuses on abdominal ultrasound, a common but difficult exam to perform remotely. The process begins with a simple scan of the patient's torso using a depth camera, similar to those found in modern virtual reality headsets. This camera captures a cloud of points that outlines the shape of the body. The researchers then convert this raw data into a structured, three-dimensional grid, essentially turning the patient's surface into a digital volume made of tiny cubes. Inside this volume, they assign a value to each cube that represents how much resistance it would offer to the ultrasound probe.

Initially, this resistance map is just a mathematical guess based on the shape of the body, assuming a smooth, predictable increase in stiffness as the probe goes deeper. But the researchers realized that a static guess is insufficient. To make the model truly accurate, they introduced a way to correct it using real data. As the probe moves over the patient, sensors measure the exact force and twisting motion applied at every moment. The system then takes these measurements and adjusts the internal values of the digital grid. It is a process of refinement: the computer compares what it predicted the force should be against what the sensors actually measured, and then it tweaks the digital map to align the two. This allows the model to learn that one part of the patient's abdomen might be softer than another, or that a specific area offers more resistance, without needing to know the patient's anatomy in advance.

The team tested this approach on four volunteer subjects, comparing the performance of their new, data-updated model against a traditional model that relied only on the initial shape guess. The results were striking. When the system used the updated model, the error in the magnitude of the force felt by the operator dropped significantly. On average, the difference between the predicted force and the actual force was reduced by 7.42 Newtons, a substantial improvement in precision. Furthermore, the direction of the force felt by the operator became much more accurate, with the angle of error decreasing by an average of 3.71 degrees. Perhaps most notably, the accuracy of the torque, or the twisting force, improved dramatically. The error in the direction of the twist was reduced by an average of 64.0 degrees, meaning the operator could feel the correct rotational resistance of the tissue much more clearly.

While the system excelled at predicting the direction and strength of the push, it did not perfectly capture the magnitude of the twisting force, a limitation the researchers attribute to the complex way the probe is held and the slight offsets that occur when force is applied. Nevertheless, the study demonstrates that by feeding real-time measurements back into the digital model, the system can create a much more transparent and realistic experience for the remote operator. The method is fast enough to be calculated in milliseconds, suggesting it could eventually run in real-time during an actual scan. This advancement moves the field closer to a future where a specialist in one city can guide a scan in a remote village with the same tactile confidence as if they were standing right beside the patient, ensuring that distance no longer compromises the quality of medical care.

Technical Summary: Measurement and Potential Field-Based Patient Modelling for Model-Mediated Tele-ultrasound

Problem Statement
Teleoperated ultrasound (US) offers a solution for providing diagnostic imaging to remote communities lacking local expertise. However, effective teleoperation requires accurate force feedback to allow sonographers to apply appropriate probe contact forces, which are critical for image quality. Direct force feedback is often impractical due to large communication time delays. While Model-Mediated Teleoperation (MMT) addresses this by generating feedback from a local model of the remote environment, existing methods face limitations. Prior approaches using point clouds or simple geometric models (e.g., ellipsoids) often fail to capture the spatial variations in the physical properties (stiffness) of the patient's torso. Furthermore, previous methods that estimated stiffness parameters (e.g., via recursive least squares) were often limited to one degree of freedom (1-DOF) or required continuous transmission of parameters to the operator, failing to account for complex spatial variations.

Methodology
The authors propose a model-mediated teleoperation method that updates an internal potential field model of the patient using measured positions, forces, and torques. The methodology consists of four main stages:

Patient Surface Extraction: A Time-of-Flight (TOF) depth camera captures a raw point cloud of the patient's torso. This is converted into a structured point cloud in cylindrical coordinates using a contour extraction algorithm and a Kalman filter to estimate surface radii. This structured format allows for efficient transmission as a matrix of estimated radii.
Force and Torque Rendering: The patient is modeled as a static voxelized volume (voxmap) where each voxel contains a potential field value ( $p$ ). The ultrasound transducer is represented as a dynamic "point shell" (a set of points with inward-pointing normal vectors). Forces and torques are rendered based on the interpenetration of the point shell with the voxels. The total force and torque are calculated by summing contributions from overlapping voxels, weighted by the potential field values and the geometry of the penetration.
Stiffness Estimation via Potential Field Update: The core innovation is the method for determining the potential field values ( $p$ ). Initially, the field is modeled using Laplace's equation ( $\nabla^2 p = 0$ ) with Dirichlet boundary conditions to simulate a smooth stiffness gradient. To incorporate real-world data, the authors formulate a convex quadratic optimization problem that minimizes the error between the Laplace solution and the measured forces and torques. The objective function combines the spatial Laplace operator with regularization terms for measured force ( $f$ ) and torque ( $\tau$ ) data:
$\min_p \|Lp - b\|^2 + \lambda\|Np - f\|^2 + \lambda\|Wp - \tau\|^2$
This approach solves for a unique potential field that preserves the smoothness of the Laplace model while adjusting to local variations indicated by the measurements.
Experimental Setup: The system was evaluated on four volunteer subjects. A Philips Lumify transducer instrumented with a 6-DOF force sensor shell and IR markers was used to measure pose and contact forces. Scans included vertical presses, axial sweeps, and oblique presses. Data was split into "known" trajectories (used to generate the force-informed model) and "unknown" trajectories (used for evaluation).

Key Contributions

Hybrid Potential Field Model: The paper introduces a method to update a volumetric potential field model (initialized by Laplace's equation) with measured contact forces and torques, enabling the capture of local stiffness variations across the patient's torso.
Efficient Data Structure: The conversion of raw point clouds into a structured cylindrical representation facilitates efficient transmission of the patient model in teleoperation scenarios.
Convex Optimization for Stiffness Mapping: The authors demonstrate a convex quadratic formulation that integrates spatial smoothness constraints with empirical force/torque measurements to estimate a stiffness map without requiring iterative parameter transmission.

Results
The study evaluated the accuracy of rendered forces and torques by comparing the proposed "force-informed" model against a baseline model using only Laplace's equation.

Force Accuracy: The addition of measurements significantly reduced errors. On known trajectories, the force magnitude Root Mean Squared Error (RMSE) decreased by an average of 7.42 N, and the force vector angle error decreased by 3.71°. On unknown trajectories, the force RMSE decreased by 6.70 N.
Torque Accuracy: The force-informed model produced a significant reduction in torque vector angle error (average reduction of 64.0° on known trajectories and 58.5° on unknown trajectories). However, the magnitude of torque errors remained largely unchanged or slightly increased, which the authors attribute to lateral torque artifacts caused by slight offsets in the applied axial force relative to the sensor origin.
Robustness: The force-informed model was found to be less sensitive to the selection of boundary conditions compared to the Laplace-only model.
Computation Time: The offline computation time for a single contact was approximately 2.74 ms, suggesting the potential for real-time implementation.

Significance and Claims
The paper claims that incorporating measured forces and torques into the internal potential field model significantly improves the transparency and accuracy of model-mediated tele-ultrasound. Specifically, the method allows for a more accurate rendering of the stiffness map, accounting for local variations that simple geometric or Laplace-only models miss.

The authors modestly note that while the method improves force and torque direction accuracy, further research is required to determine if these improvements translate to a noticeable benefit for sonographers during actual teleoperated scans. They acknowledge current limitations, including the exclusion of friction ("drag") forces and the focus on quasi-static stiffness estimation without damping. Future work is proposed to integrate the full system (including pose tracking and stiffness estimation) into the human teleoperation workflow and to investigate damping estimation using transducer velocity. The paper asserts that the methodology is applicable to both human teleoperation (using mixed-reality headsets) and robotic teleoperation.

Technical Summary: Measurement and Potential Field-Based Patient Modelling for Model-Mediated Tele-ultrasound

More like this