📄 medicine

GHOST: Automatic multimodal registration of high-resolution impedance manometry and video-fluoroscopy swallow studies

The paper introduces GHOST, a geometry-driven deep learning framework that achieves robust, fully automated cross-modal registration of high-resolution impedance manometry and videofluoroscopy swallow studies, significantly improving sensor identification accuracy and stability across varying image qualities to facilitate clinical diagnosis and rehabilitation planning for oropharyngeal dysphagia.

Original authors: Manuel Maria Loureiro da Rocha, Dionne S. Brandsma, Lisette van der Molen, Maarten J. A. van Alphen, Michiel W. M. van den Brekel, Françoise J. Siepel

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Manuel Maria Loureiro da Rocha, Dionne S. Brandsma, Lisette van der Molen, Maarten J. A. van Alphen, Michiel W. M. van den Brekel, Françoise J. Siepel

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your body is a busy highway system, and the throat is a critical tunnel where food and air must cross paths without crashing. Sometimes, this tunnel gets clogged or the traffic lights fail, a condition doctors call "swallowing trouble." To fix this, specialists use two high-tech tools to peek inside the tunnel. The first is like a video camera (X-ray movie) that shows the food moving in real-time. The second is a super-sensitive ruler made of tiny pressure sensors that measures how hard the muscles squeeze. Usually, doctors have to watch the video and look at the ruler's numbers separately, then try to guess how they fit together in their heads. It's like trying to solve a puzzle while wearing blindfolds on one eye and reading a map upside down. The big question scientists are asking is: Can we build a robot brain that automatically snaps the video and the ruler measurements together perfectly, even when the camera gets blurry or the sensors hide behind a lump of food?

This is exactly what a team of researchers from the University of Twente and the Netherlands Cancer Institute set out to do. They created a new computer program called GHOST (which stands for Geometric History for Occluded Sensor Tracking). Their goal was to solve a tricky problem: when patients swallow, the tiny sensors on the medical tube often get blocked by food or move out of the camera's view. Old computer programs tried to guess which sensor was which just by looking at their order from top to bottom, like assuming the first person in a line is always the tallest. But if the first person steps out of line, the whole guess falls apart.

The researchers tested their new GHOST system on 16 different swallowing tests from 12 patients with head and neck cancer. They taught the computer to spot the sensors using three different "eyes" (AI models called YOLO11n, YOLO12n, and YOLO26n) and then used a clever trick to keep track of them. Instead of just guessing the order, GHOST treats the sensors like a flexible, invisible chain of glowing dots. Even if a dot disappears behind a piece of food or the camera gets a bit grainy (simulating a lower-quality X-ray), the program remembers where that dot should be based on the shape of the chain and where it was a split second ago. It's like a game of "connect the dots" where the computer fills in the missing dots so you can still see the whole picture.

The results were impressive. The computer was incredibly good at spotting the sensors, finding them correctly more than 98% of the time. But the real magic happened when they tried to identify which sensor was which. In the old method, the computer got confused easily, especially when the video quality was poor or the sensors were hidden. GHOST, however, stayed calm and consistent. In the best tests, it correctly identified the sensors 96.6% of the time, which is a huge jump from the old method's performance. Even when the video was made to look "low quality" (noisier and darker), GHOST kept its cool, only dropping its accuracy slightly, while the old method struggled much more.

The paper suggests that this tool is a major step forward because it doesn't need a human to manually click and drag sensors to match them up. It works automatically, even when the sensors are partially hidden or the camera isn't perfect. However, the authors are careful to note that this isn't a magic cure-all yet. They tested it on a relatively small group of patients (12 people), so it needs to be tried on many more to see if it works for everyone. Also, right now, the computer is a bit slow, meaning it's best used to analyze videos after the patient has left the room, rather than watching the patient in real-time. But, the study concludes that GHOST proves it is possible to automatically and accurately link the video and the pressure data, paving the way for future tools that could help doctors diagnose swallowing problems faster and more easily.

Technical Summary: GHOST – Automatic Multimodal Registration of HRIM and VFSS

Problem Statement

Oropharyngeal dysphagia (OD) affects a significant portion of at-risk populations, including patients with head and neck cancer (HNC). While Video-Fluoroscopic Swallow Studies (VFSS) are the gold standard for diagnosis, they lack quantitative pressure data. High-Resolution Impedance Manometry (HRIM) provides this data but lacks anatomical context. Combining these modalities (mano-fluoroscopy) offers a comprehensive view but remains clinically challenging due to the cognitive load of mentally integrating two independent data streams.

Existing computer-assisted approaches have attempted to automate the registration of manometric sensors onto fluoroscopic images. However, prior methods relying on template matching for detection and simple spatial ordering (top-to-bottom) for identification suffer from significant limitations:

Detection Instability: Template matching fails under the variable imaging conditions of swallowing.
Identification Fragility: Spatial ordering assumes the first and last sensors are always visible. In clinical practice, catheter movement often causes sensors to leave the fluoroscopic field of view (FOV) or become occluded, leading to incorrect identity assignments and registration failure.

Methodology

The authors propose GHOST (Geometric History for Occluded Sensor Tracking), a fully automated, geometry-driven framework designed to robustly register HRIM and VFSS data by tracking sensor identities even during occlusion or FOV loss.

1. Sensor Detection

The framework utilizes deep learning-based object detection. The authors evaluated three YOLO architectures (YOLO11n, YOLO12n, and YOLO26n) trained on a dataset of 12 HNC patients (16 examinations).

Data: 10,829 annotated sensor bounding boxes across 526 frames.
Training: Leave-one-patient-out cross-validation with extensive data augmentation (spatial transformations, photometric perturbations, and dropout to simulate occlusions).
Performance: All models achieved high detection metrics (Precision, Recall, and F1-score > 98% in high-quality settings).

2. The GHOST Framework

GHOST treats sensor identification as a multi-object tracking (MOT) problem, managing a dynamic 2D point cloud representation of the catheter. The pipeline consists of three stages:

Point Cloud Representation: Detected bounding boxes are converted into "slots" (five-point representations: four corners and center). A reference point cloud ( $S_{ref}$ ) is maintained, storing both currently visible sensors and "ghost slots" for sensors that were previously detected but are currently missing.
Non-Rigid Registration: To align the current frame's detections ( $S_i$ $S_{i}$ ) with the reference ( $S_{ref}$ $S_{r e f}$ ), the system employs:
- Lucas-Kanade Optical Flow: For initial pre-alignment based on proximal sensor movement.
- Coherent Point Drift (CPD): A non-rigid registration algorithm that models the catheter as a Gaussian mixture model, accommodating the non-linear deformations of the catheter during swallowing.
Multi-Object Association & State Update:
- Hungarian Algorithm: Used to compute the globally optimal assignment of detected slots to reference slots based on Euclidean distances.
- Ghost Propagation: If a sensor is not detected, its identity is preserved in a "ghost slot." The position of these ghost slots is propagated using the transformation estimated during registration and smoothed via an exponential moving average.
- Growth Constraints: New sensors are only added to the proximal or distal ends of the chain if specific temporal and spatial distance criteria are met, preventing spurious insertions.
- First Sensor Validation: A specific probabilistic model validates the presence of the first sensor to prevent initialization errors.
- Global Correction: A final step enforces a strictly monotonically increasing sequence of sensor identities from proximal to distal.

Key Results

The study evaluated GHOST against a baseline spatial-ordering strategy (similar to Geiger et al.) across three detector backbones and two imaging conditions (High Quality and emulated Low Quality).

Detection Performance: All YOLO variants performed consistently well, with mean Precision, Recall, and F1-scores exceeding 98% in high-quality settings. YOLO12n generally achieved the highest F1-scores, while YOLO26n showed the highest AP50 but slightly lower recall in low-quality settings.
Identification Performance (RID):
- High Quality: GHOST significantly outperformed the baseline. With YOLO26n, GHOST achieved a mean Identification Recall (RID) of 96.6%, compared to 81.0% for the baseline. (Note: YOLO12n achieved 95.2% in this setting).
- Low Quality: GHOST maintained robustness, achieving an RID of 88.1% with YOLO11n versus 75.8% for the baseline. (Note: YOLO26n achieved 80.3% in this setting).
- Stability: GHOST substantially reduced the standard deviation of performance across patient folds, indicating a more stable identification process. The baseline frequently dropped below 40% in difficult folds, whereas GHOST's minimum remained near 80%.
Failure Modes: The primary failure mode for GHOST was the simultaneous occurrence of false positives and false negatives at opposite ends of the catheter, which could disrupt the association chain. However, it remained superior to the baseline in handling partial occlusions and temporary FOV loss.

Significance and Claims

The paper claims that GHOST provides a practical foundation for fully automated cross-modal analysis of mano-fluoroscopy examinations.

Robustness: The framework successfully addresses the critical limitation of prior work by preserving sensor identities even when sensors are occluded or leave the fluoroscopic field of view, a common occurrence in dysphagic patients.
Clinical Utility: By automating the registration process without manual intervention, GHOST reduces the interpretive burden on clinicians. This facilitates the development of computer-assisted tools that can integrate quantitative HRIM data with anatomical VFSS visualization.
Future Impact: The authors state that such tools may improve the overall usefulness of mano-fluoroscopy, particularly in complex clinical cases involving patients with altered anatomies (e.g., post-HNC surgery), by enabling faster and more comprehensive assessment of swallowing function.

The authors remain modest regarding the current scope, noting that the evaluation was performed on a relatively small cohort (12 patients) and that the current implementation is a post-processing tool rather than a real-time system. They emphasize that future work is required to validate generalizability on larger, diverse cohorts and to optimize computational efficiency for real-time application.