ConVibNet: Needle Detection during Continuous Insertion via Frequency-Inspired Features

This paper introduces ConVibNet, a real-time deep learning framework that enhances continuous needle detection in ultrasound-guided interventions by leveraging temporal dependencies and a novel intersection-and-difference loss to achieve superior tip localization accuracy and robustness compared to existing baselines.

The Gist

Imagine you are trying to thread a needle while wearing thick, foggy winter gloves, and the room is filled with static noise. Now, imagine you have to do this while someone is constantly pushing the needle through a block of jello. That is roughly what doctors face when they perform ultrasound-guided procedures, like biopsies or anesthesia. They need to see a tiny metal needle inside the body, but on the ultrasound screen, the needle often looks like a faint, flickering ghost that disappears and reappears.

This paper introduces a new "super-vision" system called ConVibNet that helps doctors (and eventually robots) see that needle clearly, even when it's almost invisible.

Here is how it works, explained through simple analogies:

1. The Problem: The "Ghost Needle"

In a normal ultrasound, the needle is hard to see because it's thin and the body tissue creates a lot of "static" (noise). It's like trying to spot a silver thread in a pile of silver glitter. Existing computer programs try to find the needle by looking at a single picture, but if the needle is hidden behind a shadow or a weird artifact, the computer gets confused and loses track.

2. The Secret Sauce: Shaking the Needle

To make the needle easier to find, the researchers use a clever trick. They attach a tiny motor to the needle that makes it vibrate (shake) very fast, about 2.5 times per second.

Think of it like this: If you are in a dark room and someone is standing perfectly still, you can't see them. But if they start tapping their foot to a specific rhythm, you can hear them and know exactly where they are. The needle is doing the same thing—it's "tapping" its way through the tissue, creating a unique rhythm that the computer can listen for.

3. The Innovation: Listening to the "Rhythm" (Frequency)

The old computer programs (like the previous version, VibNet) were great at finding the needle when it was just sitting still and vibrating. But they failed when the doctor started pushing the needle deeper (continuous insertion). Pushing the needle creates extra movement that confuses the computer, like someone trying to talk over a drumbeat.

ConVibNet is the upgraded version. It doesn't just look at one picture; it watches a movie of the needle moving.

• The Analogy: Imagine you are trying to find a specific dancer in a crowded, chaotic dance floor.
  • Old Method: You take a snapshot. If the dancer is blocked by someone else, you lose them.
  • ConVibNet: You watch the whole video. Even if the dancer is blocked for a second, you know they are still dancing to the same beat. You can predict where they will be in the next frame because you understand their rhythm.

4. The New "Brain" Trick: The Intersection-and-Difference Loss

The paper introduces a special mathematical rule (a "loss function") to teach the computer how to learn better. Think of this as a training drill for the AI.

• The Intersection (The "Agreement" Rule): The computer looks at two slightly different moments in time and asks, "Where do these two pictures agree?" If the needle is in the same spot in both, the computer gets a gold star. This teaches the AI to be consistent and not get distracted by random noise.
• The Difference (The "Change" Rule): The computer also looks at what changed between the two moments. "Ah, the needle moved 2 millimeters to the left." This teaches the AI to understand motion and direction.

By combining these two rules, the AI learns to ignore the "static" (the noise) and focus only on the needle's unique vibration and movement path.

5. The Results: Faster and Smarter

The researchers tested this new system against older models and found:

• Accuracy: It found the tip of the needle with incredible precision (within about 2.8 millimeters). That's roughly the width of a pencil eraser.
• Speed: It works in real-time, fast enough to keep up with a live doctor's hand movements.
• Reliability: It succeeded in finding the needle nearly 80% of the time, whereas older methods only succeeded about 64% of the time.

Why Does This Matter?

Currently, doctors have to rely on their eyes and experience to find the needle, which can be tiring and risky if the needle gets lost in the "fog."

ConVibNet is like giving the doctor a pair of X-ray glasses that only highlight the needle's vibration. In the future, this technology could be built into robotic arms that can insert needles automatically, making procedures safer, faster, and less painful for patients. It turns a difficult, "needle-in-a-haystack" problem into a straightforward task of following a rhythmic beat.

Technical Summary

1. Problem Statement

Ultrasound (US)-guided needle interventions (e.g., biopsies, anesthesia) rely heavily on accurate needle placement. However, automated detection is hindered by:

• Poor Visibility: Needles often suffer from low contrast, speckle noise, and occlusion by artifacts in US images.
• Intermittency: The needle shaft and tip frequently disappear from the imaging plane during insertion.
• Limitations of Existing Methods:
  • Single-frame methods (e.g., standard CNNs) fail when the needle is occluded or has low contrast.
  • Temporal methods (e.g., CNN-LSTM) often struggle when background textures differ between training and inference.
  • Hardware-augmented methods (e.g., echogenic needles) increase cost and may introduce artifacts.
  • Previous Frequency-based methods (VibNet): While robust, the original VibNet was restricted to static needle detection and could not handle continuous insertion in real-time due to computational bottlenecks (Hough Transform) and a lack of temporal continuity modeling for dynamic motion.

2. Methodology: ConVibNet

The authors propose ConVibNet, a real-time framework that extends VibNet to handle continuous needle insertion by leveraging frequency-domain features and temporal correlations.

A. Core Architecture & Frequency Analysis

• Frequency Domain Features: The system induces mechanical vibrations (~2.5 Hz) on the needle using a stepper motor. Short-Time Fourier Transform (STFT) analysis confirms that the needle tip and shaft exhibit distinct, strong frequency components compared to surrounding tissue and background, even when visually invisible in B-mode images.
• Network Modification:
  • Removal of Hough Transform: The original VibNet used a Deep Hough Transform (DHT) for tip localization, which is computationally expensive. ConVibNet replaces this with a segmentation head to enable real-time inference.
  • Input: Processes sequences of $L=30$ consecutive US frames (30 fps).
  • Output: Generates a segmentation mask for the final frame of the sequence, from which the tip position and shaft angle are derived.

B. Novel Loss Function: Intersection-and-Difference Loss

To address the challenge of continuous motion and temporal consistency, the authors introduce a specialized loss function alongside the standard Focal Loss (to handle class imbalance).

• Input Strategy: Two input sequences are processed independently, where the final frames are separated by $\Delta t$ steps (e.g., 5 frames).
• Intersection Loss ($L_{inter}$): Computes the Binary Cross-Entropy (BCE) between the element-wise product (intersection) of predictions and ground truths. This enforces spatial consistency in fine-grained regions across overlapping sequences.
• Difference Loss ($L_{diff}$): Computes the BCE between the absolute differences of predictions and ground truths. This acts as a temporal regularizer, forcing the model to learn motion dynamics and temporal correlations between frames.
• Total Loss: $L = L_f^{(t)} + L_f^{(t+\Delta)} + \alpha L_{inter} + \beta L_{diff}$.
  • Note: The difference loss is activated only after the model reaches a stable state (from the 2nd epoch) to prevent convergence issues.

C. Data Acquisition & Annotation

• Platform: An ex vivo porcine tissue setup with an 18-gauge needle.
• Ground Truth Generation: Since direct annotation of invisible needles is impossible, the authors used an NDI optical tracking system (Polaris Vicra) to track passive markers on the needle hub.
  • The tip location in the first visible frame is manually annotated.
  • Subsequent tip locations are calculated via 3D trajectory synchronization and spatial conversion, allowing for accurate ground-truth masks even when the needle is invisible in the US image.
• Dataset: 106 videos (120 trials initially), covering insertion angles of 15° and 30°. Split into 80% training, 10% validation, and 10% testing.

3. Key Contributions

1. First Real-Time Continuous Detection: Successfully adapted frequency-based needle detection from static scenarios to continuous insertion scenarios.
2. Novel Loss Function: Introduced the Intersection-and-Difference Loss, which explicitly models motion correlations across consecutive frames to improve temporal awareness and robustness against occlusion.
3. Efficient Architecture: Replaced the computationally heavy Hough Transform with a segmentation head, achieving real-time inference speeds compatible with clinical US frame rates.
4. Robust Data Pipeline: Developed a semi-automated annotation pipeline using optical tracking to generate reliable ground truth for "invisible" needles in ultrasound.

4. Results

The model was evaluated on the curated dataset against two baselines: VibNet (w/o DHT) and UNet-LSTM.

Metric	ConVibNet (Ours)	VibNet (w/o DHT)	UNet-LSTM
Tip Error	2.80 ± 2.42 mm	3.55 ± 2.65 mm	3.60 ± 2.75 mm
Angle Error	1.69 ± 2.00°	1.59 ± 2.13°	1.59 ± 1.78°
Success Rate	79.6%	63.7%	62.7%
Inference Time	33 ms (30 Hz)	N/A	N/A

• Performance Gain: ConVibNet improved tip localization accuracy by 0.75 mm over the best baseline and increased the success rate by 15.9%.
• Qualitative Results: ConVibNet produced continuous segmentation masks and accurate tip localization even in challenging scenarios (e.g., 30° insertion with tissue occlusion), whereas baselines produced fragmented detections or missed the tip entirely.
• Ablation Study: Confirmed that both the Intersection Loss (spatial consistency) and Difference Loss (temporal dynamics) are complementary. The optimal configuration ($\alpha=0.5, \beta=0.02$) yielded the best results.

5. Significance and Future Work

• Clinical Impact: ConVibNet demonstrates high potential for integration into autonomous needle insertion systems and motorized robotic platforms, addressing a critical bottleneck in US-guided interventions.
• Robustness: By leveraging frequency features and temporal modeling, the system overcomes the limitations of low contrast and occlusion that plague traditional image-based methods.
• Limitations & Future Directions:
  • Current validation is limited to specific angles (15°, 30°) and ex vivo tissue.
  • The model does not explicitly disentangle operator motion from induced vibration.
  • Future work aims to validate on diverse probe types, incorporate needle bending models, and utilize robotic platforms with optical fiber sensors for more robust ground-truth acquisition.

In summary, ConVibNet represents a significant advancement in medical image analysis by successfully bridging the gap between frequency-domain signal processing and deep learning for real-time, continuous medical device tracking.