💻 computer science

Deep Learning-Enhanced Robotic Subretinal Injection with Real-Time Retinal Motion Compensation

This paper presents a fully autonomous robotic system that integrates intraoperative OCT imaging with LSTM-based deep learning to predict and compensate for physiological retinal motion, achieving precise and safe subretinal injections with a mean tracking error below 16.4 μm.

Original authors: Tianle Wu, Mojtaba Esfandiari, Peiyao Zhang, Russell H. Taylor, Peter Gehlbach, Iulian Iordachita

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tianle Wu, Mojtaba Esfandiari, Peiyao Zhang, Russell H. Taylor, Peter Gehlbach, Iulian Iordachita

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to thread a needle while standing on a boat in rough seas. Now, imagine that needle is thinner than a human hair, the "thread" is a layer of tissue in your eye that can never grow back if damaged, and the "boat" is your own eye, which is constantly jiggling because of your heartbeat and breathing.

This is the challenge doctors face when performing subretinal injections to treat eye diseases like Age-Related Macular Degeneration (AMD). The goal is to slip medicine under the retina without poking a hole in the delicate tissue below it.

This paper describes a robotic system designed to solve this problem by acting like a super-smart, steady hand that can "dance" along with the eye's natural movements.

Here is how the system works, broken down into simple steps:

1. The Problem: The Eye is Never Still

Even when you are lying still, your eye moves slightly due to your heartbeat and breathing. It's like a tiny trampoline bouncing up and down. If a surgeon tries to insert a needle while the eye is moving, they might miss the target or, worse, tear the sensitive tissue. Humans have hand tremors, and even the best surgeons can't perfectly match the speed of a bouncing eye.

2. The Solution: A Robot with "Crystal Ball" Vision

The researchers built a robot that doesn't just watch the eye; it predicts where the eye will be a split-second in the future.

The Eyes (iOCT): The robot uses a special camera called an intraoperative Optical Coherence Tomography (iOCT). Think of this as a high-speed ultrasound that takes cross-section pictures of the eye layers in real-time.
The Brain (Deep Learning): The robot uses an AI brain called an LSTM (Long Short-Term Memory) network. Imagine this AI as a dancer who has watched the eye move thousands of times. Instead of just guessing the next step, it learns the rhythm and patterns of the eye's bounce. It can predict exactly where the top layer of the retina (the ILM) will be a fraction of a second from now.
- The Comparison: The team tested this AI against a simpler method (like trying to predict a wave by just drawing a perfect sine curve). The AI was much better because real eye movements aren't perfectly perfect waves; they are messy and irregular. The AI handled the messiness much better.

3. The Dance: Synchronized Movement

Once the AI predicts where the retina will be, the robot moves the needle to match that prediction.

The "Surfing" Analogy: Imagine the needle is a surfer and the retina is a wave. Instead of trying to stab the wave from above, the robot moves the needle with the wave, keeping a safe, constant distance just above the surface.
The Speed Control: The robot doesn't move at a constant speed. It uses a "dynamic proportional" strategy. If the eye moves fast, the needle speeds up. If the eye slows down, the needle slows down. It's like a car with cruise control that automatically adjusts its speed to match the car in front of it, ensuring a smooth ride.

4. The Safety Checks

Before the robot even tries to insert the needle, it runs a "sanity check."

It makes sure the needle is in the right spot.
It checks if the camera can clearly see the layers.
If the data looks weird (like if the needle appears to be in the wrong part of the eye), the robot stops and asks a human to reset it, rather than risking damage.

5. The Results: Testing the System

The team tested this robot in two ways:

In a Computer Simulation: A perfect, ideal world where nothing goes wrong.
In Real Pig Eyes (Ex Vivo): They used pig eyes in a lab setting. To make it realistic, they attached the eyes to a machine that shook them up and down to mimic human breathing and heartbeats.

The Outcome:

The robot successfully tracked the moving eye layers with high precision.
In the "pre-insertion" phase (hovering above the eye), the robot stayed within about 16 micrometers of its target. To put that in perspective, a human hair is about 50 to 70 micrometers thick. The robot was staying within a distance smaller than the width of a hair, even while the eye was moving.
They successfully injected fluid under the retina in the pig eyes, creating a "bleb" (a small bubble of fluid), which proves the needle got into the right space without tearing the tissue.

Summary

This paper presents a robotic system that uses AI to predict eye movements and robotics to move in perfect sync with them. It turns a dangerous, jittery procedure into a smooth, synchronized dance, significantly reducing the risk of damaging the eye. While the paper notes that more testing is needed in real human surgeries, the lab results show that this "smart robot" approach is a major step forward in making these delicate eye surgeries safer and more precise.

Technical Summary: Deep Learning-Enhanced Robotic Subretinal Injection with Real-Time Retinal Motion Compensation

Problem Statement
Subretinal injection is a critical procedure for delivering therapeutic agents to treat retinal diseases like age-related macular degeneration (AMD), particularly for emerging treatments such as stem cell transplantation and gene therapy. However, performing this procedure manually is fraught with challenges due to the delicacy of the retinal pigment epithelium (RPE), physiological hand tremors (~182 µm), and the thinness of the retina (100–570 µm). Furthermore, physiological factors such as respiration and heartbeat induce reciprocating retinal motion (axial movement of ~81 µm at ~1 Hz), which significantly degrades needle positioning accuracy in autonomous robotic systems. Existing autonomous methods often assume a static eye, neglecting this motion and risking irreversible damage to the non-regenerative RPE layer.

Methodology
The authors propose a fully autonomous robotic system that integrates intraoperative optical coherence tomography (iOCT) with deep learning-based motion prediction to synchronize needle motion with retinal displacement. The system utilizes the SHER 2.0 robotic manipulator and follows a structured five-phase workflow:

Motion Estimation: A U-Net segmentation model processes iOCT B5-scans (five evenly spaced 2D slices) to extract the z-position of the internal limiting membrane (ILM).
Needle Registration: Before insertion, the needle is positioned ~500 µm above the ILM. A 1D registration process maps the needle tip position from the image frame (pixels) to the robot's local frame (millimeters), utilizing temporal filtering (IQR outlier removal and median calculation) to reduce noise.
Sanity Check: The system verifies the registration accuracy and ensures the needle is not misaligned (e.g., in the lower half of the B-scan) before proceeding.
Motion Synchronization: The robot tracks the ILM motion in real-time. A Long Short-Term Memory (LSTM) neural network predicts the ILM's future z-position based on a sequence of 20 previous frames. This prediction outperforms a baseline Fast Fourier Transform (FFT) sine wave model, particularly in capturing non-periodic motion patterns.
Insertion: A dynamic proportional speed control algorithm adjusts the needle's insertion velocity ( $v$ ) based on the distance to the target depth ( $d_{target} - d_{current}$ ), ensuring smooth synchronization with the moving ILM layer.

Key Contributions

Fully Automated System: Development of a robotic subretinal injection system capable of real-time motion compensation, adapting to varying retinal motion amplitudes and frequencies in conditions approximating clinical settings.
Dynamic Control Strategy: Implementation of a proportional speed control algorithm that replaces constant velocity insertion with an adaptive approach, enhancing precision during motion synchronization.
LSTM-Based Prediction: Introduction of a robust LSTM motion prediction model that achieves high accuracy and fast inference (<0.25 s) with minimal input data, significantly outperforming baseline FFT approaches.
Comprehensive Validation: Rigorous testing in both simulation and ex vivo open-sky porcine eye environments to demonstrate system effectiveness under ideal and biological conditions.

Experimental Results
The system was validated using 18 datasets from 8 porcine eyes, simulating retinal motion with amplitudes of 0.05–0.15 mm and frequencies of 8–10 bpm.

Prediction Accuracy: The LSTM model demonstrated superior performance over the FFT baseline. In the best-case scenario (0.05 mm amplitude, 8 bpm), the LSTM achieved a Root Mean Squared Error (RMSE) of 6.69 µm and a Maximum Absolute Error (MaxAE) of 18.21 µm. In the worst-case scenario (0.15 mm amplitude, 10 bpm), the RMSE was 33.73 µm and MaxAE was 137.18 µm. The LSTM consistently showed lower errors and less latency than the FFT model.
Control Precision: In ex vivo experiments, the mean tracking error during pre-insertion phases (motion synchronization above the ILM) was 16.4 µm. While simulation results showed higher precision (RMSE of 12.61 µm), ex vivo results reflected the challenges of biological uncertainty, with RMSE increasing to 36.25 µm during the post-insertion phase due to tissue deformation and segmentation noise.
Success Rate: Out of 8 injection trials in ex vivo eyes, 5 resulted in successful subretinal injections, evidenced by the formation of a stable bleb.

Significance and Claims
The paper claims that integrating predictive modeling, adaptive registration, and synchronized control significantly enhances the safety and accuracy of autonomous robotic subretinal injections in dynamic retinal environments. The authors assert that their approach addresses the critical limitation of neglecting physiological eye motion in current autonomous systems. By achieving a mean tracking error below 16.4 µm in pre-insertion phases and successfully compensating for reciprocating motion, the system demonstrates the potential of AI-driven robotic assistance to mitigate risks associated with RPE damage. The authors modestly note that while the current ex vivo setup uses an open-sky configuration, future work will focus on optimizing control strategies for larger velocity variations and validating the system in intact porcine eyes to better replicate the clinical surgical environment.