💻 computer science

OphEdit: Training-Free Text-Guided Editing of Ophthalmic Surgical Videos

OphEdit is a novel, training-free framework that leverages deterministic ODE inversion and selective tensor injection to enable precise, text-guided editing of ophthalmic surgical videos while rigorously preserving anatomical geometry and temporal consistency.

Original authors: Ritul Jangir, Arkya Jyoti Bagchi, Aiman Farooq, Mangalton Okram, Saurabh Seetaram Korgaonkar, Deepak Mishra

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ritul Jangir, Arkya Jyoti Bagchi, Aiman Farooq, Mangalton Okram, Saurabh Seetaram Korgaonkar, Deepak Mishra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a high-definition video of a very delicate eye surgery. Now, imagine you want to change just one small thing in that video—like swapping the tool the surgeon is holding or changing the liquid being injected—without messing up the rest of the scene. You want the eye to look exactly the same, the movements to stay smooth, and the background to remain untouched, but the specific action to change based on a simple text instruction.

Doing this with current technology is like trying to repaint a moving car while it's driving down the highway without smearing the paint or changing the car's shape. It's incredibly hard.

This paper introduces OphEdit, a new tool that does exactly this for eye surgery videos, but with a clever twist: it doesn't need to be "taught" or trained on thousands of new examples. It's a "training-free" solution.

Here is how it works, broken down into simple analogies:

1. The Problem: The "Replay" vs. The "Edit"

Usually, if you want to change a video, AI models try to generate the entire video from scratch based on your new text.

The Old Way: It's like asking an artist to redraw the whole movie scene from memory because you asked them to change the color of a tool. The result often looks weird; the tools might float, the eye might look distorted, or the timing might be off.
The Goal: We want to keep the original video's "soul" (the exact shape of the eye, the smooth motion) but just swap the "costume" (the tool or liquid) based on your text.

2. The Solution: OphEdit's "Magic Blueprint"

OphEdit uses a two-step process that acts like a master architect and a precise painter.

Step A: The "Blueprint" Phase (Inversion)
First, the system looks at the original video and works backward to figure out the "blueprint" or the mathematical recipe that created it.

The Analogy: Imagine taking a finished cake and working backward to figure out the exact amount of flour, sugar, and eggs used, plus the specific way the baker mixed them.
The Secret Sauce: While doing this, OphEdit doesn't just look at the whole cake; it specifically grabs the "Attention Value" (V) tensors. Think of these as the "structural memory" of the video. They hold the information about where the eye is, how the tissue looks, and the texture of the skin. It saves these specific "structural memories" but ignores the parts that just decide "what to look at."

Step B: The "Painting" Phase (Editing)
Now, the system starts generating the video again, but this time it follows your new text instructions (e.g., "Use a blue dye instead of clear gel").

The Trick: As it paints the new video, it constantly checks its saved "structural memories" from Step A.
The Injection: It forces the new video to stick to the original eye's shape and texture. It's like having a stencil of the original eye. You can change the color of the liquid inside the stencil (the text instruction), but the stencil ensures the outline of the eye never warps or breaks.

3. The "Smooth Motion" Secret

One of the biggest risks in editing videos is "jitter"—where the video looks like it's shaking or flickering.

The Analogy: Imagine walking across a room. If you take big, jerky steps, you look unstable. If you take smooth, calculated steps, you glide.
The Fix: OphEdit uses a "second-order" math trick. Instead of taking one big step to figure out the next frame, it takes a "half-step" to check its balance, then takes the full step. This ensures the motion is as smooth as the original surgery, preventing the video from looking like a shaky phone recording.

4. The Results: What They Found

The researchers tested this by asking an expert eye surgeon to judge the videos. They tried to change things like:

Swapping a phacoemulsification tool (an ultrasonic probe) for a different type of handpiece.
Changing a clear gel injection to a blue dye injection.

The Verdict:

OphEdit was much better at keeping the eye looking real and the tools looking correct compared to other general video editing tools (like FateZero).
While other tools often made the eye look distorted or the tools disappear, OphEdit kept the "anatomy" (the body parts) intact while successfully changing the "action."
It achieved this without needing to be retrained on new data, making it a fast and efficient tool for creating diverse medical training videos.

In short: OphEdit is like a smart editor that can swap the props in a surgery video based on your text, but it uses a "structural memory" to ensure the actor (the eye) doesn't break character or fall apart during the scene.

Technical Summary: OphEdit

Problem Statement
The generation of high-fidelity surgical videos is critical for medical training and AI development, yet obtaining large-scale, annotated datasets remains a significant bottleneck due to privacy regulations, the high cost of expert annotation, and the under-representation of rare clinical events. While recent text-to-video (T2V) models have advanced natural-domain video synthesis, applying them to ophthalmic surgery is challenging. Existing approaches primarily focus on regenerating entire video sequences from scratch, which is computationally inefficient and lacks the fine-grained controllability required to modify specific procedural steps (e.g., instrument swaps or phase transitions) within an existing video. Furthermore, standard video editing frameworks struggle to maintain the strict anatomical integrity and temporal consistency essential for clinical realism, often introducing structural distortions or jittering artifacts that compromise the utility of the output for medical purposes.

Methodology
OphEdit proposes a novel, training-free framework for text-guided editing of ophthalmic surgical videos. The system operates on a pre-existing surgical video generated by a T2V model (specifically leveraging the Ophora foundation) and modifies it based on a target text prompt without requiring additional model fine-tuning or paired datasets. The methodology consists of two primary stages:

Deterministic Inversion with Attention Value Storage:
The process begins by inverting the original surgical video into a noisy latent representation using a deterministic second-order ODE solver (DDIM-Solver). During this inversion, the framework captures and stores the Attention Value (V) tensors from the self-attention layers of the transformer. The authors focus on Value tensors rather than Query or Key tensors because V tensors encapsulate the rich anatomical structures and intricate textures of the eye. To balance computational efficiency with structural fidelity, these tensors are selectively stored only from the final $m$ attention blocks (operating at high spatial resolution) and the final $n$ inversion steps (where global structural layout is established).

To address the issue of temporal jittering common in surgical videos, the inversion employs a second-order integration scheme. Instead of a single linear discretization, the trajectory is refined by calculating an intermediate "half-step" at the log-SNR midpoint. This involves performing two forward passes at each step: one at the full step and one at the midpoint, ensuring rigorous spatio-temporal alignment.
Timestep-Aligned Attention Value Injection:
In the editing (denoising) phase, the system generates the edited video by guiding the reverse diffusion trajectory with the target text prompt ( $p_t$ ). To preserve the original anatomical geometry while applying semantic changes, OphEdit utilizes a selective injection mechanism. A binary injection schedule restricts the replacement of the conditional Value tensor with the stored source Value tensor to the initial high-noise regime (the first $n_{inject}$ steps).

During these initial steps, the framework enforces structural alignment by replacing the conditional branch's Value tensor with the stored source tensor within the Classifier-Free Guidance (CFG) formulation. This forces the network to adhere to the spatial and structural priors of the original anatomy while allowing the target prompt to guide semantic details. In later steps, the model denoises freely to synthesize fine textures and semantic details. The editing process mirrors the second-order integration scheme used in inversion, performing dual-pathway latent updates (full step and midpoint) to ensure temporal smoothness and eliminate flickering artifacts.

Key Contributions
The paper outlines three main contributions:

First Training-Free Framework for Ophthalmic Surgery: OphEdit is the first framework tailored specifically for the text-guided editing of ophthalmic surgical videos, enabling precise manipulation of procedural elements (instrument type, tissue response, surgical phase) via natural language without the need for exhaustive manual recording or costly model fine-tuning.
Novel Inversion-Editing Pipeline: The authors introduce a pipeline that captures Attention Value (V) tensors during second-order solver inversion and selectively injects them into the denoising process. This approach rigorously preserves anatomical fidelity while mapping text-driven semantic modifications onto the video stream.
Second-Order Integration for Temporal Consistency: To prevent jittering artifacts, the framework introduces a second-order integration scheme. By computing dual-pathway latent updates at the log-SNR midpoint, the method ensures rigorous spatio-temporal alignment, preserving the continuous procedural dynamics essential for clinical realism.

Results
The framework was evaluated using the Ophora dataset, which includes 12 distinct phases of ophthalmic surgery. The system was tested against an adapted general-domain video editing baseline, FateZero. An ophthalmologist performed a blind evaluation on 19 distinct phase-specific interventions, scoring the edits on Anatomical Consistency, Correct Operation, and Overall Edit.

Quantitative Performance: OphEdit (using the DDIM-Solver variant) achieved an Anatomical Consistency score of 61.40% and a Correct Operation score of 43.85%, outperforming FateZero (57.89% and 24.56%, respectively). The OphEdit variant using the second-order solver (DDIM-2nd Order) achieved the highest rate of generating correct edits (35.08%), compared to 26.31% for the standard solver and 24.56% for FateZero.
Qualitative Observations: Visual comparisons demonstrated that OphEdit successfully replaced instruments (e.g., swapping a phacoemulsification handpiece for a coaxial irrigation aspiration handpiece) and altered procedural substances (e.g., injecting trypan blue dye instead of sodium hyaluronate) while maintaining anatomical correctness. In contrast, FateZero frequently failed to change objects correctly and produced visible artifacts and temporal inconsistencies.
Distribution: The clinical evaluation scores showed that OphEdit (DDIM-Second Order) had the highest concentration of successful edits in the upper-right quadrant of the performance distribution, indicating accurate surgical modifications with preserved ocular structures.

Significance
The paper claims that OphEdit represents the first application of training-free video editing in the ophthalmic surgical domain. Its significance lies in offering a scalable solution for generating diverse, annotated medical datasets without the prohibitive costs associated with manual recording or model fine-tuning. By addressing the strict anatomical and temporal constraints of the medical domain, the work enables the creation of realistic surgical variations (such as instrument swaps and procedural variations) that are difficult to capture in real clinical environments. This capability supports the development of AI systems for surgical training, performance assessment, and robot-assisted surgery, potentially overcoming current data scarcity bottlenecks in medical AI.

1. The Problem: The "Replay" vs. The "Edit"

2. The Solution: OphEdit's "Magic Blueprint"

3. The "Smooth Motion" Secret

4. The Results: What They Found

Technical Summary: OphEdit

More like this