💻 computer science

Feedback Matters: Augmenting Autonomous Dissection with Visual and Topological Feedback

This paper proposes a feedback-enabled framework for autonomous tissue dissection that leverages visual and topological analysis of endoscopic images to guide online policy adaptation, incorporating visibility metrics and optimal control to significantly enhance system robustness and reduce errors in dynamic surgical environments.

Original authors: Chung-Pang Wang, Changwei Chen, Xiao Liang, Soofiyan Atar, Florian Richter, Michael Yip

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chung-Pang Wang, Changwei Chen, Xiao Liang, Soofiyan Atar, Florian Richter, Michael Yip

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a surgeon trying to cut a piece of delicate tissue, like separating a thin layer of skin from a muscle. Now, imagine doing this while wearing thick gloves and looking at the tissue through a tiny, foggy camera. If you cut a little bit, you can't always tell if you've fully separated the two pieces or if a tiny, invisible thread is still holding them together. If you miss that thread, the surgery isn't done. If you cut too much, you damage healthy tissue.

This paper presents a new "smart assistant" for robotic surgeons that solves this problem by adding a feedback loop. Instead of just cutting blindly, the robot cuts, checks its work, and if it made a mistake, it fixes it before moving on.

Here is how the system works, broken down into simple steps:

1. The "Cut and Check" Loop

Think of this like editing a photo. You don't just take one picture and hope it's perfect. You take a picture, look at it, realize the lighting is bad or the subject is blurry, and then you adjust the camera or the subject before taking the next shot.

The Action: The robot makes a cut based on a goal given by a human.
The Check: Immediately after cutting, the robot looks at the tissue to see: "Did I actually separate these two pieces, or is there still a connection?"
The Fix: If the robot sees a tiny connection it missed, it doesn't just guess. It calculates exactly where the cut is incomplete and generates a new, specific instruction to finish the job.

2. The "Stretching" Trick (Exposure Maximization)

This is the paper's most creative idea. Sometimes, tissue is slippery or folds over itself, hiding the cut line. It's like trying to see if a zipper is fully closed when the fabric is bunched up. You can't tell if the zipper is stuck or just hidden.

The robot has a special "stretching" mode. Before it decides if the cut is done, it gently pulls the tissue apart (like stretching a rubber band) to flatten it out.

Why? When you stretch the tissue, any remaining connections get pulled tight and become very obvious. If the tissue is fully cut, it stretches easily. If a thread is still holding it, you'll see a "tent" or a tight spot.
The Result: This stretching makes the "check" phase much more accurate, ensuring the robot doesn't miss hidden connections.

3. Two Types of "Brains"

The researchers tested this feedback system with two different types of robotic "brains" to prove it works for everyone:

The Planner: This robot thinks step-by-step, like a GPS giving turn-by-turn directions. It maps out a path and follows it.
The Learner: This robot learned by watching human surgeons perform the task, similar to how a student learns by shadowing a master. It tries to mimic what it saw.

The Result: Both types of robots failed more often when they didn't have the feedback system. But when they used the "Cut, Stretch, Check, and Fix" loop, they became much more successful. The "Planner" got better at finding tiny missed threads, and the "Learner" got better at cutting the full length without stopping too early.

4. What They Actually Tested

The paper is very specific about what they tested. They did not test this on real human patients in a hospital. Instead, they used:

Chicken skin and beef: Real animal tissues taken from a butcher (ex vivo).
Silicone models: Fake tissues made of rubber to simulate human organs.
A research robot: They used the da Vinci Research Kit, a robotic system designed for scientists to test new ideas, not the one used in actual operating rooms yet.

The Bottom Line

The paper argues that for robots to be truly autonomous in surgery, they can't just be "dumb" machines that follow a script. They need to be able to see what they just did, realize if they made a mistake, and actively move the tissue to get a better look before trying again.

By adding this "stretch and check" feedback, the robots made fewer mistakes, cut more accurately, and were much better at finishing the job completely, even when the tissue was tricky to see. The authors admit their current system works best on thin layers of tissue and plan to test it on thicker, 3D tissues in the future.

Technical Summary: Feedback Matters: Augmenting Autonomous Dissection with Visual and Topological Feedback

1. Problem Statement

Autonomous surgical systems face significant challenges in highly dynamic environments where tissue properties and visual cues evolve rapidly. While feedback mechanisms in surgical robotics have advanced in areas like tool tracking, needle tracking, and error detection, existing methods remain limited in handling the specific topological and perceptual challenges of tissue dissection.

Current automated dissection approaches often rely on predefined goals and lack the ability to:

Verify task completion in real-time.
Adapt when dissection progresses differently than expected.
Distinguish between connected and severed tissue when visual cues are ambiguous (e.g., due to lack of tissue stretching).
Actively manipulate the environment to expose hidden state information necessary for safe execution.

The core problem addressed is the lack of reliable, feedback-enabled autonomy that can detect failures, reason about topological changes, and actively correct dissection errors in real tissue environments.

2. Methodology

The authors propose a feedback-enabled framework for autonomous tissue dissection that operates in a closed loop. The system explicitly reasons about topological changes from endoscopic images after each action and utilizes active manipulation to improve feedback reliability. The pipeline consists of three iterative components:

A. Autonomous Dissection Agents

The framework is designed to be agnostic to the underlying control policy, integrating with both:

Learning-based Policy: A 3D visuomotor policy using behavioral cloning (Action Chunking with Transformer, ACT). It utilizes a "slow-fast" strategy, combining high-frequency monocular depth estimates (Video Depth Anything) with lower-frequency point clouds (modified Stereo Anywhere) to execute bimanual retraction and cutting.
Planning-based Agent: A planner that generates 3D paths by projecting tracked image keypoints into the robot's workspace. It executes a sequence of grasping, retracting along the goal's normal vector, and cutting.

B. Tissue Connectivity Estimation

To determine if a cut is complete, the system estimates tissue connectivity using a combination of Keypoint Tracking and a rule-based approach:

Tracking: Keypoints are sampled around the dissection target. The system tracks their temporal evolution using either Image Keypoint Tracking (IKT) or Model-Based Tracking (MBT) based on an XPBD (Extended Position-based Dynamics) simulator.
Connectivity Logic: The system calculates the elongation ratio ( $\rho$ ) of edges connecting neighboring keypoints. If an edge stretches beyond a threshold $\tau$ times its original length, it is classified as "disconnected" (severed).
Output: A set of remaining connected edges ( $E_{uncut}$ ) that require further action.

C. Exposure Maximization Control

The accuracy of connectivity estimation depends heavily on the visibility of the dissected region. The authors introduce an Exposure Maximization Controller that actively deforms tissue to create an optimal viewing configuration.

Objective: Solve a finite-horizon optimal control problem to maximize visibility while ensuring the tissue is stretched orthogonally to the dissection path.
Cost Functions:
1. Camera-facing normal alignment: Encourages surface normals to face the camera.
2. Deformation expanding: Encourages expansion orthogonal to the dissection goal and minimizes shearing parallel to it.
Constraints: Enforces that the region of interest remains within the camera's field of view.
Solver: Uses a sampling-based solver (MPPI) accelerated by GPU (Warp) to find the optimal robot action sequence.

D. Recovery Feedback Planning

If the connectivity estimation indicates incomplete dissection, a Recovery Planner generates corrective actions:

Formulation: The problem is modeled as a Minimum Set Covering problem to find the smallest set of new dissection segments that intersect all remaining uncut edges.
Algorithm: A greedy approximation algorithm (Algorithm 1) iteratively selects candidate line segments that cover the maximum number of uncut edges, generating a refined dissection goal for the next iteration.

3. Key Contributions

Feedback-Enabled Framework: A novel architecture that explicitly reasons about topological changes and integrates structured feedback into autonomous dissection loops.
Active Visibility Maximization: The introduction of visibility metrics and an optimal controller that actively manipulates tissue to maximize exposure, significantly improving the reliability of topological state estimation.
Versatile Integration: Demonstration of the framework's compatibility with both planning-based and learning-based dissection agents.
Recovery Mechanism: A method to detect incomplete cuts and generate precise corrective goals, moving beyond simple execution to error correction.

4. Experimental Results

The authors conducted extensive experiments on the da Vinci Research Kit (dVRK) using both tissue phantoms and ex vivo animal tissues (beef and chicken skin).

Tracking Performance: LiteTracker achieved the highest detection accuracy (92.8%) for connectivity estimation compared to CoTracker3 and RAFT.
Exposure Maximization: Compared to a naive normal planner, the proposed controller increased the visible area of the dissected region by 23 times (from ~69 px to ~1653 px) and improved connectivity estimation accuracy from 5.56% to 86.00%.
Dissection Autonomy:
- Success Rate: Both learning-based and planning-based agents with feedback achieved an 80% success rate, significantly outperforming baselines without feedback (20–45%).
- Error Reduction: The feedback framework drastically reduced "Remaining Attachments" (from 16 to 3 for planning-based; from 0 to 0 for learning-based, but with better length deviation) and "Length Deviation" (from ~5.8mm to 1.84mm for learning-based).
- Iterative Improvement: Over three attempts, the feedback-enabled agents showed a sharp decrease in remaining attachments and excessive cuts, whereas non-feedback baselines showed minimal improvement.
Real-World Demonstration: The system successfully performed a sequential 7.6 cm dissection on ex vivo chicken tissue, demonstrating effectiveness in a multi-step scenario.

5. Significance and Claims

The paper claims that feedback is the core mechanism for mastering skills in both human cognition and robotic autonomy. The significance of this work lies in its ability to:

Bridge the gap between passive sensor interpretation and active intervention, allowing robots to "seek" information (via exposure maximization) rather than just reacting to it.
Enhance robustness in complex surgical scenarios by enabling systems to detect and recover from errors (incomplete cuts) that would otherwise lead to task failure.
Provide a generalizable framework that improves autonomy regardless of whether the underlying agent is data-driven (learning-based) or model-driven (planning-based).

The authors modestly note that the current framework focuses on thin-shell tissues and may struggle with occluded or volumetric tissues. They identify future work as expanding these mechanisms to handle volumetric dissection and validating the system on porcine tissue for realistic surgical settings.