💻 computer science

A Unified Benchmark for RCM-Constrained Visual Servoing: Modeling-Controller Interaction and Robustness Analysis in Laparoscopic Robots

This paper introduces an open-source simulation framework that unifies three RCM modeling approaches and six IBVS control architectures to systematically evaluate their interactions, sensitivities, and robustness for safe visual servoing in laparoscopic robots.

Original authors: Jing Zhang, Mengtang Li

Published 2026-07-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jing Zhang, Mengtang Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to move a long, flexible straw (a laparoscope) through a small, rigid hole in a wall (a trocar) to look at something on the other side. The golden rule of this game is that the straw must pivot exactly at the hole. If the straw slides sideways through the hole, it tears the wall (damages the patient's tissue).

This paper is about building a "test track" to see how different computer brains (controllers) and different rulebooks (mathematical models) handle this pivoting rule.

Here is the breakdown of what the authors did, using simple analogies:

1. The Problem: Too Many Ways to Pivot

In the past, scientists have invented many different math formulas to tell a robot how to keep that straw pivoting in the hole. Some formulas treat the hole like a flat floor; others treat it like a specific point in space.

The Issue: Because everyone used different math and different test scenarios, it was impossible to know which method was actually the best. It was like comparing race cars where one driver was on a dirt track and the other on a highway.
The Solution: The authors built a unified simulation playground. They created a single, fair test track where they could plug in three different "rulebooks" (RCM models) and six different "drivers" (control systems) to see how they performed under the exact same conditions.

2. The Three "Rulebooks" (RCM Models)

The paper tested three main ways to mathematically describe the "pivot point":

Rulebook A (The Tangent Plane): Imagine drawing a flat sheet of paper touching the hole. The math tries to keep the straw moving parallel to that paper. The paper found that if you draw the paper wrong (e.g., flat relative to the room instead of the hole), the straw starts sliding sideways, tearing the "wall."
Rulebook B (The Projection): Imagine shining a laser beam straight down the straw's axis. The math projects the hole onto that beam to find the pivot. This is generally very accurate.
Rulebook C (The Virtual Joint): Imagine the straw has a secret, invisible joint inside it that can slide in and out. The math treats this sliding as a real movement. The paper found that if you don't strictly lock this invisible joint, the straw drifts in and out of the hole, even if it looks like it's pivoting correctly.

3. The Six "Drivers" (Control Frameworks)

Once the rulebook is chosen, a "driver" has to actually move the robot arm. The paper tested six types of drivers:

The "Direct" Drivers (PI & IK): These are like drivers who take the shortest path immediately. They are fast and simple, but if the road gets tricky (a "singularity," or a weird robot pose), they can get confused and crash, causing the straw to jerk violently out of the hole.
The "Optimizer" Drivers (QP): These are like cautious, strategic drivers. Before moving, they solve a complex puzzle: "How do I move the target and keep the straw in the hole without hitting my joint limits?" They are slower to calculate but much more stable when the road gets bumpy.

4. Key Discoveries from the Test Track

The authors ran simulations to see what happens when things go slightly wrong or when the robot faces tricky situations.

The "Open-Loop" vs. "Closed-Loop" Trap:
- Open-Loop (Guessing): Some drivers just calculate the path once and hope for the best. The paper found that even a tiny mistake in the starting math adds up over time, like a compass that is off by one degree. Eventually, the straw slides right out of the hole.
- Closed-Loop (Checking): The best drivers constantly check: "Am I still in the hole?" If they drift even a millimeter, they correct it immediately. This keeps the straw perfectly safe.
The "Drift" Danger:
In one test, the robot had to zoom in and out (change the size of the view). The "Tangent Plane" rulebook (Rulebook A) failed here. Because the math defined the "flat paper" relative to the room instead of the hole, the act of zooming in forced the straw to slide sideways through the hole. The "Virtual Joint" rulebook (Rulebook C) also failed unless you explicitly told it not to slide in and out.
The "Singularity" Crash:
When the robot arm gets into a weird, stretched-out position (a singularity), the "Direct" drivers (PI and IK) tend to panic. Their math breaks down, and the straw flies out of the hole. The "Optimizer" drivers (QP), however, handle this gracefully. They know how to bend the rules slightly to stay safe without crashing.

5. The Big Takeaway

The paper concludes that you cannot just pick a "good" math model and a "good" driver separately. They are tightly coupled, like a lock and a key.

If you pick a math model that allows the straw to slide in and out, a "Direct" driver might make that slide worse.
If you pick a tricky robot pose, a "Direct" driver will fail, but an "Optimizer" driver might save the day.

In short: The authors didn't invent a new robot or a new surgery. They built a fair test track that proves: to keep a surgical robot safe, you need a control system that constantly checks its work (closed-loop) and uses smart, cautious math (optimization) to handle tricky robot poses, all while using a rulebook that strictly forbids the straw from sliding sideways.

The code for this test track is now open-source, so other scientists can use this same "playground" to test their own ideas without reinventing the wheel.

Technical Summary: A Unified Benchmark for RCM-Constrained Visual Servoing

Problem Statement

In robot-assisted laparoscopic minimally invasive surgery (MIS), maintaining the Remote Center of Motion (RCM) constraint at the trocar is critical for preventing tissue damage and ensuring safe, stable automatic Field-of-View (FoV) adjustment. While control-based RCM strategies offer flexibility over mechanical solutions, the field lacks a unified, reproducible benchmark. This absence hinders systematic comparisons between different RCM modeling formulations and Image-Based Visual Servoing (IBVS) control architectures. Consequently, it remains unclear how specific modeling choices (e.g., tangent-plane definitions, constraint dimensionality) interact with controller structures (e.g., Pseudoinverse, Quadratic Programming) to affect performance, particularly regarding structural sensitivities and robustness near kinematic singularities.

Methodology

The authors propose an open-source simulation framework that integrates three representative RCM modeling approaches and six IBVS-compatible control architectures within a unified velocity-level formulation.

1. Unified RCM Modeling

The framework standardizes three distinct RCM modeling methods into a common constraint structure ( $J_{rcm,i} \dot{u} = v_{rcm,des}$ ):

RCM 1 (Tangent-Plane): Calculates a scalar parameter $\lambda$ based on the projection of the trocar point onto a tangent plane. The constraint enforces zero lateral velocity within this plane.
RCM 2 (Axis Projection): Projects the trocar point onto the laparoscope shaft axis to determine $\lambda$ . The constraint is formulated in the local RCM frame, controlling linear velocities in the x and y directions.
RCM 3 (Virtual Joint): Introduces $\lambda$ as a virtual joint variable. The constraint equation includes the derivative of this virtual variable ( $\dot{\lambda}$ ), allowing for a 3D constraint formulation.

2. Control Frameworks

Six control strategies are evaluated against the unified RCM constraints:

Pseudoinverse (PI) Methods:
- PI 1: Treats RCM and visual tasks equally in a single pseudoinverse solution.
- PI 2: Uses null-space projection to prioritize the RCM constraint over the visual task.
Quadratic Programming (QP) Methods:
- QP 1: Models both RCM and visual tasks as equality constraints with joint limits.
- QP 2: Models RCM as an equality constraint and the visual task as a cost function.
- QP 3: A hierarchical approach solving RCM constraints first, then the visual task in the null space.
Inverse Kinematics (IK) Method: Assumes perfect RCM satisfaction to derive a direct relationship between connector and end-camera twists, solving via IK.

3. Simulation Setup

Experiments were conducted in CoppeliaSim using a 6-DoF UR5 manipulator with a rigid laparoscope. The visual servoing task involved tracking a target rectangle (position, scale, orientation) to adjust the FoV. Five specific case studies were designed to isolate variables:

Tangent Plane Selection: Comparing plane definitions in RCM 1 (global vs. local frame).
Constraint Dimensionality: Comparing $N_{rcm}=2$ vs. $N_{rcm}=3$ in RCM 3.
Open- vs. Closed-Loop: Comparing geometric enforcement (zero desired velocity) vs. error-feedback enforcement.
Controller Robustness: Comparing all six frameworks under straight-line and helical (singular) trajectories.
Reduced Task Complexity: Evaluating 2-DoF visual servoing (position only) and its effect on axial drift.

Key Results

The simulation studies revealed several critical structural sensitivities:

Modeling Sensitivity: In RCM 1, defining the tangent plane in the global frame ( $s$ ) caused significant lateral drift during insertion, whereas defining it in the local RCM frame preserved the physical constraint. In RCM 3, relaxing the axial constraint ( $N_{rcm}=2$ ) allowed the virtual variable $\lambda$ to drift, resulting in axial RCM errors exceeding 20 mm.
Open-Loop Limitations: Open-loop formulations (geometric enforcement without error feedback) consistently resulted in non-zero steady-state RCM errors and axial drift. Closed-loop formulations, which incorporate RCM position error feedback, successfully drove errors to sub-millimeter levels.
Controller Robustness: Under nominal motion, PI, IK, and QP methods performed similarly. However, near kinematic singularities (e.g., helical trajectories), PI and IK methods failed due to Jacobian ill-conditioning, leading to large RCM deviations and loss of tracking. QP-based methods maintained stability and task convergence by explicitly handling constraints and regularization.
Trade-offs: Relaxing axial constraints (e.g., $N_{rcm}=2$ ) reduced joint motion costs but introduced axial drift, highlighting a trade-off between motion efficiency and strict constraint enforcement.

Significance and Contributions

The paper does not propose a new RCM algorithm but rather provides a structured benchmark to evaluate the interaction between modeling and control. Its primary contributions are:

Unified Formulation: A velocity-level framework that expresses three distinct RCM models under a common constraint structure, enabling direct comparison.
Modular Benchmark: An open-source simulation infrastructure integrating three RCM models and six control frameworks for reproducible testing.
Systematic Insights: A study exposing how modeling choices (tangent-plane definition, dimensionality) and controller architectures (PI vs. QP) interact to determine system accuracy and stability.
Reproducibility: The release of code and video demonstrations to serve as a baseline for future research in RCM-constrained visual servoing.

The authors conclude that robustness in RCM-constrained visual servoing is not determined by modeling or control in isolation, but by the structural coupling between the two. Future work is noted to potentially include dynamic effects and more realistic physical conditions.