Surgical Vision ROS2: a ROS2 Humble pipeline for intraoperative wound assessment and variable-autonomy arm control

Technical Summary: Surgical Vision ROS2

Problem Statement
Teleoperated surgical robots currently rely entirely on human operators to inspect tissue, judge wound states, and decide on arm movements. The robot arm lacks an independent perceptual view of the surgical site and cannot modify its behavior based on visual cues. This arrangement is effective for experienced surgeons but poses risks in training environments (where trainees may miss cues) and during long procedures (where attention may fade). The paper identifies a need for a "perception-conditioned assistance" layer that independently monitors the wound and constrains the arm's autonomy, acting as a secondary, cheaper judgment to catch errors the operator might miss.

Methodology
The authors present Surgical Vision ROS2, a software pipeline built on ROS2 Humble and OpenCV that couples wound perception to variable-autonomy arm control.

• Architecture: The system consists of four decoupled ROS2 nodes connected in a linear producer-consumer chain:

  1. lesion_detector_node: Reads images, computes a clinical-style score, and publishes the state.
  2. behavior_manager_node: Maps the score to an autonomy tier and publishes the mode.
  3. surgeon_console_node: Enforces the autonomy tier (gating operator commands).
  4. moveit_bridge_node: Forwards approved commands to MoveIt2 for execution.
  • Communication occurs via string-based topics (/scene_state, /autonomy_mode, /arm_command), ensuring modularity.

• Scoring Framework (ABCDE): The vision logic adapts the dermatological ABCDE framework (Asymmetry, Border, Colour, Diameter, Evolution) to surgical wounds. It calculates five sub-scores (0–2) based on image metrics (e.g., contour solidity, Laplacian variance, HSV color classification) to generate a total score (0–10).

  • Verdict Mapping: Scores 0–1 map to PASS (Full autonomy), 2 maps to CAUTION (Reduced speed, per-motion confirmation), and 3+ maps to FAIL (Hold state, requiring manual "CONFIRM" to resume).

• Autonomy Tiers:

  • Full: Standard teleoperation.
  • Reduced: Executes commands at 50% speed with mandatory operator confirmation for each motion.
  • Hold: Freezes the arm until the operator explicitly types "CONFIRM."

• Segmentation Evolution:

  • Original Approach: Used grayscale adaptive thresholding. This failed significantly, locking onto suture stitches rather than the wound body due to local contrast issues in uniform tissue.
  • Replacement Approach: Implemented a global HSV colour-distance segmentation. It samples the background color from image corners, computes the Euclidean distance of each pixel from the median background, applies Otsu binarization, and uses morphological operations to remove suture artifacts.

Key Contributions

1. Runnable Pipeline: A four-node ROS2 graph that couples wound perception to arm-autonomy gating with clean topic-level separation. The architecture allows perception and control nodes to be replaced independently.
2. Empirical Validation of Failure and Fix: The paper documents a specific failure mode where the initial segmentation method achieved only 6.7% accuracy on a synthetic benchmark because it segmented sutures instead of wounds. It details the implementation and validation of a replacement segmentation method that improved accuracy to 64.4%.
3. Educational Template: A compact (<700 lines of Python) example of a perception-to-actuation loop with operator-in-the-loop gating, suitable for ROS2 education and robotics research.

Results
The system was evaluated on a 45-image synthetic benchmark (15 images each for "Healthy," "Concerning," and "Critical" classes) generated by a deterministic script.

• Original Pipeline: Achieved 6.7% (3/45) three-class verdict accuracy. The confusion matrix showed no consistent diagonal structure; the method failed to identify healthy wounds, often misclassifying them as "concerning" or "critical" due to detecting suture rings.
• Replacement Pipeline: Achieved 64.4% (29/45) accuracy.
  • Healthy Class: 14/15 correctly identified.
  • Critical Class: 15/15 correctly identified.
  • Concerning Class: 0/15 correctly identified (all misclassified as "Healthy").
• Analysis of Residual Failure: The authors attribute the failure to classify "Concerning" wounds to the ABCDE thresholds. These thresholds were hand-tuned against the distributions produced by the broken segmentation. Consequently, they do not discriminate the distributions produced by the working segmentation (e.g., "Concerning" wounds now have high solidity values that trigger a "PASS" verdict). The paper notes that recalibration is required but defers it to avoid overfitting to the small benchmark.

Significance and Claims
The paper explicitly states that Surgical Vision ROS2 is not a clinical tool. It has not been tested on patient data, nor has it been validated on a physical robotic arm. Its significance lies in:

• Research Utility: Providing an open-source template for researchers to couple any perception signal to a MoveIt2-driven arm for variable-autonomy teleoperation.
• Educational Value: Demonstrating a decoupled, topic-based architecture for perception-conditioned control.
• Methodological Transparency: Offering a clear case study of how a specific segmentation failure (locking onto sutures) can cascade into total system failure, and how a methodological pivot (global color distance) can resolve it, while highlighting that fixing segmentation alone is insufficient without recalibrating downstream decision thresholds.

The authors conclude that while the pipeline works as a simulation and research framework, further work (larger benchmarks, threshold recalibration, and testing on non-synthetic imagery) is necessary before any clinical relevance can be considered.