Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning

Technical Summary: Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning

Problem Statement
Robot-Assisted Surgery (RAS), particularly with systems like the da Vinci Research Kit (dVRK), faces a fundamental barrier to full automation: inherent kinematic inaccuracies. These errors arise from hardware factors such as hysteresis, mechanical flexibility, and joint clearance, leading to unreliable internal proprioceptive sensors. Consequently, a discrepancy exists between the commanded trajectory and the robot's actual motion. Classical model-based calibration methods (e.g., geometric calibration, gravity compensation) often fail to generalize across different surgical environments and suffer from high implementation costs. While learning-based approaches offer promise, they typically require precise end-effector pose measurements that are difficult to obtain in practice. The core challenge addressed is developing a low-level controller capable of executing precise trajectories despite unreliable internal state data and configuration-dependent kinematic errors.

Methodology
The authors propose a Teacher-Student learning framework designed for sim-to-real transfer, leveraging closed-loop visual feedback to compensate for internal sensor deficits.

1. Teacher Policy (Simulation):

   • Training: Trained using Reinforcement Learning (Proximal Policy Optimization, PPO) within a high-fidelity simulator (NVIDIA Isaac Sim/Orbit-Surgical).
   • Privileged Information: The teacher has access to "privileged" data unavailable on the physical robot, including true joint positions, velocities, and accurate end-effector poses.
   • Architecture: A Multi-Layer Perceptron (MLP) that takes future trajectory waypoints and true state as input to output joint increments.
   • Robustness Fine-Tuning: The teacher undergoes a second training stage involving procedural joint perturbations (noise and bias) to transform it from a perfect executor into a "recovery expert," capable of guiding a student through state deviations.

2. Student Policy (Real-World Deployment):

   • Training: Trained via Imitation Learning using the DAgger (Data Aggregation) algorithm to mitigate distribution shift. The student learns to mimic the teacher's actions but operates under realistic constraints.
   • State Representation: The student fuses unreliable proprioceptive data (biased joint positions/velocities from the dVRK) with reliable external visual feedback. Visual input consists of 2D keypoints tracked on the instrument end-effector.
   • Architecture: A Transformer-based model (4 encoder, 1 decoder layers) inspired by Action Chunking Transformer (ACT), utilizing self-attention to model temporal and multi-modal dependencies.
   • Domain Randomization: Training involves randomizing joint biases (±0.05 rad), camera poses (±2 cm translation, ±10° rotation), and observation noise to ensure the policy learns view-invariant features and generalizes to kinematic offsets.

3. Sim-to-Real Transfer:

   • A pyramidal Lucas-Kanade (LK) optical flow algorithm tracks 2D keypoints in real-time on the physical dVRK.
   • The trained student policy is deployed directly on the physical robot without further fine-tuning, using the visual feedback loop to correct for kinematic errors in real-time.

Key Contributions

• Visuomotor Controller: A low-level controller trained in a teacher-student paradigm that actively compensates for internal kinematic errors by fusing unreliable proprioception with external visual feedback.
• Sim-to-Real Validation: Successful deployment on a physical dVRK platform, demonstrating competitive performance against classical control methods and other learning-based policies (ACT).
• Robustness Demonstration: Experimental evidence showing the policy's ability to generalize across workspace relocations and camera viewpoint variations, establishing a foundation for task-agnostic surgical automation.

Experimental Results
Experiments were conducted on a modified peg-transfer task using the dVRK.

• Simulation Performance: The student policy achieved a position error of ~1.2 mm and orientation error of ~0.03 rad, slightly higher than the teacher (0.6 mm, 0.015 rad) but stable and precise.
• Relocation Generalizability: When the task workspace was shifted horizontally (−4 cm to +6 cm):
  • IK Replay (Baseline): Failed completely (0% success) at offsets > ±2 cm, as it relies on a flawed internal kinematic model that does not account for configuration-dependent errors.
  • Proposed Method: Maintained >80% success rate within ±2 cm and achieved 55% success at the largest offset (+6 cm), demonstrating the ability to learn a mapping from visual error to corrective action.
• Viewpoint Generalizability: Under camera pose variations (translation ±3 cm, rotation ±10°):
  • The proposed method outperformed the ACT baseline significantly (e.g., 40% vs. 15% success at ±10° rotation), attributed to the view-invariant features learned during training.
• Visual Feature Robustness:
  • The policy tolerated moderate Gaussian noise on keypoints (success rate dropped only slightly from 90% to 80%).
  • Performance degraded significantly when 2 keypoints were masked (30% success) and failed entirely (0% success) when 4 keypoints were masked or when visual input was removed entirely, confirming reliance on visual feedback rather than proprioception alone.

Significance and Claims
The paper claims to provide a foundational and reliable control methodology for surgical automation. By shifting from model-based calibration to a learning-based approach that actively compensates for hardware inaccuracies using external vision, the work addresses the critical bottleneck of kinematic uncertainty in surgical robots. The authors assert that this research paves the way for more advanced, robust, and intelligent surgical automation systems that can operate effectively despite the physical limitations of current hardware.

The authors maintain a modest tone regarding limitations, acknowledging that the current visual perception module requires manual initialization, is sensitive to occlusion and lighting, and that the framework currently models linear errors rather than the non-linear systematic errors present in the dVRK. Future work is proposed to address these specific constraints and expand the evaluation to a broader range of surgical tasks.