💻 computer science

Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation

This paper proposes a Vision-Language Procedural Reasoning (VL-PR) framework that leverages a multimodal large language model to dynamically adapt reward functions based on real-time anatomical context, thereby enhancing the reliability and efficiency of autonomous robotic guidewire navigation in complex vascular environments.

Original authors: Wentong Tian, Jiyuan Zhao, Tianliang Yao, Yuxiang Fan, Zhengyu Shi, Dong Liu, Peng Qi

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wentong Tian, Jiyuan Zhao, Tianliang Yao, Yuxiang Fan, Zhengyu Shi, Dong Liu, Peng Qi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to navigate a tiny, flexible snake (a guidewire) through a complex, winding maze of pipes inside a human body. The goal is to get from the starting point to a specific target without hitting the walls or getting stuck. Doing this manually is hard, and even robots struggle because the "rules" for moving change constantly. Sometimes you need to zoom straight ahead; other times, you need to be incredibly careful at a fork in the road; and if you hit a wall, you need to back up immediately.

This paper introduces a new way to teach robots how to do this job. They call it the Vision-Language Procedural Reasoning (VL-PR) framework. Here is how it works, broken down into simple concepts:

1. The Problem: The Robot's "Static" Brain

Usually, robot training is like teaching a dog with a single, unchanging set of rules. For example, "Move forward fast" might be the rule for the whole trip. But in a vascular maze, that doesn't work.

The Issue: If the robot tries to move fast when it's approaching a sharp turn or a split in the pipes, it crashes. If it tries to be super slow when it's in a straight, safe hallway, it wastes time.
The Old Way: Most robots use a "static reward." They get points for moving forward and lose points for hitting walls, but the importance of these points never changes. It's like driving a car where the speed limit is always 60 mph, even when you are entering a school zone or merging onto a highway.

2. The Solution: The "Co-Pilot" with a Brain

The authors added a special "Co-Pilot" to the robot. This Co-Pilot is a Multimodal Large Language Model (MLLM). Think of this as a highly experienced human navigator who can look at the X-ray images (vision) and understand the medical instructions (language).

What the Co-Pilot Does: It doesn't grab the steering wheel. Instead, it watches the robot's journey and says, "Okay, right now we are in a straight hallway," or "Oh no, we are at a fork in the road," or "We hit a wall, we need to back up."
The Magic: It translates what it sees into a "context." It tells the robot, "Right now, safety is the most important thing," or "Right now, speed is the most important thing."

3. The Mechanism: The "Dynamic Scorecard"

The robot's learning system uses a "reward function" (a scorecard) to decide what to do.

Without the Co-Pilot: The scorecard is fixed. "Moving forward = +10 points. Hitting wall = -10 points."
With the Co-Pilot (VL-PR): The scorecard changes dynamically based on the Co-Pilot's advice.
- In a straight hallway: The Co-Pilot says, "Go!" The scorecard changes to give huge points for moving fast and ignores minor safety concerns.
- At a fork: The Co-Pilot says, "Be careful." The scorecard changes to give huge points for staying in the center and avoiding walls, even if it means moving slower.
- If a mistake happens: The Co-Pilot says, "Retreat." The scorecard changes to reward backing up and stopping the damage.

This allows the robot to use one single brain (policy) to handle the whole journey, but it shifts its priorities instantly as the situation changes, just like a human expert would.

4. The Results: A Faster, Smarter Robot

The team tested this on a physical robot using a realistic plastic model of human blood vessels (a "phantom"). They tried three different types of vessels: coronary (heart), carotid (neck), and renal (kidney).

Success Rate: The new method was a clear winner. It successfully navigated the robot to the target 100% of the time in heart and kidney scenarios, and 97% in the tricky neck scenario.
Comparison: Older robot methods (without the Co-Pilot) only succeeded about 70% of the time.
Efficiency: The new robot was also much faster. It took fewer steps to reach the target. For example, in the heart scenario, it took about 41 steps, while the old methods took over 80 steps. It was like the robot found a shortcut because it knew exactly when to speed up and when to slow down.

Summary Analogy

Imagine playing a video game where the rules change every second.

Old Robots: They keep trying to run at full speed because that's what they were trained to do. They crash into walls at the turns.
This New Robot: It has a smart friend (the MLLM) whispering in its ear. When the path is straight, the friend says, "Run!" When the path gets twisty, the friend says, "Walk carefully." When it hits a wall, the friend says, "Back up!"
The Result: The robot finishes the level faster and never crashes, beating the old robots and even performing as well as a human expert.

The paper concludes that this "Vision-Language Procedural Reasoning" makes robotic surgery navigation more reliable, efficient, and safe by giving the robot the ability to understand where it is in the procedure and adjust its behavior accordingly.

Technical Summary: Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation

Problem Statement
Robotic-assisted endovascular interventions require precise, stable navigation of guidewires through complex, patient-specific vascular anatomies. While reinforcement learning (RL) has emerged as a framework for mapping sensory feedback to actuation, current autonomous navigation methods face significant limitations. Most existing approaches rely on static reward functions optimized across the entire procedural continuum. This static formulation fails to address the heterogeneous, phase-dependent demands of real-world navigation, where operational modes must shift dynamically (e.g., rapid progression in straight segments versus strategic steering at bifurcations). Furthermore, existing frameworks lack a mechanism to integrate high-level contextual reasoning to provide stage-aware feedback, leading to suboptimal trade-offs between efficiency, safety, and precision in long-horizon tasks.

Methodology
To address these challenges, the authors propose a Vision-Language Procedural Reasoning (VL-PR) framework. This approach integrates a Multimodal Large Language Model (MLLM) as a procedural reasoning module within a unified RL pipeline. The core innovation lies not in generating low-level control commands directly, but in using the MLLM to infer high-level navigation contexts that dynamically adapt the reward function.

Procedural Reasoning Module (MLLM):
- The MLLM (specifically a Qwen2.5-VL-3B-Instruct backbone, fine-tuned via 4-bit QLoRA) processes real-time multimodal observations, including local vascular images, guidewire tip configurations, and geometric cues (centerline direction, curvature, bifurcations).
- It infers a discrete procedural state $g_t$ from a defined space: $\{cruise, bifurcation, retraction, final\ align\}$ .
- The module is trained using expert-derived semantic labels with a loss function combining categorical cross-entropy, temporal consistency regularization, and entropy regularization to prevent overconfident predictions in ambiguous regions.
Context-Aware Reward Adaptation:
- Instead of switching policies or augmenting policy inputs with semantic variables, the framework maintains a single unified control policy (implemented via Soft Actor-Critic, SAC).
- The reward function $r_t$ is decomposed into four interpretable components: progress ( $\phi_{prog}$ ), safety ( $\phi_{safe}$ ), deviation from centerline ( $\phi_{dev}$ ), and bending stability ( $\phi_{bend}$ ).
- The MLLM's inferred procedural distribution $p_\psi(g_t | o_t)$ $p_{ψ} (g_{t} ∣ o_{t})$ modulates a state-dependent weight matrix $W_g(\xi_t)$ $W_{g} (ξ_{t})$ . This matrix dynamically adjusts the importance of reward components based on the current phase and a risk indicator $\xi_t$ $ξ_{t}$ .
  - Cruise: Emphasizes efficiency; safety penalties increase with risk.
  - Bifurcation: Prioritizes geometric alignment and steerability.
  - Retraction: Suppresses progress rewards to encourage withdrawal when wall contact or failure risks are high.
  - Final Align: Shifts priority to geometric precision.
Training and Deployment:
- Training occurs in a high-fidelity physics-based simulation (SOFA framework) acting as a digital twin of a physical vascular phantom.
- The learned policy is deployed on a physical robotic platform (ALLVAS™ system) without additional fine-tuning (zero-shot deployment).
- A visual perception pipeline (YOLOv5 for detection, U-Net for segmentation) estimates guidewire states from monocular camera frames to provide real-time state observations.

Key Contributions

VL-PR Framework: Establishment of a framework bridging high-level multimodal reasoning with RL to facilitate intelligent decision-making across complex vascular anatomies.
Context-Aware Reward Adaptation: Introduction of a strategy to dynamically prioritize competing objectives (e.g., balancing traversal efficiency with safety-critical precision) based on inferred procedural states.
Physical Validation: Demonstration on a physical robotic system that procedural reasoning enhances task reliability and navigational efficiency in realistic vascular phantoms and challenging anatomical structures.

Experimental Results
The framework was evaluated across three distinct vascular scenarios: coronary, carotid, and renal interventions.

Success Rate: The proposed VL-PR method achieved a 100% success rate in coronary and renal interventions and 97% in carotid interventions. This significantly outperformed standard RL baselines (SAC, PPO, TD3, DDPG), which ranged from 60% to 73% success rates. The performance matched that of expert manual operation (93–100%).
Efficiency: The method reduced the average number of navigation steps significantly. For coronary navigation, the proposed method required 41.6 steps compared to 80.6 steps for standard SAC (a 48.4% reduction).
Ablation: Removing the MLLM reasoning module (using a rule-based baseline) resulted in a performance drop, with success rates falling to 73–77% and step counts increasing significantly, confirming the critical role of context-aware reward adaptation.
Reasoning Accuracy: The MLLM achieved procedural state classification accuracies between 91.2% and 94.0%, vastly outperforming rule-based baselines (22–27%).

Significance and Claims
The paper claims that the VL-PR framework offers a scalable solution for complex, multi-task robotic endovascular procedures by enabling a single policy to resolve competing objectives while maintaining a consistent global task goal. By decoupling high-level procedural reasoning from low-level motion control, the approach ensures consistency between training and deployment while improving robustness across diverse anatomies. The authors assert that this method enhances task reliability and streamlines navigational efficiency compared to static-reward methods, providing a pathway toward more autonomous and intelligent robotic interventions that can adapt to the varying challenges of real-world vascular navigation. Future work is identified as extending reasoning to richer structured representations, improving sim-to-real generalization, and incorporating formal safety constraints.