SUREON: A Benchmark and Vision-Language-Model for Surgical Reasoning

The paper introduces SUREON, a large-scale surgical video QA dataset derived from academic lecture narrations, and presents two specialized vision-language models that demonstrate superior surgical reasoning capabilities by explicitly interpreting intent, rationale, and future steps in surgical scenes.

The Gist

Imagine you are trying to teach a robot how to be a surgeon.

For a long time, we taught robots by showing them thousands of pictures and saying, "That is a scalpel," or "That is a gallbladder." It was like teaching a child to identify fruits: "This is an apple. This is a banana." The robot got really good at naming things, but it didn't understand why the surgeon was doing anything. It couldn't tell you, "The surgeon is cutting here because there's a hidden tumor," or "They are stopping because the patient is bleeding."

The paper you shared, SUREON, is like a giant leap forward. It's not just teaching the robot to see; it's teaching the robot to think like a surgeon.

Here is the story of how they did it, broken down into simple parts:

1. The Problem: The Robot Only Knows "What," Not "Why"

Current surgical AI is like a tourist with a phrasebook. It can point at a tool and say, "That's a grasper." But if you ask, "Why did the surgeon switch tools just now?" the robot is lost. It doesn't understand the story of the surgery, the risks, or the decision-making process.

2. The Solution: The "Teacher's Lecture" Library

The researchers realized that the answers were already sitting on YouTube and medical websites, but they were messy. Surgeons give lectures and narrate surgeries to teach students. They say things like, "Now I'm cutting this artery because the lymph node is too big to save it without damaging the vessel."

This narration is pure gold. It contains the reasoning, the intent, and the safety checks. But it's unstructured and hard for a computer to use.

3. The Magic Factory: Turning Lectures into Lessons

To fix this, the team built a "data factory" using a team of specialized AI agents (think of them as a team of very strict editors):

• The Scout: Finds the exact moment in the video where the surgeon's voice matches what is happening on screen (e.g., "I'm cutting now" matches the video of the scissors cutting).
• The Translator: Turns that moment into a quiz question. Instead of just a video, it creates: "Question: Why did the surgeon cut this branch? Answer: Because the lymph node was too big."
• The Editor: Checks the work to make sure the answer is actually in the video and the transcript.

They did this for 206,800 different scenarios, covering 12 different types of thinking, from "What tool is this?" to "What will happen next?" and "Is this safe?"

4. The Training: Two Steps to Genius

They didn't just dump all this data into a robot brain. They trained it in two distinct phases, like a medical student:

• Phase 1: The Textbook Study (Supervised Fine-Tuning):
  They showed the robot (a model called SureonVLM) all the new questions and answers. It learned to memorize the patterns. It became very good at picking the right answer from a list, beating even the biggest, most expensive AI models (like GPT-5) on surgical questions.

• Phase 2: The "Think Aloud" Drill (Reinforcement Learning):
  This is the cool part. They created a second version called SureonVLM-R1. Instead of just giving an answer, they forced the robot to "think out loud" before answering.

  • Old Robot: "Answer: A."
  • New Robot: "Hmm, I see the lymph node is huge. If I leave it, I might hurt the artery. So, I must cut the branch. Therefore, the answer is A."

  They rewarded the robot only when its "thinking" was logical and led to the right answer. This taught the robot to reason, not just guess.

5. The Results: A Surgeon's Mind in a Box

When they tested these new robots:

• Safety First: On questions about safety (e.g., "Is this action dangerous?"), the new models were 30% better than the best general AI models.
• Understanding Intent: They could explain why a surgeon made a tough decision, something previous AIs couldn't do.
• The "Thinking" Token: The most exciting part is that the model actually generates a "Chain of Thought" (a step-by-step explanation) that looks very similar to how a human surgeon explains their logic.

The Big Picture Analogy

Think of previous surgical AI as a photographer. It takes great pictures and can tell you, "That's a knife."

The SUREON model is like a senior surgeon teaching an intern. It doesn't just show you the knife; it explains, "I'm using this knife here because the tissue is slippery, and if I don't cut it now, the patient will bleed."

Why does this matter?
Because in surgery, knowing what is happening isn't enough. You need to know why it's happening to keep patients safe. This paper proves that by listening to how surgeons teach, we can build AI that doesn't just see the operating room, but actually understands the surgery.

Technical Summary

for reasoning and` for the final response).
* Task-specific reasoning (e.g., generating prepositions for forecasting, correct letter sequences for temporal ordering).
* Goal: To encourage coherent, multi-step reasoning trajectories and explicit "Chain-of-Thought" (CoT) generation without direct supervision of the reasoning path during the RL phase.

3. Key Contributions

1. SUREON Dataset: The first large-scale, structured video QA dataset for surgical reasoning, moving beyond fixed ontologies to cover safety, decision rationale, and forecasting.
2. Multi-Agent Curation Framework: A novel pipeline that transforms unstructured expert narrations into high-quality, grounded QA pairs using specialized agents to filter hallucinations and ensure temporal alignment.
3. SUREON Benchmark: A rigorously expert-validated benchmark (354 samples) for evaluating surgical reasoning capabilities.
4. SureonVLM-R1: A reasoning-optimized VLM that demonstrates explicit, interpretable surgical reasoning (e.g., inferring intent from visual cues like "absence of cautery sparks").

4. Results

Performance on SUREON Benchmark

• Accuracy: SureonVLM and SureonVLM-R1 achieved ~84-85% accuracy on the multiple-choice benchmark, significantly outperforming frontier general-domain models (GPT-5.1: ~68%, Gemini 3.1 Pro: ~60%).
• Critical Capabilities:
  • Safety Practice Identification: 92-93% accuracy (vs. 47-62% for commercial models).
  • Decision Reasoning: 98-100% accuracy.
  • Open-Ended Performance: While general models performed better on open-ended semantic matching (LLM-Judge scores), the specialized models showed superior clinical alignment in multiple-choice scenarios.

Performance on Standard Surgical Tasks

• The models were evaluated on standard benchmarks (Cholec80, HeiChole, Endoscapes) for phase recognition, tool detection, and Critical View of Safety (CVS).
• Result: SureonVLM outperformed general-domain models (e.g., 0.63 F1 on Cholec80 phase recognition vs. 0.17 for Qwen3-VL), proving that reasoning training does not degrade fine-grained perceptual abilities.

Ablation Study

• Progressive Adaptation (T+S): The largest performance gain came from progressive fine-tuning on SUREON.
• Open-Ended Training (O): Crucial for improving generative metrics (Exact Match) without sacrificing multiple-choice accuracy.
• CoT Supervision (C): Essential for stabilizing the subsequent GRPO training, even if it didn't directly boost SFT metrics.

5. Significance and Conclusion

• Paradigm Shift: The paper demonstrates that pedagogical data (expert lectures) is a viable and powerful source for training surgical AI, bypassing the need for expensive, manually annotated reasoning datasets.
• Interpretability: SureonVLM-R1 exhibits explicit reasoning behaviors (e.g., inferring that a vessel was sacrificed due to a bulky lymph node), which is critical for safety-critical intra-operative decision support.
• Domain Specificity: The results highlight that general-purpose VLMs, even with strong reasoning capabilities, fail to generalize to the nuanced, high-stakes context of surgery without domain-specific grounding.
• Limitations: The dataset inherits the "teaching bias" of lectures (routine steps may be underrepresented), and the evaluation relies partially on LLM judges which may favor fluency over clinical precision.

In summary, SUREON bridges the gap between surgical perception and reasoning, providing a blueprint for developing AI that can not only "see" the operating room but also "understand" the surgeon's intent and rationale.