💻 computer science

RVLM: Recursive Vision-Language Models with Adaptive Depth

The paper introduces RVLM, a unified framework that enhances medical AI by combining an iterative generate-execute loop for auditable, code-grounded reasoning with an adaptive depth controller (RRouter) that optimizes computational resources based on task complexity, demonstrating high consistency and accuracy on brain MRI and chest X-ray datasets.

Original authors: Nicanor Mayumu, Zeenath Khan, Melodena Stephens, Patrick Mukala, Farhad Oroumchian

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nicanor Mayumu, Zeenath Khan, Melodena Stephens, Patrick Mukala, Farhad Oroumchian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a complex medical mystery, like finding a hidden tumor in a brain scan or spotting a problem in a chest X-ray.

The Old Way (The "Black Box" Detective):
Currently, most AI doctors work like a magician pulling a rabbit out of a hat. You show them the picture, and poof—they give you an answer. "There is a tumor." But they won't tell you how they found it, what clues they looked at, or why they are sure. If a human doctor asks, "How did you know?" the AI just says, "My computer brain told me." This is dangerous in medicine because doctors need to trust the evidence, not just the result.

The New Way (RVLM: The "Methodical Detective"):
This paper introduces a new system called RVLM (Recursive Vision-Language Model). Instead of guessing the answer in one go, RVLM acts like a super-organized detective who writes down every single step of their investigation in a notebook (which happens to be computer code).

Here is how it works, using simple analogies:

1. The "Do-It-Yourself" Toolkit (The REPL)

Imagine the AI isn't just looking at the X-ray with its eyes; it has a toolbox.

The Old AI: Looks at the whole picture and guesses.
The RVLM: Takes the picture, zooms in on a specific spot, cuts out a piece to look closer, brightens the contrast, and compares it to a different type of scan. It writes a little script (code) to do this: "Okay, let's crop the left side of the brain, make it brighter, and ask a sub-AI, 'Do you see a tumor here?'"
The Result: Every claim the AI makes is backed by a "receipt" (the code). If a doctor wants to check the work, they can run the code again and see exactly what the AI saw. It's like showing your math homework instead of just the final answer.

2. The "Smart Manager" (RECURSIONROUTER)

The second problem the paper solves is about efficiency.

The Old Way: Imagine a manager who tells every employee, "You must work for exactly 12 hours, no matter what."
- If the task is easy (like finding a small, clear tumor), the employee wastes 9 hours doing nothing.
- If the task is hard (like a messy, complex tumor), 12 hours isn't enough, and they run out of time before solving it.
The RVLM Way: This system has a Smart Manager (called RECURSIONROUTER). Before the work starts, the manager looks at the case and asks: "How hard is this?"
- Easy Case: "This looks simple. Let's just do 3 quick checks and stop." (Saves time and money).
- Hard Case: "This looks messy. Let's plan for 6 or 7 deep checks to make sure we don't miss anything."
- The "Stall" Check: The manager also watches the work. If the AI gets stuck and keeps repeating the same thing without finding new clues, the manager says, "Stop! We aren't learning anything new," and ends the session early.

3. The "Translator" (The Report Generator)

Since the AI's "notebook" is full of computer code (which looks scary to humans), the system has a final step: a Translator.

It takes all the code, the zoomed-in images, and the step-by-step logic, and turns it into a clean, professional medical report that a human doctor can read.
It even adds a little note at the bottom: "This was generated by AI; please double-check."

Why Does This Matter?

Trust: Doctors can see the "workings" of the AI. They can verify the math and the logic.
Safety: It prevents the AI from "hallucinating" (making things up) because it has to prove its findings with code.
Speed & Cost: It doesn't waste money on easy cases, but it doesn't give up on hard ones either.

In a Nutshell:
RVLM turns the AI from a mysterious oracle that gives answers into a transparent, hard-working intern that shows its homework, asks for help when stuck, and knows exactly when to stop working so it doesn't waste time. It's a step toward making AI a reliable partner in the hospital rather than a scary black box.

1. Problem Statement

The paper identifies two critical limitations in current Medical AI systems, specifically Vision-Language Models (VLMs):

The Interpretability Challenge: Conventional VLMs operate via single-pass inference, producing "black-box" predictions without revealing intermediate reasoning steps. This opacity conflicts with clinical evidentiary standards and regulatory frameworks (e.g., EU AI Act, ISO/IEC 42001) that require auditability, traceability, and the ability for clinicians to verify the logic behind a diagnosis.
The Fixed-Budget Problem: Existing iterative reasoning systems (e.g., Chain-of-Thought, ReAct) rely on a fixed iteration budget (a static max_iterations hyperparameter). This approach is inefficient: it wastes computational resources on simple cases (over-iteration) and provides insufficient depth for complex, ambiguous cases (under-iteration). The optimal reasoning depth is a property of the specific medical task, not a system-wide constant.

2. Methodology

The authors propose a unified framework combining two core components: RVLM (the recursive reasoning engine) and RECURSIONROUTER (the adaptive depth controller).

A. RVLM: Recursive Vision-Language Model

RVLM replaces single-pass inference with an iterative generate-execute loop embedded in a persistent Python REPL (Read-Eval-Print Loop) environment.

Architecture: A vision-capable root Large Language Model (LLM) acts as the controller within a stateful Python environment.
First-Class Image Support: The environment treats images as objects, not just text tokens. It provides specific primitives:
- context_images: Maintains a working memory of images.
- describe_image(): Generates natural language descriptions of specific regions.
- llm_query_with_images(): Enables visual question answering and cross-modal comparison.
- Programmatic Manipulation: The model can write Python code to crop images, apply contrast enhancement (CLAHE), compute difference maps (e.g., T1ce minus T1n), and extract quantitative metrics.
Workflow:
1. Modality-Aware Decomposition: The model issues targeted descriptions for each imaging modality (e.g., T1, T2, FLAIR for MRI; PA, Lateral for X-ray) separately to prevent attention dilution.
2. Cross-Modal Synthesis: It compares independent descriptions to detect discrepancies (e.g., a hyperintense ring in FLAIR vs. no edema in the segmentation mask).
3. Evidence Accumulation: Every diagnostic claim is grounded in executable code, creating a verifiable trace.
4. Reporting: A secondary LLM pass converts the code-based reasoning trace into a formal, structured clinical PDF report.

B. RECURSIONROUTER: Adaptive Iteration Depth

This component dynamically determines the optimal number of reasoning steps ( $n^*$ ) for a specific case, eliminating the need for a fixed hyperparameter.

Complexity Estimation: Before execution, the router analyzes task features to predict complexity. For brain tumors (BraTS), features include:
- Label Entropy ( $H$ ): Shannon entropy of sub-region volume distribution (NCR, ED, ET).
- Total Volume ( $V$ ): Larger tumors require more analysis.
- Sub-region Count ( $R$ ): Number of distinct tumor components.
- Tiny Region Indicator ( $T$ ): Presence of hard-to-characterize small regions.
Budget Mapping: These features are weighted to produce a complexity score $s \in [0, 1]$ , which maps to a recommended iteration budget (e.g., 3 iterations for simple cases, up to 6 for complex ones).
Stall Detection: During execution, the router monitors progress. If two consecutive iterations are unproductive (no new variables or meaningful output) and the budget is met, the loop terminates early to save compute.

3. Key Contributions

Executable Reasoning Framework: Extends the Recursive Language Model (RLM) paradigm to multimodal vision, introducing primitives for visual reasoning and programmatic image manipulation. Every claim is backed by reproducible Python code.
Adaptive Computation: Introduces a pre-flight complexity estimator and per-iteration stall detector that treats recursion depth as a latent variable inferred from input features, removing the fixed-budget limitation.
Regulatory Alignment: Maps system mechanisms directly to clinical AI governance requirements (EU AI Act, NIST AI RMF), demonstrating how "interpretability" can be architected into the system rather than added post-hoc.
Zero-Shot Generalization: The framework operates without task-specific fine-tuning, adapting to different modalities (Brain MRI vs. Chest X-ray) solely through prompt engineering and modality-aware decomposition.

4. Experimental Results

The system was evaluated using Gemini 2.5 Flash on two distinct benchmarks without fine-tuning:

BraTS 2023 Meningioma (Brain MRI):
- Consistency: Achieved 90% consistency on high-salience findings (e.g., mass presence, enhancement) across 16 independent runs.
- Cross-Modal Detection: Successfully detected discrepancies between signal characteristics (e.g., FLAIR hyperintensity) and segmentation boundaries, a capability absent in single-pass baselines.
- Efficiency: For a simple single-region case, the router reduced the iteration budget from a fixed 12 to 3, a four-fold reduction in cost.
MIMIC-CXR (Chest X-Ray):
- Generated structured "Findings" and "Impression" reports.
- Correctly identified view-specific artifacts (e.g., cardiac magnification in AP portable views) and recognized view-specific pathologies.
Comparison: RVLM outperformed single-pass VLMs in providing detailed radiological features (e.g., "dural tail sign") and producing an auditable trail of code and intermediate variables.

5. Significance and Impact

Clinical Trust: By grounding diagnoses in executable code, RVLM provides a "procedural transparency" that allows clinicians to audit the reasoning path, satisfying high-risk regulatory requirements for medical AI.
Operational Efficiency: The adaptive depth mechanism ensures that computational resources are matched to case complexity, reducing costs for simple screenings while maintaining depth for complex diagnostics.
Future-Proofing: The "prompt as clinical protocol" design allows the system to adapt to new imaging modalities (e.g., CT, Histopathology) simply by updating the prompt templates, without retraining the model weights.
Open Source: The authors have released the codebase to encourage community extension to new modalities and large-scale quantitative benchmarking.

In conclusion, RVLM represents a shift from "black-box" prediction to transparent, auditable, and adaptive reasoning, offering a principled foundation for deploying trustworthy AI in high-stakes clinical environments.