Technical Summary: Med-Scout

1. Problem Statement: Geometric Blindness in Medical MLLMs

Despite the linguistic fluency and semantic richness of current Multimodal Large Language Models (MLLMs) in medical diagnosis, the paper identifies a critical perceptual deficit termed "geometric blindness." While models can generate clinically plausible text, they frequently fail to ground their outputs in the objective geometric constraints inherent to medical images. This leads to "plausible yet factually incorrect hallucinations," such as mislocalizing lesions to the contralateral side of the body or failing to recognize structural inconsistencies.

The authors attribute this misalignment to prevailing training paradigms (Supervised Fine-Tuning and standard Reinforcement Learning) that prioritize semantic alignment and linguistic likelihood over physical grounding. These methods lack explicit mechanisms to penalize violations of geometric constraints, such as relative anatomical scales, precise topology, and pixel-level structural consistency. Preliminary studies on models like Qwen3-VL and Lingshu revealed that even state-of-the-art MLLMs struggle with:

1. Scale Inconsistency: Failing to recognize the same lesion when viewed at different scales (local crop vs. global view).
2. Topological Blindness: Relying on rigid semantic priors (e.g., "heart is on the left") rather than reasoning about actual spatial positions when images are rotated.
3. Anomaly Insensitivity: Overlooking artificial structural discontinuities (e.g., cut-paste artifacts) in favor of generating standard reports.

Chain-of-Thought (CoT) prompting was found insufficient to cure this deficit, indicating a fundamental lack of low-level geometric reasoning capabilities rather than a mere prompting artifact.

2. Methodology: Med-Scout Framework

Med-Scout is a geometry-aware Reinforcement Learning (RL) post-training framework designed to "cure" geometric blindness by leveraging intrinsic geometric logic from unlabeled medical images. Instead of relying on costly expert annotations, the framework transforms raw medical images into three verifiable geometric proxy tasks inspired by clinical reasoning patterns:

A. Three Strategic Proxy Tasks

1. Hierarchical Scale Localization ($T_{scale}$): Mimics the clinical "zoom-in" diagnostic process. The model must identify the scale level of local patches (Level 1: 20% area; Level 2: 6.25% area) and predict their normalized bounding boxes within the global image. This enforces absolute spatial grounding across scales.
2. Topological Jigsaw Reconstruction ($T_{topo}$): Translates anatomical deduction into a puzzle. The image is partitioned into a $2 \times 2$ grid, shuffled, and the model must reconstruct the original sequence of indices. This demands robust understanding of global anatomical layouts and logical deduction.
3. Anomaly Consistency Detection ($T_{anom}$): Frames fine-grained scrutiny as a "spot-the-difference" game. A patch in a $4 \times 4$ grid is replaced with a structurally incoherent block from a reference image (selected via modality-specific protocols like adjacent slices for CT/MRI or BiomedCLIP similarity for X-ray). The model must identify the inconsistent patch index.

B. Optimization via Dense Geometric Rewards (DGR)

The framework utilizes Group Relative Policy Optimization (GRPO) to optimize these tasks. A key innovation is the Dense Geometric Reward (DGR) mechanism, which replaces sparse binary feedback with continuous, granular rewards based on the degree of geometric deviation:

• Scale: Rewards are calculated based on classification correctness and Intersection over Union (IoU) for bounding boxes.
• Topology: Rewards are based on element-wise alignment, granting partial credit for correctly positioned patches even if the full sequence is imperfect.
• Anomaly: Rewards are computed using a distance-based exponential decay function (Euclidean distance between predicted and ground truth coordinates), encouraging the model to approach the correct spatial region even if the exact index is missed.

The total reward $R$ combines accuracy ($R_{acc}$), format compliance ($R_{fmt}$), and reasoning structure ($R_{reason}$) for CoT modes.

C. Med-Scout-Bench

To rigorously quantify geometric blindness, the authors constructed Med-Scout-Bench, a novel benchmark comprising 10,800 samples (10% of a 108k synthetic dataset). It is strictly balanced across CT, MRI, and X-ray modalities and evaluates the three proxy tasks as open-set VQA queries, assessed via an LLM-as-a-Judge paradigm to ensure robust semantic correctness evaluation.

3. Key Contributions

1. Unveiling Geometric Blindness: The paper provides empirical evidence that current MLLMs, despite semantic fluency, fail to ground outputs in geometric constraints, a deficit that CoT prompting cannot resolve.
2. Geometry-Aware RL Post-Training: The proposal of Med-Scout, a framework that uses specialized proxy tasks and dense geometric rewards to actively align MLLMs with visual logic without expensive annotations.
3. Med-Scout-Bench: The release of a rigorous benchmark specifically designed to quantify geometric perception in medical imaging, addressing a gap in existing evaluation standards.
4. Substantial Performance Gains: Demonstrating that curing geometric blindness leads to significant improvements not only in geometric tasks but also in broader medical VQA and report generation.

4. Experimental Results

Extensive evaluations were conducted on various backbones, including general-purpose models (Qwen3-VL, InternVL) and medical specialists (Lingshu, HuatuoGPT-Vision).

• Med-Scout-Bench Performance: Med-Scout significantly mitigated geometric blindness, improving performance by over 40% on the benchmark. For instance, Qwen3-VL-8B improved from 39.7% to 83.6% average accuracy. Notably, open-source models trained with Med-Scout outperformed leading proprietary models (GPT-5, Gemini-3-Flash).
• Generalization to Medical VQA: The enhanced geometric perception generalized to standard radiological VQA benchmarks (RadImageNet-VQA, VQA-RAD, SLAKE) and comprehensive medical VQA (PMC-VQA, OmniMedVQA, MedXpertQA). Improvements ranged from 1.6% to 4.2% across various tasks.
• Report Generation: In report generation tasks (MIMIC-CXR, IU-Xray), Med-Scout achieved State-of-the-Art (SOTA) results. Lingshu-7B with Med-Scout surpassed proprietary models in CIDEr scores, confirming that geometric accuracy improves the clinical fidelity of generated reports.
• Ablation Studies:
  • Task Necessity: Removing any single proxy task led to performance degradation, confirming the holistic nature of the geometric perception learned.
  • Reward Density: The Dense Geometric Reward (DGR) mechanism outperformed standard sparse binary rewards, proving that granular feedback is crucial for efficient RL post-training.
  • SFT vs. RL: While Supervised Fine-Tuning (SFT) achieved high scores on the internal benchmark, it failed to generalize to external tasks (often showing negative shifts). In contrast, RL-driven optimization consistently improved generalization, indicating that RL fosters true geometric reasoning rather than pattern overfitting.
  • Energy Landscape Analysis: Visualization of the energy landscape showed that Med-Scout establishes a distinct barrier between factual and spatially inverted (hallucinated) descriptions, whereas baseline models exhibited overlapping distributions.

5. Significance and Claims

The paper claims that Med-Scout addresses a fundamental divergence in training priorities: while general MLLMs focus on linguistic fluency, medical applications require strict adherence to geometric constraints. By extracting intrinsic geometric rules from unlabeled data and optimizing via RL, Med-Scout provides a cost-efficient, scalable pathway to reliable clinical AI.

The authors assert that this approach does not merely improve task-specific metrics but fundamentally alters the model's internal representation, enabling it to respect the physical truth of medical images. This "curing" of geometric blindness is presented as a prerequisite for trustworthy medical AI, ensuring that models do not just mimic clinical language but genuinely perceive and reason about the visual evidence in a manner consistent with clinical reality. The work suggests that future medical MLLMs should integrate such geometry-aware optimization to bridge the gap between semantic knowledge and perceptual discipline.