🤖 AI

An Explainable Vision-Language Model Framework with Adaptive PID-Tversky Loss for Lumbar Spinal Stenosis Diagnosis

This paper presents an explainable vision-language model framework that utilizes a Spatial Patch Cross-Attention module and a novel Adaptive PID-Tversky Loss to achieve high-accuracy, interpretable diagnosis and automated report generation for Lumbar Spinal Stenosis from MRI scans, effectively addressing class imbalance and spatial precision challenges in clinical settings.

Original authors: Md. Sajeebul Islam Sk., Md. Mehedi Hasan Shawon, Md. Golam Rabiul Alam

Published 2026-04-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Md. Sajeebul Islam Sk., Md. Mehedi Hasan Shawon, Md. Golam Rabiul Alam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery inside a person's lower back. The "crime scene" is a set of MRI scans (detailed pictures of the spine), and the "mystery" is Lumbar Spinal Stenosis (LSS)—a condition where the space in the spine gets too narrow, squeezing the nerves and causing pain.

Usually, a human detective (a radiologist) has to stare at these pictures for hours, looking for tiny clues. It's tiring, and sometimes two detectives might disagree on what they see.

This paper introduces a super-smart AI assistant designed to help these detectives. Here is how it works, explained through simple analogies:

1. The Detective's New Toolkit: "The Translator"

Most old AI models were like a camera that just took a picture and said, "Something is wrong here." They couldn't explain why.

This new framework is like a bilingual translator who speaks both "Picture" and "Doctor."

The Vision Part: It looks at the MRI scan like a hawk, spotting the exact shape of the spinal canal.
The Language Part: It reads medical textbooks and knows the words doctors use.
The Magic: It connects the two. Instead of just drawing a line around the problem, it writes a report saying, "I see the spinal canal is squeezed here, which matches the description of 'severe stenosis'."

2. The "Smart Spotlight": Finding the Needle in the Haystack

Medical data is tricky. Imagine you have a giant bag of marbles. 90% are red (healthy spines), and only 10% are blue (sick spines). If you train a robot to find the blue ones, it gets lazy. It just guesses "Red" every time because it's right 90% of the time, but it misses the sick patients.

The authors invented a Dynamic Spotlight (called the Adaptive PID-Tversky Loss).

The Analogy: Think of a teacher grading a test. If a student gets the easy questions right, the teacher doesn't care. But if they get the hard questions wrong, the teacher focuses all their energy there.
How it works: This AI "teacher" constantly checks its own work. If it keeps missing the difficult, rare cases (the sick spines), it automatically turns up the volume on those specific errors. It forces the AI to stop ignoring the hard stuff and focus on the tricky, narrow parts of the spine that are easy to miss.

3. The "High-Res Map": Not Just a Blurry Photo

Old AI models often looked at the whole spine and took a "big picture" guess. It's like looking at a map of a country and trying to find a specific street address. You might know the country, but you'll miss the house.

This new system uses a Spatial Patch Cross-Attention module.

The Analogy: Instead of looking at the whole map, this AI puts a magnifying glass on tiny, specific patches of the image. It zooms in on the exact spot where the nerve is being squeezed.
The Result: It doesn't just say "There's a problem." It says, "The problem is right here, in this specific 10x10 pixel square," and then uses the text prompt to confirm, "Yes, this looks like a compressed nerve."

4. The "Auto-Reporter": Writing the Diagnosis

Once the AI finds the problem, it doesn't just give a number. It writes a radiology report that looks like it was written by a human doctor.

The Analogy: Imagine a robot that doesn't just say "Error 404," but instead writes a letter: "Dear Doctor, the patient's spinal canal is 40% narrower than normal. This is causing pressure on the nerve roots. I recommend surgery."
Why it matters: This makes the AI explainable. A human doctor can read the report, understand the logic, and trust the AI's suggestion.

The Results: How Good is it?

The team tested this system on thousands of MRI scans:

Accuracy: It correctly identified the severity of the disease about 91% of the time.
Precision: It could draw the outline of the squeezed area with 95% accuracy (like drawing a perfect circle around a target).
Reports: The reports it wrote were so good that they scored 93% on a scale of how well they matched human-written reports.

The Bottom Line

This paper presents a team-up between a super-observant eye and a fluent writer.

It solves the problem of AI being a "black box" (you don't know how it thinks) by making it write reports.
It solves the problem of AI missing rare diseases by using a "smart spotlight" that forces it to pay attention to the hard cases.
It acts as a co-pilot for doctors, doing the heavy lifting of scanning and measuring, so the human doctor can focus on making the final, life-changing decision.

It's not about replacing the doctor; it's about giving the doctor a pair of super-glasses and a super-scribe to make sure no patient gets left behind.

1. Problem Statement

Lumbar Spinal Stenosis (LSS) diagnosis relies heavily on the manual interpretation of multi-view Magnetic Resonance Imaging (MRI) scans. This process is labor-intensive, prone to inter-observer variability, and often results in diagnostic delays. Existing deep learning solutions face three critical limitations:

Black Box Nature: Traditional Convolutional Neural Networks (CNNs) provide raw classification scores or pixel masks without transparent clinical reasoning, hindering trust in medical decision-making.
Class Imbalance: Medical datasets often suffer from severe class imbalance (e.g., fewer severe cases than normal ones). Standard loss functions (like Cross-Entropy) cause models to overfit majority classes, failing to segment complex minority cases.
Loss of Spatial Hierarchy: Many Vision-Language Models (VLMs) rely on global pooling mechanisms (e.g., [CLS] tokens) that discard fine-grained spatial details crucial for delineating spinal canal boundaries.
Lack of Explainability: Current models struggle to bridge the gap between quantitative visual predictions and qualitative, radiologist-style clinical reports.

2. Methodology

The authors propose an end-to-end Explainable Vision-Language Framework comprising five key stages:

A. Data Preprocessing & Pseudo-Mask Generation

Denoising: Raw MRI images undergo Non-Local Means (NLM) filtering to remove scanner artifacts while preserving sharp anatomical edges.
Adaptive Pseudo-Mask Generation: Since pixel-level annotations are scarce, the framework generates "pseudo-ground truth" masks by combining Otsu's thresholding and Canny edge detection. Crucially, these masks are refined using morphological operations conditioned on radiologist reports (e.g., stronger dilation for severe stenosis reports, localized erosion for mild cases) to integrate expert clinical knowledge.

B. Multimodal Feature Extraction

The framework integrates advanced VLMs (LLaVA-Med, BiomedCLIP, SmolVLM) using two concurrent channels:

Text Channel: Clinical prompts describing stenosis grades are tokenized and encoded (using PubMedBERT, SmolLM2, or Vicuna) into semantic embeddings.
Vision Channel: MRI scans are processed via Vision Transformer (ViT) backbones. Unlike standard VLMs, the authors exclude the global [CLS] token and instead extract spatial patch embeddings (196 patches for a 14×14 grid) to retain fine-grained anatomical details.

C. Spatial Patch Cross-Attention Module

To bridge the semantic gap between text and vision:

A custom 8-head Cross-Attention mechanism is designed.
Queries ( $Q$ ) are derived from clinical text embeddings.
Keys ( $K$ ) and Values ( $V$ ) are derived from the spatial patch embeddings.
This allows the model to perform text-guided localization, directing attention to specific MRI regions corresponding to the clinical description. The output is reshaped into a 2D spatial grid and upsampled via a CNN decoder to generate the final segmentation mask.

D. Adaptive PID-Tversky Loss Function

To address class imbalance, the authors introduce a novel loss function inspired by Proportional-Integral-Derivative (PID) control theory:

Mechanism: The loss dynamically adjusts penalty weights ( $\alpha$ and $\beta$ ) for False Positives and False Negatives based on the model's current error rates (FP and FN) at each epoch.
Control Logic: It acts as a feedback loop. If the model under-segments minority classes (high FN), the controller increases the penalty weight for those errors, forcing the network to focus on difficult, under-represented stenosis patterns.
Formula: Combines the Tversky Index with a focal parameter ( $\gamma$ ) and PID-based weight adaptation.

E. Automated Radiology Report Generation (ARRG)

The framework includes a module that converts segmentation masks and quantitative metrics (e.g., affected area percentage) into natural language.
It maps the calculated stenosis severity to a template (Grade A–D) and uses an autoregressive language model to generate multi-line, radiologist-style clinical narratives.

3. Key Contributions

End-to-End Multimodal Framework: A unified system that performs severity classification, precise segmentation, and automatic report generation simultaneously.
Adaptive PID-Tversky Loss: A novel loss function that dynamically mitigates class imbalance in medical segmentation, outperforming static loss functions by focusing learning on difficult minority cases.
Spatial Patch Cross-Attention: A module that replaces global pooling with patch-level attention, enabling text-guided, high-precision localization of spinal anomalies.
Explainability: The system translates complex AI predictions into interpretable clinical reports, establishing a "human-in-the-loop" workflow.

4. Experimental Results

The framework was evaluated on a dataset of 515 patients (48,345 images) using three VLM backbones.

Classification Performance:
- LLaVA-Med achieved the highest accuracy (90.69%) and macro F1-score (0.9072).
- It demonstrated exceptional discrimination for extreme cases (Grade A and D), with AUCs of 0.991 and 0.993, respectively.
Segmentation Performance:
- LLaVA-Med achieved a Dice Score of 0.9624 and IoU of 0.9288.
- BiomedCLIP followed closely with a Dice of 0.9512.
- SmolVLM (lightweight) showed lower absolute accuracy (Dice 0.8603) but exhibited superior cross-grade stability (variance < 0.0011), performing consistently across all severity levels.
Report Generation (ARRG):
- LLaVA-Med led in linguistic fluency (CIDEr: 92.80%, METEOR: 91.05%).
- SmolVLM showed strong semantic alignment and resource efficiency, outperforming BiomedCLIP in several metrics.
Ablation Study:
- Replacing the proposed Adaptive PID-Tversky Loss with standard Cross-Entropy resulted in significant performance drops (e.g., BiomedCLIP Dice dropped from 0.9512 to 0.7411), confirming the loss function's critical role in handling class imbalance.

5. Significance and Conclusion

This study presents a significant advancement in medical AI by addressing the "black box" problem and the class imbalance challenge simultaneously.

Clinical Impact: The framework provides not just a diagnosis but a justification (via segmentation masks) and a narrative (via automated reports), facilitating trust and adoption in clinical workflows.
Technical Innovation: The integration of control theory (PID) into deep learning loss functions offers a robust solution for imbalanced medical data without requiring massive data augmentation.
Future Outlook: While the system shows high accuracy, the authors emphasize the necessity of human-in-the-loop validation to ensure clinical safety. The framework serves as a benchmark for transparent, interpretable AI that enhances rather than replaces radiologist expertise.