MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models

MedPruner is a training-free, model-agnostic hierarchical token pruning framework that eliminates anatomical redundancy in 3D medical vision-language models through inter-slice filtering and dynamic token selection, enabling significant computational efficiency while maintaining or improving performance with fewer than 5% of visual tokens.

The Gist

Imagine you are a doctor trying to diagnose a patient using a 3D CT scan. A CT scan isn't just one picture; it's a stack of hundreds of thin slices, like a loaf of bread. To understand the whole patient, an AI needs to look at every single slice.

The problem? It's too much bread.

Current AI models try to eat the whole loaf at once. They look at every single slice, even the ones that look exactly like the one before it (like the middle of a loaf where nothing interesting is happening). This overwhelms the AI's brain, making it slow, expensive to run, and sometimes confused by all the repetitive "background noise."

Enter MedPruner. Think of MedPruner as a super-smart, efficient sous-chef who helps the AI chef prepare the meal without burning out the kitchen.

Here is how it works, broken down into simple steps:

1. The "Loaf of Bread" Problem

Imagine you have a 100-page report, but pages 10 through 90 are just blank white space. If you force a reader to read every single page, they will get tired and miss the important stuff on page 5 and page 95.

• The Old Way: The AI reads every single page (slice) of the medical scan, even the blank ones.
• The MedPruner Way: It quickly scans the stack and says, "Hey, these 50 pages are identical. Let's skip them and only read the ones where the picture actually changes."

2. The Two-Step Magic Trick

MedPruner uses a two-step process to trim the fat without losing the meat:

Step A: The "Anchor" Filter (Skipping the Boring Slices)
Imagine you are watching a movie where the camera is zoomed in on a character's face. For 10 seconds, the face doesn't move.

• MedPruner picks the first frame as an "Anchor" (a reference point).
• It looks at the next frame. If the face hasn't moved much, it says, "No need to watch this one; I already know what it looks like." It skips it.
• It only keeps the frames where something new happens (like the character turning their head).
• Result: Instead of watching 100 frames, the AI only watches the 10 frames where the action happens.

Step B: The "Spotlight" Filter (Zooming in on the Details)
Now, even within the important slices, not every pixel matters. In a slice showing a tumor, the tumor is the star, but the healthy tissue around it is just background.

• MedPruner asks the AI: "Where are you looking?"
• It uses a Dynamic Spotlight. If the AI is paying 90% of its attention to a tiny spot (the tumor), MedPruner says, "Great, keep that spot! We can throw away the rest of the pixels in this slice because they aren't important right now."
• If the AI is looking at a complex area with many details, MedPruner keeps more pixels.
• Result: It keeps the "nucleus" (the core) of the information and discards the fluff.

3. Why This is a Big Deal

• It's Training-Free: You don't need to teach the AI a new way to learn. You just give it this "sous-chef" tool to organize its work. It works with any AI model out of the box.
• It's Fast: By cutting out the redundant slices and pixels, the AI runs much faster. In tests, it reduced the amount of data the AI had to process by 95% (keeping less than 5% of the original tokens) while still getting the diagnosis right.
• It's Smarter: Old methods tried to cut a fixed amount (e.g., "cut 50% of everything"). MedPruner is flexible. If a slice is boring, it cuts 99%. If a slice is complex, it cuts only 20%.

The Bottom Line

MedPruner is like a smart filter for medical AI. It stops the AI from drowning in a sea of repetitive data. Instead of forcing the AI to read the whole encyclopedia, it gives it a perfectly summarized cheat sheet that contains all the critical facts and none of the filler.

This means doctors can get AI assistance faster, cheaper, and more reliably, even for complex 3D scans, without needing supercomputers to do the math.

Technical Summary

1. Problem Statement

While specialized Medical Vision-Language Models (VLMs) have achieved success in 2D and 3D medical imaging, their deployment for 3D volumetric data (e.g., CT, MRI) faces severe computational bottlenecks. The authors identify two primary limitations in current architectures:

• Massive Anatomical Redundancy: Current methods typically slice 3D volumes into 2D images and directly concatenate the resulting tokens. Since consecutive slices in 3D volumes share extreme spatial similarity, this approach introduces massive redundancy, exhausting the Large Language Model's (LLM) context window and hindering the processing of auxiliary clinical information.
• Inflexibility of Static Pruning: Existing token pruning methods rely on fixed, predefined pruning ratios. This fails to account for the heterogeneity of information density across slices. For instance, a slice containing a tumor boundary requires high token retention, while a slice of uniform tissue does not. Fixed ratios either risk losing critical pathological details or waste computational resources on irrelevant backgrounds. Furthermore, static approaches ignore the variance in attention distributions across different vision backbones.

2. Methodology: MedPruner

The authors propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework. It operates via a two-stage mechanism designed to reduce redundancy at both the slice level and the token level.

A. Inter-slice Anchor-based Filtering (IAF)

This module addresses slice-level temporal redundancy.

• Mechanism: Instead of fixed-interval sampling, IAF uses a dynamic, content-aware strategy. It maintains a dynamic "anchor" slice ($I_{anc}$).
• Process: As the model traverses the volume, it calculates the pixel-wise mean L1 distance ($\Delta$) between the current slice and the active anchor.
• Decision Logic:
  • If $\Delta > \gamma$ (a sensitivity threshold), the slice contains significant new anatomical features. It is preserved and becomes the new anchor.
  • If $\Delta \le \gamma$, the slice is deemed redundant and filtered out.
• Outcome: The dense volume is distilled into a sparse subsequence of only the most informative anchor frames, drastically reducing the sequence length before token-level processing.

B. Dynamic Information Nucleus Selection (DINS)

This module addresses token-level redundancy within the preserved slices.

• Mechanism: It derives token importance directly from the self-attention layers of the vision encoder.
• Process:
  1. Attention Aggregation: Attention scores are averaged across all heads to create an aggregated attention matrix.
  2. Importance Scoring: The average attention along the sequence dimension yields an initial importance vector, which is normalized using a temperature-scaled softmax.
  3. Nucleus Selection: Instead of a fixed ratio, the method sorts tokens by importance and selects the minimal set of top-ranked tokens (Primary Tokens) such that their cumulative attention mass reaches a predefined threshold $\tau$.
• Redundant Token Handling: Unselected tokens are not discarded entirely. They undergo bipartite matching and clustering to retain global structural context, then are concatenated with the primary tokens.
• Outcome: This ensures that slices with concentrated attention are heavily compressed, while slices with dispersed critical details retain more tokens, adapting to the specific information density of each slice.

3. Key Contributions

1. First Specialized Framework for 3D Medical VLMs: To the authors' knowledge, this is the first work to propose a model-agnostic token pruning framework specifically tailored for 3D medical VLMs.
2. Training-Free & Hierarchical: The approach requires no retraining of the underlying VLM and employs a two-stage mechanism (slice-level and token-level) to dynamically prune redundant information.
3. Dynamic Adaptation: By using cumulative attention weights rather than fixed ratios, the method adapts to the heterogeneous information density of medical scans and the specific attention biases of different models.
4. Extensive Validation: The framework was validated across three diverse 3D medical benchmarks (M3D, 3D-RAD, AMOS-MM) and three different VLMs (Hulu-Med, MedGemma, Qwen3-VL).

4. Experimental Results

The authors evaluated MedPruner against existing methods (Hulu-L1, VisionZip, HiPrune) and original baselines.

• Performance Retention: MedPruner consistently maintained or exceeded the performance of uncompressed baselines.
  • On the M3D dataset, MedPruner achieved the highest BLEU-4 scores (12.580) with Hulu-Med while retaining only ~52% of tokens.
  • On the 3D-RAD dataset, it achieved the best accuracy (79.280) with Hulu-Med.
• Extreme Compression: A standout result was observed with MedGemma-1.5. MedPruner enabled the model to maintain or surpass original performance while retaining fewer than 5% of visual tokens (specifically 2.46% on AMOS-MM).
• Efficiency: The method significantly reduced inference time. For example, on the AMOS-MM dataset with Hulu-Med, processing time dropped from 9.21s to 7.93s while maintaining 99.19% of the original performance.
• Ablation Studies:
  • Removing IAF caused a performance drop, confirming the necessity of slice-level filtering.
  • Removing the "Primary Token" selection and "Redundant Clustering" led to lower accuracy, proving that retaining structural context via clustering is vital.
  • Sensitivity analysis on the threshold $\tau$ showed diminishing returns beyond a certain point, validating the efficiency of the adaptive selection.

5. Significance

• Clinical Viability: MedPruner bridges the gap between high-performance 3D medical VLMs and the strict latency/compute constraints of real-time clinical deployment.
• Resource Efficiency: By drastically reducing token counts (down to <5% in some cases) without sacrificing diagnostic integrity, it makes running large 3D medical models on standard hardware feasible.
• Generalizability: The "model-agnostic" nature of the framework means it can be applied to various existing and future medical VLMs without the need for expensive retraining, offering a scalable solution for the integration of VLMs into complex medical workflows.