🤖 machine learning

Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening

This paper proposes a parameter-efficient framework that adapts a frozen DINOv2-Small model using learnable prompt tokens and an adaptive focal loss to enable intrinsically interpretable, high-performance screening of Mild Cognitive Impairment from neuropsychological drawing tests while overcoming data scarcity and class imbalance.

Original authors: Javad Khoramdel, Farhad Hoseyni, Amirhossein Nikoofard

Published 2026-07-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Javad Khoramdel, Farhad Hoseyni, Amirhossein Nikoofard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the clues are hidden in the shaky lines of a drawing. This is the world of Mild Cognitive Impairment (MCI), a sneaky early warning sign that the brain might be starting to slow down, often before full-blown dementia sets in. Doctors usually spot this by looking at how people draw simple things like clocks or cubes, but it's a tricky game. The line between a "healthy" drawing and a "troubled" one is often blurry, like trying to tell the difference between a slightly crooked smile and a frown.

To crack this case, scientists use Artificial Intelligence (AI), specifically a type called Vision Transformers. Think of these AI models as super-smart students who have read millions of books and seen billions of pictures of the real world. They are great at spotting patterns, but teaching them to understand medical drawings is usually like trying to teach a grandmaster chess player to play checkers by rewriting their entire brain. It takes a massive amount of time, money, and computer power. This paper asks a simpler question: Can we teach this super-smart AI to spot the clues in a drawing without rewriting its brain, just by giving it a few special hints?

The researchers behind this study, Javad Khoramdel and his team, say "yes." They built a clever, lightweight system that acts like a detective with three different pairs of glasses. Instead of forcing the AI to relearn everything from scratch, they kept the AI's "brain" frozen (unchanged) and simply added three tiny, custom-made "prompt tokens." You can think of these tokens as little sticky notes the AI sticks to the drawing before looking at it. One note says, "Look for clock mistakes," another says, "Check the cube corners," and the third says, "Scan the trail paths."

The magic happens because the AI uses these notes to focus its attention. When the AI looks at a clock drawing, the "clock note" tells it exactly which parts of the image matter most, creating a heat map that shows doctors exactly where the AI is worried. This is a big deal because usually, AI is a "black box" that just gives an answer without explaining why. Here, the explanation is built right into the system.

But there's a catch: the data is messy. The dataset they used has way more healthy people than sick ones (about 651 healthy vs. 267 with MCI), and many people score right on the edge of the cutoff line, making them hard to classify. To handle this, the team invented a special "MoCA-adapted focal loss." Imagine this as a teacher who doesn't just grade a test as "pass" or "fail." Instead, if a student's score is right on the borderline, the teacher gives them extra attention and tries harder to understand their specific struggles. The AI does the same thing, learning to pay extra attention to the confusing cases near the edge of the diagnosis line.

The results are promising. When they tested their system, it correctly identified MCI patients with a score (F1) of 0.641, which is significantly better than a much heavier, more complex system called ResViT (which scored 0.531). The new system also achieved an AUC of 0.795, a measure of how well it separates the healthy from the sick. Perhaps most importantly, it did all this while only training 1.19 million parameters, a tiny fraction of the 32 million parameters required by the older, heavier models.

The team also tested what happens if you remove their special tricks. They found that if they took away the "MoCA-probability modulator" (the part that focuses on the borderline cases), the system's performance dropped noticeably. This suggests that handling the "gray area" of diagnosis is crucial. They also found that using three separate "notes" (one for each drawing type) worked better than using just one generic note for all drawings, proving that the AI needs to treat each drawing task as a unique puzzle.

In short, this paper suggests that we don't need to build massive, expensive AI brains to detect early cognitive decline. Instead, we can take a smart, pre-trained AI, give it a few simple, task-specific hints, and teach it to look at the right spots with a special focus on the tricky, borderline cases. It's a lighter, faster, and more transparent way to help doctors catch the early signs of memory loss, turning a complex medical mystery into a solvable puzzle.

Technical Summary: Parameter-Efficient Prompt Tuning of Vision Foundation Model with Adaptive Focal Loss for Interpretable MCI Screening

1. Problem Statement

Mild Cognitive Impairment (MCI) represents a critical early stage of cognitive decline preceding Alzheimer's disease. Automated detection from neuropsychological drawing tests (specifically the Clock Drawing Test, Cube Copying Test, and Trail Making Test) faces three fundamental constraints:

Data Scarcity and Imbalance: Datasets are often small with skewed class distributions (e.g., a 2.4:1 ratio of healthy controls to MCI patients).
Diagnostic Ambiguity: Clinical boundaries are uncertain, with MoCA scores clustering densely around the cutoff of 25, making hard binary classification suboptimal.
Limitations of Existing Methods: Current state-of-the-art approaches rely on computationally expensive, fully fine-tuned hybrid architectures (e.g., combining ResNet and ViT) that lack parameter efficiency. Furthermore, interpretability in these models is typically a post-hoc approximation (e.g., attention rollout or Grad-CAM) rather than an intrinsic architectural property.

2. Methodology

The authors propose a parameter-efficient framework that adapts a frozen DINOv2-Small vision foundation model using Prompt Tuning and a novel MoCA-adapted Focal Loss.

2.1 Architecture: Parameter-Efficient Prompt Tuning

Instead of fine-tuning the entire backbone, the model keeps the DINOv2-Small weights frozen. Adaptation is achieved through:

Modality-Specific Learnable Tokens: Three distinct learnable prompt tokens are introduced, one for each drawing task (clock, cube, trail). Each token is initialized randomly and optimized end-to-end.
Cross-Attention Aggregation: Each task-specific token acts as a query in a shared cross-attention layer, attending to the patch tokens of the corresponding image processed by the frozen backbone. This produces task-conditioned embeddings ( $e_k$ ) that summarize the most informative spatial patches for that specific cognitive function.
Learnable Query Attention Fusion: The three task embeddings are aggregated via a learnable query vector that attends over the embeddings. This produces a fused patient vector and a set of modality importance weights ( $\alpha$ ), indicating which drawing test most influenced the final decision.
Intrinsic Interpretability: The cross-attention weights from the prompt tokens to the image patches serve as direct spatial saliency maps, requiring no additional backward computation or post-hoc approximation.

2.2 Loss Function: MoCA-Adapted Focal Loss

To address diagnostic ambiguity and class imbalance, the authors introduce a unified loss function that generalizes standard soft-label approaches. It integrates:

Soft Anchor: Converts the continuous MoCA score ( $m_i$ ) into a soft target ( $s_i$ ) using a sigmoid function centered at the clinical cutoff ( $C=25.5$ ).
MoCA-Probability Modulator: A novel modulator that reweights the loss based on the discrepancy between the model's predicted probability ( $p_i$ ) and the MoCA-derived expectation ( $s_i$ ). Samples where the prediction diverges from the score-based expectation are up-weighted.
Adaptive Per-Sample Weighting: Corrects for unequal frequencies of subjects across MoCA score bins (MoCA-bin adaptive weighting), ensuring rare, borderline cases are not ignored.

2.3 Training Strategies

The framework employs several targeted augmentation and sampling strategies:

Image Inversion: Inverting pixel intensities to align the white-background drawings with the natural-image statistics of the pre-trained DINOv2.
Class-Balanced Batch Sampling: Ensuring equal representation of classes in each batch.
Type-Preserving Mixup: Interpolating images and labels within the same drawing type to maintain clinical coherence.
MoCA-Neighbour Drawing Swap: Replacing one or two drawings in a triplet with those from a subject with a similar MoCA score to increase structural diversity.

3. Key Contributions

Parameter-Efficient, Interpretable Classification: The framework utilizes a frozen foundation model with only 1.19 million trainable parameters (less than 6% of the total). It yields spatial attention maps and modality importance weights as direct byproducts of the inference process.
MoCA-Aware Focal Loss: A unified loss formulation that strictly generalizes prior soft-label methods by incorporating continuous cognitive scores into the training target, loss modulation, and adaptive weighting.
Targeted Augmentation: Strategies specifically designed to address the photometric domain gap (image inversion) and the limited diversity of neuropsychological datasets (drawing swap, type-preserving Mixup).

4. Experimental Results

The model was evaluated on a dataset of 918 subjects (651 healthy, 267 MCI) using stratified five-fold cross-validation.

Performance: The proposed architecture achieved an MCI-class F1 score of 0.641 ± 0.026 and an AUC of 0.795 ± 0.024.
Comparison: It outperformed the computationally heavier ResViT baseline (a hybrid ResNet50/ViT model) by 0.110 in MCI-class F1, despite using significantly fewer trainable parameters.
Ablation Studies:
- Removing the MoCA-probability modulator caused the largest performance drop (F1MCI decreased by 0.017), confirming its critical role in handling boundary ambiguity.
- Task-specific prompt tokens were found superior to shared tokens or CLS-only approaches for MCI sensitivity, as they allow the model to specialize in task-specific deficits rather than averaging them out.
- Prompt tuning outperformed LoRA and Top Tuning strategies, achieving the best MCI recall with the fewest additional parameters.

5. Significance and Claims

The paper claims that architectural complexity (stacking multiple large networks) is unnecessary for this task. Instead, a self-supervised vision foundation model, when adapted via lightweight, task-specific prompt tuning, provides a superior foundation.

Key claims regarding significance include:

Efficiency: The method achieves state-of-the-art performance with a fraction of the trainable parameters required by full fine-tuning or hybrid architectures.
Intrinsic Interpretability: Unlike prior works that rely on post-hoc approximations, this framework provides spatial saliency maps and modality importance scores as inherent structural consequences of the architecture.
Handling Ambiguity: The MoCA-adapted focal loss effectively addresses the inherent uncertainty near clinical boundaries by leveraging continuous cognitive scores rather than rigid binary labels.
Clinical Relevance: The ability to identify which specific drawing modality (clock, cube, or trail) drove a prediction offers a level of interpretability potentially useful for clinical decision support, although the authors note this requires future validation against expert judgment.

The authors maintain a modest stance, acknowledging limitations such as the use of a single dataset from one institution, the exclusion of temporal kinematic data (pen velocity/pressure), and the need for external validation of the modality weights against clinical experts.