Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

1. Problem Statement

Adverse drug events (ADEs) are a major cause of preventable harm, driving the need for automated pill recognition systems. However, deploying these systems in real-world healthcare settings faces significant challenges:

• Visual Complexity: Real-world scenarios involve cluttered scenes, overlapping pills, partial occlusions, specular reflections, and diverse backgrounds (e.g., pill organizers), which differ vastly from controlled laboratory conditions.
• Data Scarcity: Annotating large-scale medical datasets is costly and logistically difficult, necessitating Few-Shot Learning (FSL) to adapt models with minimal labeled examples.
• Evaluation Gap: Existing FSL research often relies on "clean" benchmarks where training and testing data share similar distributions (isolated pills, minimal background). These studies fail to assess how models generalize under the severe visual domain shifts and annotation heterogeneity found in actual deployment.

The core problem addressed is: How do few-shot object detection models adapt to novel pill classes when the training data (base domain) differs significantly in visual realism and annotation style from the deployment environment (target domain)?

2. Methodology

The study employs a deployment-oriented evaluation framework rather than proposing a new architecture. It utilizes a standard two-stage object detection pipeline based on Faster R-CNN (via the FsDet library).

A. Datasets and Experimental Design

The study compares two distinct base-training datasets to analyze the impact of "visual realism":

1. CURE: A large-scale dataset (8,973 images, 196 classes) featuring single pills under controlled conditions with full-image bounding boxes. It lacks multi-object interaction and occlusion.
2. MEDISEG: A dataset (8,262 images, 32 classes) featuring multi-pill scenes in realistic settings (dosette boxes, clutter) with instance-level bounding boxes.

Protocol:

• Base Training: Models are pre-trained exclusively on either CURE or MEDISEG.
• Few-Shot Adaptation: Models are fine-tuned on a novel deployment dataset (disjoint from base classes) using a 5-way K-shot setting ($K \in \{1, 5, 10\}$).
• Evaluation: Performance is tested on a held-out query set from the novel deployment dataset, featuring heavy overlap and clutter. A specific overlap-only stress test subset is also used.

B. Evaluation Metrics

Due to heterogeneous annotation strategies (full-image vs. instance-level boxes), standard Average Precision (AP) is deemed incomparable. Instead, the study focuses on classification-centric and error-based metrics:

• Foreground Classification Accuracy: Semantic recognition capability once an object is localized.
• False Negative (FN) Rate: Measures missed detections (recall failure).
• Loss Metrics: RPN classification loss, total detection loss, and classification loss to assess optimization stability.

3. Key Contributions

1. Semantic Saturation in Low-Shot Regimes: Demonstrated that semantic pill recognition adapts rapidly; foreground classification accuracy saturates even with 1-shot supervision, regardless of the base training source.
2. Primacy of Training Data Realism: Identified that the visual realism of base training data is a more dominant factor for generalization than dataset size or shot count. Models trained on realistic, multi-object data (MEDISEG) significantly outperform those trained on isolated data (CURE) in cluttered scenarios.
3. Decoupling of Recognition and Localization: Revealed a systematic failure mode where semantic classification remains robust, but localization and recall degrade sharply under heavy overlap. This failure is often masked by aggregate metrics like AP in clean benchmarks.
4. FSL as a Diagnostic Tool: Reframed few-shot fine-tuning not just as an adaptation strategy, but as a method to probe deployment readiness, exposing stability-robustness trade-offs and diminishing returns beyond intermediate supervision levels (5-shot).

4. Key Results

• Generalization under Domain Shift: Both CURE and MEDISEG-trained models achieved high foreground classification accuracy (>98%) in the 1-shot regime on the novel dataset, proving that semantic representations transfer well despite domain shifts.
• The "Realism" Advantage:
  • In the overlap-only stress test (1-shot), MEDISEG-trained models achieved 0.406 foreground accuracy, while CURE-trained models collapsed to 0.131 (a 210% relative improvement).
  • This gap persisted across all shot counts, with MEDISEG models showing significantly lower False Negative rates and better region proposal reliability in cluttered scenes.
• Supervision Levels & Diminishing Returns:
  • Increasing shots from 1 to 5 yielded significant gains in stability (reduced variance) and recall, particularly for MEDISEG models.
  • Moving from 5-shot to 10-shot provided diminishing returns, suggesting that intermediate supervision levels offer the best balance between annotation effort and robustness.
• Failure Modes: Qualitative analysis showed that while models could correctly identify pill types (semantic), they frequently failed to localize them (bounding boxes) when pills were heavily occluded or overlapping, a failure mode specific to low-shot adaptation in complex scenes.

5. Significance and Implications

• Redefining Evaluation: The paper argues that standard benchmarks with isolated objects are insufficient for safety-critical medical AI. Evaluation must prioritize cross-dataset domain shift and cluttered scene robustness.
• Data Strategy: For few-shot medical applications, investing in realistic, multi-object training data (even with fewer classes) is more critical for deployment success than simply scaling up clean, single-object datasets.
• Clinical Deployment: The findings suggest that while semantic recognition is achievable with minimal data, localization remains a bottleneck in real-world settings. Systems must be designed with the understanding that high classification accuracy does not guarantee reliable detection in cluttered environments.
• Diagnostic Utility: Few-shot learning protocols can serve as a "stress test" to determine if a model is ready for deployment by revealing how it handles occlusion and annotation heterogeneity before full-scale rollout.

In conclusion, the study establishes that while modern detectors can learn pill semantics quickly with few examples, their deployment readiness is heavily dependent on the visual realism of the pre-training data and the specific challenges of localization in cluttered scenes, rather than just the quantity of labeled examples.