MetaDent: Labeling Clinical Images for Vision-Language Models in Dentistry

1. Problem Statement

Despite the rapid advancement of Vision-Language Models (VLMs) in general medical imaging, their application to intraoral photography remains underexplored and ineffective. The authors identify two primary bottlenecks:

• Lack of Fine-Grained Data: Existing dental datasets are often small, single-center, or limited to specific categories (e.g., only caries or only cancer). They lack the diversity and hierarchical complexity required for nuanced clinical diagnosis.
• Annotation Limitations: Current datasets rely on closed-set categorical labels or vague text, which fail to capture the hierarchical, contextual, and multi-faceted nature of dental abnormalities. Consequently, state-of-the-art VLMs struggle with fine-grained understanding, producing inconsistent descriptions and missing critical diagnostic findings.

2. Methodology

The authors propose MetaDent, a comprehensive framework comprising a large-scale dataset, a novel annotation strategy, and standardized benchmarks.

A. Dataset Curation (Meta Dataset)

The dataset consists of 60,669 dental images aggregated from three distinct sources to ensure diversity:

1. DS1 (Clinical): 4,373 images from Wuhan University School of Stomatology.
2. DS2 (Public): 9,390 images from the "Teeth or Dental" public dataset.
3. DS3 (Web-Crawled): 46,906 images filtered from the massive COYO-700M corpus using a fine-tuned ViT-L/16 binary classifier and deduplication techniques.

From this pool, a representative subset of 2,588 images was selected for high-quality human annotation.

B. Semistructured Annotation Framework

Instead of simple tags, the authors introduced a semistructured meta-labeling scheme performed by two experienced dentists:

• Overall Summary: A concise paragraph describing the main visual content and shooting perspective.
• Point-by-Point Abnormalities: A list of specific deviations from normal anatomy, described in free-text natural language (e.g., "Tooth #36 has extensive post-carious crown destruction").
• Spatial Context: Rough contours drawn for each abnormality.
• Quality Control: Annotators flagged uncertain cases. Inter-rater reliability (Cohen's $\kappa$) was 0.83, indicating high agreement.

C. Secondary Dataset Generation via LLMs

To create scalable benchmarks, the semistructured meta-labels were processed by Large Language Models (LLMs, specifically GPT-OSS-120B) to generate three downstream task datasets:

1. Visual Question Answering (VQA): Generated ~15,000 pairs (True/False and Multiple Choice) based on the meta-labels, including reasoning about non-mentioned normal structures.
2. Multilabel Classification: Mapped the free-text abnormalities into 18 standardized clinical classes (e.g., "Gingival inflammation," "Malocclusion").
3. Image Captioning: Generated free-form descriptive captions.

Note: A rigorous human review and error analysis were conducted to validate the LLM-generated benchmarks, confirming high fidelity (94.2% correct VQA pairs).

3. Key Contributions

• MetaDent Dataset: The largest and most diverse public dental image dataset to date (60k+ images), bridging the gap between clinical and web-sourced data.
• Novel Annotation Strategy: A "meta-labeling" approach that captures hierarchical clinical nuance, allowing flexible conversion into various task formats (VQA, classification, captioning) without losing semantic richness.
• Comprehensive Benchmark: A standardized evaluation suite covering VQA, multilabel classification, and image captioning, enabling direct comparison of state-of-the-art models.
• Open Release: The dataset, annotation tools, and benchmarks are publicly available to foster reproducible research.

4. Results

The authors evaluated five leading VLMs (GPT-4o, Gemini-2.5-flash, Qwen3-VL, Ovis2, Baichuan-Omni) on the MetaDent benchmarks. The results highlight a significant performance gap:

• Visual Question Answering (VQA):
  • Models achieved moderate accuracy (max ~68%).
  • True/False questions were slightly easier than Multiple Choice.
  • Gemini-2.5-flash and GPT-4o led the pack, but no model exceeded 70% accuracy, indicating a struggle with fine-grained reasoning.
• Multilabel Classification:
  • Performance was poor, with the best model (GPT-4o) achieving an F1-score of only 0.437.
  • Exact Match accuracy (predicting all labels correctly) was extremely low (<25%), showing models rarely capture the full set of findings.
  • Significant domain shift was observed: Models performed notably worse on the public dataset (DS2) due to differences in image tone and patient demographics (e.g., teenagers vs. adults).
• Image Captioning:
  • Generated captions had low semantic similarity (BERTScore F1 $\approx$ 0.2) and low clinical consistency.
  • Models frequently missed key abnormalities or hallucinated findings, rendering them unreliable for direct clinical interpretation.

5. Significance and Conclusion

• Current Limitations: The study demonstrates that even the most advanced general-purpose VLMs are not yet ready for clinical deployment in dentistry. They lack the fine-grained understanding necessary for accurate diagnosis and documentation.
• Domain Adaptation: The results underscore the critical need for domain-specific training and the development of specialized dental VLMs rather than relying on general medical or general-purpose models.
• Future Roadmap: MetaDent provides the necessary infrastructure (data and benchmarks) to train, fine-tune, and evaluate next-generation dental AI. The authors envision these tools enabling automated clinical documentation, early disease screening in resource-constrained settings, and improved public health monitoring.

In summary, MetaDent exposes the current "AI gap" in dental imaging and provides the essential resources to close it, moving the field from simple classification toward complex, reasoning-based clinical understanding.