Prompt-Based Caption Generation for Single-Tooth Dental Images Using Vision-Language Models

This paper addresses the lack of specialized dental datasets by proposing a framework that uses Vision-Language Models with guided prompts to generate high-quality, holistic captions for single-tooth RGB images, thereby enabling more comprehensive dental image analysis.

The Gist

Imagine you have a giant library of dental photos, but the books have no titles, no summaries, and no descriptions. They are just pictures of teeth. If you wanted to teach a computer to understand dentistry, you'd need to tell it what it's looking at: "That's a molar," "That one has a cavity," "This is the chewing surface."

Right now, most dental AI is like a specialist who only knows how to do one thing perfectly—like a mechanic who can only change tires but doesn't know how to fix the engine. They can find a cavity, but they can't write a full report on the tooth's health.

This paper is about building a smart assistant that can look at a single tooth photo and write a detailed, professional description of it, just like a human dentist would.

Here is the story of how they did it, explained simply:

1. The Problem: The "Blurry Group Photo"

The researchers found that existing dental photo collections had two big problems:

• The Group Shot Issue: Most photos showed the whole mouth, but the descriptions only talked about the front teeth or just said "gingivitis." It was like taking a photo of a whole football team but only writing a caption about the goalie's shoes. The back teeth (molars) were often hidden or ignored.
• The "One-Note" Issue: Existing descriptions were too simple. They didn't say which tooth it was, what part of the tooth was visible, or exactly what was wrong with it.

They needed a way to turn these messy, unlabelled photos into a library where every single tooth has its own detailed biography.

2. The Solution: The "AI Intern" with a Checklist

Instead of hiring a human to write thousands of descriptions (which would take forever and cost a fortune), they used a powerful AI called GPT-4o. Think of this AI as a very smart, fast, but slightly inexperienced "AI Intern."

If you just hand the Intern a photo and say, "Describe this," it might guess wrong or miss important details. So, the researchers invented a Two-Step Prompting Strategy (a fancy way of saying "giving the AI a better checklist").

• Step 1: The Rough Draft. They asked the AI to look at the photo and write a basic description.
• Step 2: The Editor. They looked at the mistakes the AI made (like confusing a canine tooth for a front tooth) and gave it a better set of instructions. They said, "Hey, don't just guess! Look specifically for the tooth number, the surface (front, back, or top), and any diseases like cavities or stains."

This is like teaching a child to draw. First, they draw a stick figure. Then you say, "Okay, now add eyes, a nose, and make sure the arms are on the sides." The second instruction makes the drawing much better.

3. The Process: Cleaning the "Raw Ingredients"

Before the AI could write, the researchers had to prepare the "ingredients" (the photos):

• The Filter: They took photos from public websites. Some were blurry, some were too dark, and some showed the whole mouth. They threw out the bad ones.
• The Cutter: They used a computer program to "crop" the photos. If a photo showed a whole mouth, the program cut it out so only one single tooth was left in the frame.
• The Masking: They made sure the file names didn't give away the answers (like naming a file "cavity.jpg"). They wanted the AI to learn by looking, not by reading the filename.

4. The Results: A New "Dental Dictionary"

After running their two-step process, they ended up with 1,520 high-quality photos of single teeth, each with a new, detailed caption.

What worked well?
The AI became surprisingly good at spotting the obvious stuff. It could tell the difference between a molar (the big back tooth) and an incisor (the front tooth). It could spot cavities, stains, and broken edges. It was like a detective who is great at finding big clues.

Where did it struggle?

• The "Invisible" Clues: The AI sometimes missed subtle gum inflammation (gingivitis). It's like trying to spot a faint bruise on a person wearing a thick sweater; the AI couldn't see the gum clearly enough.
• The "Baby Tooth" Confusion: It got confused by children's teeth, which look different from adult teeth.
• The "Angle" Problem: Sometimes, if a tooth was turned sideways, the AI thought it was a different type of tooth.

5. Why This Matters

Why go through all this trouble?
Imagine you want to build a super-dentist AI that can diagnose any problem in a mouth. To train that super-AI, you need thousands of examples where the computer knows exactly what it's looking at.

This paper provides the training manual. By using this framework, they created a massive dataset of "single-tooth stories" without needing humans to write every single one.

The Big Takeaway:
They proved that you don't need a specialized, expensive dental AI to start. You can use a general smart AI (like GPT-4o), give it the right instructions (prompts), and it can learn to describe teeth almost as well as a human. This paves the way for future AI tools that can help dentists diagnose problems faster and more accurately, one tooth at a time.

Technical Summary

Here is a detailed technical summary of the paper "Prompt-Based Caption Generation for Single-Tooth Dental Images Using Vision-Language Models."

1. Problem Statement

Current deep learning models in dentistry are highly specialized for isolated tasks (e.g., segmentation, cavity detection, or gingivitis classification) but lack a holistic understanding of dental anatomy and pathology. While Vision-Language Models (VLMs) exist for general computer vision (e.g., CLIP, BLIP) and some medical domains (e.g., MedCLIP), they are not optimized for dental images.

Key gaps identified in existing literature include:

• Lack of Single-Tooth Datasets: Existing dental caption datasets (e.g., the Gingivitis Captioning Dataset) focus on full-mouth anterior views, often obscuring posterior teeth (molars/premolars).
• Limited Scope of Captions: Existing captions often describe only specific diseases (like gingivitis severity) rather than providing a comprehensive visual description of tooth anatomy, surface, and condition.
• Clinical Relevance: Dental diagnosis and scoring are performed on individual teeth, yet most available data treats the mouth as a whole or lacks structured, multi-attribute descriptions (tooth number, surface, condition) necessary for training specialized VLMs.
• Data Quality Issues: Public datasets suffer from inconsistent views, poor lighting, occlusion, and a lack of structured labels.

2. Methodology

The authors propose a framework to generate structured, clinically aligned captions for single-tooth RGB images using GPT-4o without requiring pre-existing manual annotations. The pipeline consists of four main stages:

A. Data Collection and Preprocessing

• Sources: Images were aggregated from four public repositories (Mendeley and Roboflow), including datasets for gingivitis, children's teeth, mouth detection, and DentalMate.
• Categorization: Images were categorized by view: Anterior (front teeth), Buccal (cheek-side), and Occlusal (chewing surface).
• Single-Tooth Extraction: A tooth detection algorithm was applied to multi-tooth images to crop individual teeth.
  • Padding: For buccal and anterior views, 60 pixels of padding were added to include gum regions for context.
  • Filtering: Low-quality images (blurry, poor lighting, artifacts) were manually and algorithmically filtered.
• Anonymization: To prevent "label leakage" (where the model guesses labels based on filenames like "caries_tooth32.jpg"), all images were renamed to generic identifiers (e.g., img_0001.jpg) before processing.

B. Two-Step Prompt Engineering Strategy

The framework utilizes a two-step prompting process to refine the VLM's output:

1. Level 1 (Basic Generation): The VLM generates initial short and long captions. This step identifies common weaknesses, such as misclassifying anterior teeth (confusing canines with incisors) or omitting disease cues.
2. Level 2 (Refinement & Quality Control): Based on Level 1 analysis, a refined prompt is used. This prompt explicitly instructs the model to:
   • Identify specific attributes: Tooth Type (Incisor, Canine, Premolar, Molar), Surface (Buccal, Occlusal, etc.), and Condition (Caries, Staining, Discoloration).
   • Evaluate image quality and flag low-quality inputs.
   • Generate structured short and long captions.

C. Evaluation

• Dataset Size: From an initial pool of 2,308 cleaned images, 1,520 high-quality images were retained after the VLM's quality filtering.
• Metrics:
  • Automated Evaluation: Measured the accuracy of the VLM's predicted tooth type and surface labels against expert annotations.
  • Manual Evaluation: Experts reviewed a subset of captions to verify the accuracy of disease descriptions (caries, staining, enamel loss, discoloration).

3. Key Contributions

1. Novel Framework: Proposed a method to generate structured, multi-attribute captions for single-tooth dental images using VLMs without manual annotation.
2. Curated Dataset: Created a diverse collection of 1,520 high-quality, single-tooth RGB images covering multiple views (anterior, buccal, occlusal) and conditions.
3. Prompt Engineering Strategy: Demonstrated that a two-step prompting strategy significantly improves the semantic consistency and clinical relevance of generated captions.
4. Comprehensive Evaluation: Provided a dual-mode evaluation (automated and manual) showing that guided prompts can effectively extract clinically relevant details like tooth type, surface, and pathology.

4. Results

• Caption Quality: The two-step prompting strategy significantly improved the model's ability to distinguish between tooth types and surfaces.
  • Surface Accuracy: High accuracy in surface identification (e.g., Dataset 6 achieved 96.38% surface accuracy).
  • Tooth Type Accuracy: Improved from direct assignment to inference from captions. For example, in Dataset 2, inferred type accuracy reached 74.88% compared to 21.16% for direct assignment.
• Disease Detection: The model showed strong performance in detecting visible pathologies:
  • Caries: 70–100% accuracy across datasets.
  • Staining/Discoloration: Generally high accuracy (80–100%).
• Comparative Advantage: Unlike existing datasets that only describe gingivitis location, the generated captions provide visual guidance on tooth structure and multiple conditions simultaneously.
• Limitations Observed:
  • Gingivitis Detection: The model struggled with subtle signs of gingivitis (gum inflammation), likely due to limited gum visibility even with padding.
  • Confusion Points: Misclassification occurred between canines and incisors (due to triangular shape in anterior views) and premolars/molars.
  • Deciduous Teeth: Higher error rates were observed with children's teeth due to irregular morphology.
  • Lingual Surface: The dataset lacked lingual (tongue-side) views, limiting evaluation in that area.

5. Significance and Conclusion

This work bridges the gap between general VLMs and specialized dental diagnostics. By demonstrating that prompt-guided VLMs can generate clinically meaningful, structured captions from unannotated images, the authors provide a scalable pathway to creating large-scale, annotated dental datasets.

• Implication: This framework eliminates the need for expensive, time-consuming manual annotation for initial dataset creation.
• Future Impact: The generated captions serve as the foundational data required to train specialized, fine-tunable Vision-Language Models tailored specifically for dental diagnostics, moving beyond isolated task models to holistic dental understanding.
• Ethical Note: The study utilized open-access, retrospective data; no new ethical approval was required as the data was already publicly licensed.

In summary, the paper validates that with careful prompt engineering and data curation, general-purpose VLMs can be effectively leveraged to build the infrastructure necessary for the next generation of AI-driven dental care.