Generating Findings for Jaw Cysts in Dental Panoramic Radiographs Using a GPT-Based VLM: A Preliminary Study on Building a Two-Stage Self-Correction Loop with Structured Output (SLSO) Framework

1. Problem Statement

While Vision-Language Models (VLMs) like GPT-4o show promise in medical image interpretation, their direct application in dentistry faces significant hurdles:

• Visual Recognition Limitations: VLMs struggle with spatial consistency, particularly in identifying specific tooth numbers and detecting subtle pathological changes (e.g., mild root resorption) in panoramic radiographs.
• Hallucinations and Factual Inconsistencies: Models often generate "vague" descriptions, omit negative findings, or fabricate anatomical relationships (hallucinations), which is critical in clinical settings.
• Lack of Structured Output: Free-form text generation leads to variability, making it difficult to verify accuracy or integrate findings into structured electronic health records.
• Baseline Limitations: Standard prompting techniques like Chain-of-Thought (CoT) improve reasoning but do not sufficiently enforce consistency or suppress hallucinations in complex dental imaging tasks.

2. Methodology

The authors propose the Self-correction Loop with Structured Output (SLSO) framework, a 10-step integrated processing pipeline designed to act as an external validation mechanism for GPT-4o.

A. Dataset

• Source: 22 dental panoramic radiographs containing jaw cysts (collected from Asahi University).
• Ground Truth: Manually annotated by an experienced oral and maxillofacial radiologist, including lesion margins, tooth numbers (FDI notation), and specific radiographic features.
• Input Preprocessing: Regions of Interest (ROIs) were manually extracted and annotated with tooth margins and numbers to guide the model.

B. The SLSO Framework (10-Step Process)

The framework operates as a loop involving structured data generation, extraction, and consistency checking:

1. Input: Annotated ROI images and instructions are fed to GPT-4o.
2. Image Analysis: Multimodal analysis of the ROI.
3. Structured Data Generation: The model outputs findings in a strict JSON format based on a predefined schema (Table 1).
   • Schema Categories: X-ray transparency, Internal structure, Border, Root resorption, Tooth displacement, Relationship to anatomical structures, and Affected tooth numbers.
4. Tooth Number Extraction: The model independently extracts tooth numbers directly from the image.
5. Consistency Check (Loop 1): Compares the tooth numbers in the Structured Data (Step 3) against the extracted numbers (Step 4).
6. Regeneration: If a mismatch is detected, the structured data and tooth numbers are regenerated.
7. Finding Generation: Natural language findings (Japanese) are generated from the validated structured data.
8. Reverse Structuring: The generated text findings are converted back into structured data.
9. Consistency Check (Loop 2): Compares the restructured data against the original validated structured data.
10. Final Regeneration: If inconsistencies remain, findings are regenerated. This loop can repeat up to 5 times.

C. Experimental Setup

• Model: GPT-4o (gpt-4o-2024-11-20).
• Comparison: The SLSO framework was compared against a standard Chain-of-Thought (CoT) prompting method.
• Evaluation Metrics: Binary scoring (0/1) for exact semantic match against ground truth across seven categories.

3. Key Contributions

1. Novel Framework (SLSO): Introduction of a model-agnostic, two-stage self-correction loop that enforces structured intermediate representations (JSON) to validate natural language outputs.
2. Hallucination Suppression: Demonstrated that enforcing explicit "present/absent" judgments via a schema significantly reduces vague language and suppresses the generation of non-existent anatomical findings.
3. Tooth Number Accuracy: Showed that iterative consistency checks (extracting numbers from images vs. text) drastically improve the identification of affected teeth, a known weak point for VLMs.
4. Negative Finding Documentation: The framework successfully enforced the documentation of negative findings (e.g., "no root resorption"), which are frequently omitted in standard CoT outputs.

4. Results

• Overall Performance: SLSO outperformed CoT in most categories, though statistical significance was not reached due to the small sample size (n=22).
• Specific Improvements:
  • Tooth Number Identification: Improved by 66.9% (from 0.136 to 0.227).
  • Tooth Displacement: Improved by 33.3%.
  • Root Resorption: Improved by 28.6%.
  • Relationship with Structures: Improved by 9.0%.
  • Note: High-accuracy items (Radiolucency, Internal Structure, Boundary) showed no difference as both methods performed well (>86%).
• Case Studies:
  • Success Case: A case involving teeth #47 and #48 achieved a perfect score (1.00) after 5 regeneration cycles, correcting an initial hallucination regarding tooth numbers.
  • Failure Case: A complex maxillary anterior lesion spanning multiple teeth resulted in a lower score for SLSO (0.43) than CoT (0.57), highlighting limitations in handling extensive, multi-tooth lesions and subtle anatomical changes (e.g., nasal floor elevation).

5. Significance and Future Directions

• Clinical Utility: The study establishes that AI in dentistry should function as a supportive tool rather than a replacement. The SLSO framework provides transparent, verifiable intermediate outputs that facilitate human-in-the-loop review.
• Methodological Shift: Instead of fine-tuning the model, the study proves that framework-level interventions (structured constraints + iterative verification) can enhance reliability in medical AI.
• Limitations:
  • Dataset: Small, single-institution dataset limits generalizability.
  • Complexity: The framework struggles with lesions spanning multiple teeth and subtle anatomical variations.
  • Privacy: Current reliance on cloud-based APIs (GPT-4o) poses data privacy challenges for clinical deployment; future work requires on-premise, locally deployable models.
• Future Work: Integration with automated lesion detection models, expansion to larger diverse datasets, and development of locally hosted VLMs to ensure data security.

Conclusion: The SLSO framework represents a significant step toward reliable AI-assisted radiological reporting by combining structured data constraints with iterative self-correction, effectively reducing hallucinations and improving the consistency of critical dental findings.