Generative Artificial Intelligence for Opportunistic Osteoporosis Screening Using Lumbar Spine Imaging: Diagnostic Performance, Reproducibility, Effects of Few-Shot Prompting, and Comparison with Physicians

Technical Summary: Generative AI for Opportunistic Osteoporosis Screening

Problem Statement
Osteoporosis remains significantly underdiagnosed and undertreated despite the availability of effective therapies and the routine acquisition of lumbar radiographs and computed tomography (CT) scans for other indications. While Dual-energy X-ray absorptiometry (DXA) is the gold standard for diagnosis, many patients do not undergo bone mineral density (BMD) assessment during routine care. Although physicians form intuitive impressions of bone quality from spinal images, previous studies indicate that the accuracy of such visual assessments is only moderate, even among experienced orthopedic surgeons. While deep learning algorithms have shown promise, they require task-specific development. In contrast, multimodal generative artificial intelligence (AI) offers a widely accessible alternative capable of interpreting medical images without dedicated model training. However, the diagnostic performance of these models for opportunistic osteoporosis screening, their reproducibility, and the impact of "few-shot" prompting (providing example cases to guide responses) remain poorly understood.

Methodology
This image-based comparative study evaluated the diagnostic performance of multimodal generative AI against physicians using a dataset of 45 lateral lumbar radiographs and 45 sagittal reconstructed lumbar CT images.

• Reference Standard: Osteoporosis and low bone mass were defined using WHO DXA T-score thresholds of −2.5 and −1.0, respectively.
• Participants:
  • Physicians: 42 board-certified orthopedic surgeons and 50 non-specialist physicians (data derived from a previous study).
  • AI Models: Three publicly available multimodal generative AI systems: ChatGPT (OpenAI, GPT-5, accessed January 2026), Claude 4 Opus, and Gemini 3.1 Pro.
• Evaluation Protocol:
  • Zero-shot vs. Few-shot: AI models evaluated images under zero-shot conditions (instructions only) and few-shot conditions (provided with 20 labeled reference cases including images, classifications, and YAM values).
  • Reproducibility: Each AI model evaluated the entire dataset five times in independent chat sessions. Final classifications were determined by majority voting (≥3 of 5 responses).
  • Human-AI Combination: An exploratory post-hoc analysis combined specialist majority votes with Claude's majority votes using "OR" (positive if either is positive) and "AND" (positive only if both are positive) rules to assess complementary diagnostic errors.
• Metrics: Primary outcomes included diagnostic accuracy, sensitivity, specificity, and intra-model reproducibility (Cohen's κ).

Key Results

• Diagnostic Accuracy:
  • At the −1.0 T-score cutoff for radiographs, Claude achieved the highest accuracy (73.3%), followed by Gemini (64.4%), specialists (57.4%), ChatGPT (55.6%), and non-specialists (44.0%). Specialists significantly outperformed non-specialists (P<0.001).
  • At the conventional −2.5 cutoff, no significant difference was found between specialists and non-specialists for radiographs, though specialists were significantly more accurate for CT images (P<0.001).
  • AI models showed broadly similar accuracy to specialists in descriptive comparisons for CT images.
• Impact of Prompting:
  • Accuracy: Few-shot prompting did not consistently improve diagnostic accuracy for either physicians or AI models. Changes varied by model and condition, ranging from a 24.4-percentage-point decrease to an 11.1-percentage-point increase.
  • Reproducibility: Few-shot prompting improved intra-model reproducibility in several models, most notably Claude (κ increased from 0.543 to 0.855 for radiographs). This dissociation between improved consistency and unchanged accuracy was a key finding.
• Human-AI Complementarity:
  • The "OR" rule (combining specialist and Claude) increased sensitivity for radiographs at the −1.0 cutoff from 66.7% to 80.0% and for CT from 83.3% to 90.0%, though specificity decreased.
  • Error-rescue analysis confirmed partially non-overlapping errors: Claude correctly classified 40–50% of specialist false-negative cases, and specialists corrected a portion of Claude's errors.

Key Contributions

1. Comparative Performance: The study provides direct evidence that selected generative AI models can achieve diagnostic performance comparable to, or in some conditions superior to, physicians for opportunistic osteoporosis screening using lumbar imaging.
2. Prompting Effects: It demonstrates that few-shot prompting does not guarantee improved diagnostic accuracy but can significantly enhance intra-model reproducibility (consistency), a critical factor for clinical workflow integration.
3. Complementarity: The research offers quantitative evidence of diagnostic complementarity, showing that human and AI errors are partially non-overlapping. This suggests that combining judgments could reduce missed cases (false negatives) in screening scenarios, albeit at the cost of increased false positives.
4. Model Variability: The findings highlight substantial variability in performance and reproducibility across different generative AI models, indicating they are not interchangeable diagnostic tools.

Significance and Claims
The paper concludes that generative AI should be viewed as a potential decision-support tool rather than an autonomous system for opportunistic osteoporosis screening. The authors assert that while AI may not universally outperform physicians, its ability to provide complementary diagnostic judgments could help identify patients who require formal BMD assessment, thereby addressing the issue of underdiagnosis.

The study emphasizes that the clinical value of generative AI lies in its integration into clinical workflows alongside physician judgment. The authors caution that simple combination rules involve a trade-off between sensitivity and specificity and that the apparent complementary effects require prospective validation in independent cohorts. Furthermore, the study notes that improved reproducibility via few-shot prompting does not necessarily equate to improved diagnostic validity, underscoring the need to evaluate accuracy and consistency as distinct dimensions of AI performance.