📄 dentistry and oral medicine

DentaCoPilot: An LLM-Augmented Next-Procedure Recommender for General Dentistry, Designed for Dentist Augmentation

DentaCoPilot is an LLM-augmented next-procedure recommender for general dentistry that achieves state-of-the-art accuracy by prompting a small language model with top candidates from classical baselines, thereby delivering calibrated predictions, explicit abstention flags, and chart-grounded rationales to support rather than replace clinical decision-making.

Original authors: Rodrigues, C. C., Rebello, S. D.

Published 2026-06-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rodrigues, C. C., Rebello, S. D.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine a dental office as a busy kitchen. Right now, most "smart" dental tools are like super-accurate sous-chefs who only look at the ingredients. They can look at an X-ray and say, "That's a cavity," or "That tooth has a crack." They are great at diagnosis.

But there is a harder question that follows every diagnosis: "Now that we know what's wrong, what should we cook next?" Should we fill the tooth, pull it, or just clean it? This is the "next procedure" question, and until now, no computer system has been able to answer it well for general dentists.

This paper introduces DentaCoPilot, a new system designed not to replace the dentist, but to act like a highly organized, well-read assistant sitting right next to them.

Here is how it works, broken down simply:

1. The Problem: The "Next Step" Mystery

Current AI is like a detective who is great at finding clues (diagnosis) but bad at writing the story of what happens next (treatment).

The Old Way: A computer looks at a chart and tries to guess the next step purely by pattern-matching, like a student guessing the next word in a sentence based only on the previous word.
The New Goal: The system needs to look at the whole patient history (like a full novel) and suggest the next chapter, while admitting when it doesn't know enough to guess.

2. The Experiment: A "Fake" Patient Library

Because real patient records are hard to get (due to privacy laws), the researchers first built a giant library of "fake" patient stories.

They created 500 synthetic patients with realistic dental histories.
They asked the computer: "Given this patient's history, what is the next procedure?"
They tested two types of "brains":
- The "Classic" Brains: Traditional math models (like a very fast, strict calculator).
- The "LLM" Brains: Large Language Models (like the AI you might chat with, which is good at understanding language and reasoning).

3. The Surprise: The "Smart" AI Wasn't the Best

When they tested the pure "Smart AI" (the LLMs) against the "Classic" calculators on these fake patients, the calculators won.

Why? The fake stories were built on simple patterns (like "if you had a cleaning, you usually get a check-up next"). The calculators were perfect at spotting these simple patterns. The Smart AI got confused because it was trying to "reason" about a vocabulary of hundreds of codes without enough specific training examples.
The Lesson: Just because an AI is "smarter" and can write poetry doesn't mean it's better at predicting the next step in a specific medical routine.

4. The Solution: The "Hybrid" Assistant (The Big Win)

The researchers found a clever trick to make the Smart AI work. They didn't ask the AI to guess from scratch. Instead, they asked the Classic Calculator to give a "shortlist" of the top 10 most likely next steps first.

Then, they handed that shortlist to the Smart AI and said: "Here are the top 10 guesses. Now, use your brain to look at the patient's full story, pick the best one, explain why, and tell us if you're confident."

The Result: This "Hybrid" team (Calculator + Smart AI) performed almost as well as the best calculators, but with a huge bonus: It could explain its reasoning.
The Analogy: Think of the Calculator as a librarian who quickly pulls the 10 most relevant books off the shelf. The Smart AI is the expert critic who reads the blurbs on those 10 books and tells you exactly which one to read and why.

5. What Makes This System Different?

Most AI tries to be an autonomous robot that makes the final decision. DentaCoPilot is designed as a decision support tool. It has three special "safety features" that the paper highlights:

The "I Don't Know" Flag: If the patient's chart is too messy or missing info, the system can raise a red flag and say, "I can't guess this safely," instead of making a dangerous mistake.
The "Confidence" Label: It tells the dentist, "I'm 90% sure" or "I'm only 40% sure," so the dentist knows how much to trust the suggestion.
The "Why" Explanation: It doesn't just give a code; it writes a sentence like, "I suggest a cleaning because the patient hasn't had one in 14 months and has a history of gum disease."

6. The Real-World Test (The "MEPS" Data)

The researchers also tested their ideas on a massive, real-world public dataset of dental visits (called MEPS).

The Finding: Even on real data, the "Smart AI" alone performed poorly if it only had a tiny bit of history (just the last visit type). It needed the rich, detailed chart context to work.
The Takeaway: The AI's value comes from reading the whole story, not just the last sentence.

7. What's Next?

The paper is very careful to say: "We are not deploying this in a real clinic yet."

The results so far are based on "fake" data and public summaries.
The next step is a planned study at a dental school in India, where real dentists will use the tool to see if it actually helps them make better decisions.
They also plan to test it on a massive, real database of patient records (BigMouth) once they get permission.

Summary

DentaCoPilot is a new kind of dental assistant. It doesn't try to replace the dentist. Instead, it uses a "smart" AI to read the patient's full history, but it relies on a "dumb" calculator to give it a starting list of options. This combination allows the AI to make accurate suggestions, explain why it made them, and admit when it's unsure—all while keeping the final decision in the hands of the human dentist.

Technical Summary: DentaCoPilot

Problem Statement
While commercial dental AI in 2026 has matured significantly in diagnostic tasks (e.g., caries detection, bone-level measurement on radiographs), a critical gap remains in next-procedure recommendation for general dentistry. Existing systems, such as MultiTP, are often restricted to specific case types (e.g., partial edentulism), lack calibrated uncertainty, do not abstain when context is insufficient, and are evaluated as autonomous classifiers rather than decision-support tools. The core challenge is to sequence a treatment plan from a structured patient chart that accounts for demographics, medical history, and the specific trajectory of prior procedures, while providing the dentist with a rationale and confidence level rather than a deterministic output.

Methodology
The authors introduce DentaCoPilot, a system designed to augment, not replace, the dentist. The system takes a structured patient chart (demographics, medical flags, and a time-ordered sequence of CDT-coded procedures) and the most recent procedure as input. It outputs four signals:

A top- $K$ probability distribution over the next CDT procedure code.
A verbalized confidence label (low, medium, high).
An explicit abstain flag when the chart context is deemed insufficient.
A chart-grounded rationale citing specific prior findings.

Experimental Setup:

Datasets:
- Synthetic Corpus: 500 patients (1,284 test examples) generated via a hierarchical Markov-style process based on BigMouth data. This serves as a test of pipeline behavior and augmentation properties.
- Real-World Benchmark: The MEPS 2023 Dental Visits corpus (11,016 transitions across 5,088 patients), evaluated at treatment-category granularity (not specific CDT codes) to establish external validity.
Models Evaluated:
- Classical Baselines (B0–B3): Frequency bigram, TF-IDF + Logistic Regression, XGBoost, and a re-implementation of MultiTP-style CNN-RNN.
- LLM Variants (M1–M6): Variants using Claude Haiku, Sonnet, and Opus, employing strategies such as chain-of-thought (CoT), retrieval-augmented generation (RAG), and prompt-conditioning (injecting classical baseline candidates into the LLM prompt).
Infrastructure: All LLM inference is routed through a local Anthropic Claude Code CLI to ensure full auditability and reproducibility, with every call logged.

Key Results

Pure LLM Performance: On the synthetic corpus, pure LLM variants (M1–M4) significantly underperformed classical baselines on top-1 exact-CDT-match accuracy (ranging from 0.267 to 0.467 vs. 0.567 for baselines). This gap was attributed to the strong local Markov structure of the synthetic data, which bigrams capture perfectly, and the LLMs' tendency to select category-correct but exact-code-mismatching predictions.
The "Prompt-Conditioning" Breakthrough (M5): The most significant finding was that prompt-conditioning a Sonnet LLM with the top-10 candidates from a classical TF-IDF + Logistic Regression baseline (Model M5) closed the accuracy gap.
- M5 achieved 0.533 top-1 and 0.933 top-5 accuracy, approaching the classical baselines (0.567 top-1, 0.933–0.967 top-5).
- Crucially, M5 retained the augmentation features: chart-grounded rationale, verbalized confidence, and the ability to abstain.
Model Capability vs. Design: Upgrading the LLM backbone from Sonnet to the more capable Opus (M4, M6) yielded no improvement in accuracy, with or without priming. This suggests that for this task, prompt design and the integration of classical priors are more effective than raw model capability.
Real-World Validation (MEPS): On the MEPS corpus, history-aware baselines (Markov models) significantly outperformed a most-frequent baseline (0.479 vs. 0.297 top-1). A zero-shot LLM given only coarse category history (0.213 top-1) underperformed the Markov baseline, reinforcing the claim that the LLM's value is contingent on rich chart context, not just procedural history.
Calibration and Abstention: While the system is designed to abstain, the synthetic evaluation showed the models rarely triggered the abstain flag due to prompt design limitations. However, coverage-risk analysis confirmed that classical baselines could be calibrated to trade coverage for precision, a feature the hybrid system aims to preserve.

Significance and Claims
The paper positions DentaCoPilot as a decision-support consult rather than an autonomous classifier. Its primary contributions are:

Augmentation Framework: It demonstrates that an LLM can be effectively integrated into a clinical workflow by using a classical model to handle statistical priors (the "what is likely") while the LLM handles the reasoning layer (the "why" and "rationale").
Cost-Effective Design: The study identifies that prompt-conditioning a smaller LLM (Sonnet) with classical priors is more cost-effective and performant than using larger, more expensive models (Opus) or relying on pure LLM reasoning.
Auditability: By routing all inference through a local CLI with full logging, the system ensures that every recommendation can be audited and replayed without re-spending compute, addressing a key barrier to clinical AI adoption.
Modest Scope: The authors explicitly state that the synthetic results validate the pipeline behavior and augmentation properties (rationale, calibration, abstention) but do not claim external validity for clinical performance. The authors emphasize that the next stage of work involves a pre-registered clinician-in-the-loop evaluation at the KLE Vishwanath Katti Institute and multi-institutional validation using the BigMouth repository to confirm whether these design choices translate to real-world clinical utility.

The paper concludes that while pure LLMs currently lag behind classical methods in raw prediction accuracy on structured dental data, a hybrid architecture that leverages classical priors to guide the LLM offers a viable, cost-effective path toward clinically useful, explainable, and safe next-procedure recommendation systems.