💬 NLP

L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification

The paper introduces L2D-Clinical, a framework that adaptively selects between specialized BERT models and general-purpose LLMs for clinical text classification by learning to defer only when the LLM offers superior performance, thereby improving accuracy while minimizing API costs.

Original authors: Rishik Kondadadi, John E. Ortega

Published 2026-04-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rishik Kondadadi, John E. Ortega

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a busy hospital. You have two different doctors on your staff, and you need them to read thousands of patient notes every day to spot problems or decide if a treatment worked.

Doctor BERT is a specialist. They have read every single medical textbook ever written. They are incredibly fast, cheap to employ, and amazing at spotting obvious patterns (like "drug X caused rash Y"). However, they can be a bit rigid. If a patient writes in a vague, hesitant way ("I might feel a little dizzy"), Doctor BERT gets confused and might miss it.

Doctor LLM is a general genius. They have read the entire internet, not just medical books. They are great at understanding nuance, sarcasm, and vague language. They can figure out that "dizziness" mentioned in a specific context is a problem. But, they are slow, expensive to hire, and sometimes they get too creative, seeing problems where there aren't any.

The Problem: Choosing the Wrong Doctor

In the past, hospitals had to pick one doctor for the whole job.

If you picked Doctor BERT, you saved money and were fast, but you missed some tricky cases.
If you picked Doctor LLM, you caught almost everything, but it cost a fortune, and they sometimes made up problems that didn't exist.

The Solution: The "Smart Receptionist" (L2D-Clinical)

This paper introduces a new system called L2D-Clinical. Think of it as a super-smart receptionist who stands between the patient and the doctors.

The receptionist doesn't diagnose the patient. Instead, they look at the patient's note and ask: "Which doctor is best suited for this specific piece of paper?"

The Check: The receptionist looks at the note.
- If the note is clear and uses standard medical terms (e.g., "The drug caused a headache"), the receptionist says, "This is easy. Doctor BERT can handle this." (Fast and cheap).
- If the note is vague, full of "maybe" and "possibly," or uses complex sentence structures, the receptionist says, "This is tricky. Doctor BERT might get confused. Let's send this to Doctor LLM." (Slow and expensive, but necessary).

The Big Surprise: Even a "Worse" Doctor Helps

The most interesting part of this research is what happened in one of their tests (the "ADE" task).

In this test, Doctor BERT was actually the better doctor overall. Doctor LLM was actually worse at the job, making more mistakes.

You might think, "Why would you ever send a note to the worse doctor?"

The answer is Complementary Strengths.

Doctor BERT was great at being precise but missed the "fuzzy" cases.
Doctor LLM was messy overall but was the only one who could catch those specific "fuzzy" cases.

The Smart Receptionist learned to say: "I know Doctor LLM usually makes mistakes, but for this specific type of vague note, they are the only one who can get it right. Let's pay the extra cost just for this one."

By doing this, the team didn't just use the "best" doctor; they built a hybrid team that was smarter than either doctor working alone.

The Results: Saving Money and Saving Lives

The system worked like magic in two different scenarios:

Scenario A (The Specialist is King): When the specialist (BERT) was already very good, the receptionist only sent 7% of the notes to the expensive genius (LLM). This small boost helped catch the few tricky cases the specialist missed, improving the overall score without breaking the bank.
Scenario B (The Genius is King): When the genius (LLM) was actually the better doctor, the receptionist sent 16% of the notes to them. This was still a huge saving because they didn't send everything to the expensive genius; they only sent the ones where the specialist was likely to fail.

Why This Matters

It's Cheaper: You don't need to pay the expensive "Genius Doctor" for every single patient. You only pay when it's truly necessary.
It's Smarter: It adapts. If the specialist is good at a task, it trusts them. If the specialist is bad at a task, it leans on the genius.
It's Explainable: Unlike a black-box AI that just gives an answer, this system can tell you why it made a choice. "I sent this to the Genius because the note used the word 'possibly' and was very long." This helps doctors trust the system.

The Bottom Line

L2D-Clinical is like having a smart traffic cop for your AI. It doesn't force you to choose between a fast, cheap car and a slow, luxury car. Instead, it directs the easy trips to the fast car and the difficult, winding roads to the luxury car. The result? You get to your destination faster, safer, and with less fuel.

1. Problem Statement

Clinical text classification faces a fundamental trade-off between specialized fine-tuned models (e.g., BioBERT, ClinicalBERT) and general-purpose Large Language Models (LLMs).

Specialized Models: Excel at pattern matching within specific domains due to in-domain pretraining but may struggle with implicit reasoning, hedging, or complex semantic nuances.
LLMs: Offer broad world knowledge and strong reasoning capabilities but are costly, have higher latency, and do not always outperform specialized models on specific tasks.

The Core Challenge: Neither model type dominates across all instances. Existing "Learning to Defer" (L2D) frameworks typically assume the "expert" (usually a human) is universally superior. However, in an AI-to-AI setting, the "expert" (LLM) may be worse overall than the base model (BERT) on a specific task. The paper asks: Can a system learn to selectively defer to a potentially weaker LLM to improve overall accuracy while minimizing costs?

2. Methodology: L2D-Clinical

The authors propose L2D-Clinical, a framework that trains a lightweight deferral model to decide when a BERT classifier should route an input to an LLM.

A. Problem Formulation

Given an input text $x$ , a base classifier $f_B$ (BERT), and an expert classifier $f_L$ (LLM), the system learns a deferral function $d(x) \in [0, 1]$ .

If $d(x) \geq \tau$ (threshold), the system queries the LLM ( $f_L(x)$ ).
Otherwise, it uses the BERT prediction ( $f_B(x)$ ).

B. Feature Engineering

The deferral model (Logistic Regression) utilizes two types of features to predict BERT's potential failure:

Uncertainty Features: Derived from BERT's softmax distribution:
- Confidence: Max probability of the top class.
- Entropy: Overall uncertainty across classes.
- Margin: Difference between the top two class probabilities.
Text Features: Task-specific linguistic signals:
- Length: Log of character/word counts.
- Causal Language: Indicators for explicit causality (e.g., "caused by," "induced") which BERT handles well.
- Outcome Keywords: Terms like "discontinued," "improved," or "tolerated."
- Hedging/Implicitness: Indicators of uncertainty that often trip up BERT but are handled well by LLMs.

C. Training Strategy

Objective: Minimize binary cross-entropy to predict whether BERT made an error ( $e(x)=1$ ) on a validation set.
Model: Logistic Regression with balanced class weights to handle the low error rate of BERT.
Optimization: The threshold $\tau$ is tuned on validation data to maximize the combined system's F1 score.

D. Key Design Choices

Interpretability: Logistic regression is chosen over complex ensembles (like Random Forest) to provide interpretable coefficients, allowing clinicians to understand why a case was deferred (e.g., "high entropy" or "presence of hedging").
AI-to-AI Deferral: Unlike prior work deferring to humans, this system learns fine-grained complementarity between two AI models.

3. Experimental Setup

The framework was evaluated on two distinct clinical tasks with opposing model dynamics:

Task 1: Adverse Drug Event (ADE) Detection (ADE Corpus V2)
- Scenario: BERT > LLM. BioBERT (F1=0.911) significantly outperforms the LLM (F1=0.765).
- Goal: Test if L2D can improve a strong BERT model by leveraging the LLM's high recall on specific hard cases.
- Data: 500 test samples (due to API costs).
Task 2: Treatment Outcome Classification (MIMIC-IV)
- Scenario: LLM > BERT. GPT-5-nano (F1=0.967) outperforms ClinicalBERT (F1=0.887).
- Ground Truth Innovation: Introduced a Multi-LLM Consensus method (GPT-5.2 + Claude 4.5 agreement) validated by human experts (100% accuracy on a 5% subset) to create high-quality labels.
- Goal: Test if L2D can optimize the usage of a superior LLM to avoid unnecessary API calls while maintaining peak performance.

4. Key Results

A. Performance Gains

Task	Base Model (F1)	LLM Only (F1)	L2D-Clinical (F1)	Improvement	LLM Usage Rate
ADE Detection	0.911 (BioBERT)	0.765	0.928	+1.7	7.0%
Treatment Outcomes	0.887 (ClinicalBERT)	0.967	0.980	+9.3	16.8%

ADE: Despite the LLM being weaker overall, L2D improved F1 by 1.7 points by deferring only 7% of cases where the LLM's high recall compensated for BERT's misses (e.g., hedged language).
MIMIC: L2D improved F1 by 9.3 points over BERT, outperforming even a fixed-threshold baseline, while using the LLM for only 16.8% of cases (vs. 26.2% for fixed thresholds).

B. Cost Efficiency

Cost Reduction: By deferring only a fraction of cases, L2D-Clinical reduces LLM API costs by 81–93% compared to using the LLM for all instances, while maintaining or improving accuracy.
Latency: Average latency dropped from ~850ms (LLM only) to ~71ms (ADE) and ~153ms (MIMIC).

C. Error Analysis & Interpretability

Complementary Errors: The system succeeds because BERT and LLM make different types of errors. BERT misses implicit/hedged relationships; LLM misses explicit patterns or over-predicts on ambiguous cases.
Feature Importance: Entropy was the strongest predictor of deferral. Task-specific features (e.g., "causal language" for ADE, "discontinued" for outcomes) helped the model learn when to trust BERT vs. the LLM.

5. Key Contributions

AI-to-AI Deferral Framework: First to demonstrate that L2D works effectively even when the "expert" (LLM) is not universally superior, learning instance-level complementarity instead of global superiority.
Adaptive Deferral Behavior: The system automatically adjusts deferral rates based on relative model strengths (7% deferral when BERT is strong; 16.8% when LLM is strong).
Consensus Ground Truth: A novel methodology for generating high-quality labels for MIMIC-IV using multi-LLM consensus validated by human experts.
Interpretability: Provides transparent, feature-based reasons for deferral decisions, crucial for clinical trust and validation.
Scalability: Demonstrates significant cost and latency reductions suitable for real-world EHR deployment.

6. Significance and Implications

Practical Deployment: L2D-Clinical offers a principled way to combine specialized and general models, allowing healthcare systems to leverage the reasoning power of LLMs without incurring prohibitive costs or latency.
Robustness: The approach is model-agnostic; it adapts automatically regardless of which model is currently stronger, making it future-proof against rapid changes in the LLM landscape.
Clinical Trust: The interpretability of the deferral mechanism (e.g., "deferred due to high entropy and hedging language") allows clinicians to validate the system's logic, addressing the "black box" concern of AI in healthcare.
Limitations: The approach requires labeled validation data for training (cold-start problem), relies on English-only data, and involves privacy considerations when sending PHI to external APIs.

In conclusion, L2D-Clinical proves that intelligent, learned routing between specialized and general AI models can outperform both individual models and simple heuristics, offering a scalable, cost-effective, and interpretable solution for clinical NLP.