💬 NLP

Training-Free Adaptation of New-Generation LLMs using Legacy Clinical Models

The paper introduces Cross-Architecture Proxy Tuning (CAPT), a training-free method that leverages existing legacy clinical models to adapt new-generation general-domain LLMs to the clinical domain through model ensembling and contrastive decoding, achieving superior performance without costly retraining.

Original authors: Sasha Ronaghi, Chloe Stanwyck, Asad Aali, Amir Ronaghi, Miguel Fuentes, Tina Hernandez-Boussard, Emily Alsentzer

Published 2026-04-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sasha Ronaghi, Chloe Stanwyck, Asad Aali, Amir Ronaghi, Miguel Fuentes, Tina Hernandez-Boussard, Emily Alsentzer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "New Car, Old Map" Dilemma

Imagine you just bought the latest, fastest, most advanced self-driving car (a new-generation AI model). It's incredible at reasoning, understanding complex traffic patterns, and driving smoothly. However, it was trained on general road maps and doesn't know the specific, tricky shortcuts, local customs, or hidden hazards of a specific hospital campus.

In the real world of medicine, hospitals have "legacy" maps (older clinical AI models) that know exactly how doctors write notes, what specific terms to use, and the safety rules for patient care. But these old maps are stuck on older, slower cars.

Usually, to make the new car drive safely on the hospital campus, you have to completely rebuild the engine and retrain the car from scratch using the old map. This is like taking apart the new car and rebuilding it every time a new model comes out. It's incredibly expensive, slow, and requires massive computing power that many hospitals don't have.

The Solution: CAPT (The "Co-Pilot" System)

The authors propose a clever trick called Cross-Architecture Proxy Tuning (CAPT). Instead of rebuilding the car, they install a Co-Pilot.

Here is how it works:

The Driver (New Model): The new, powerful AI model drives the car. It handles the steering, the speed, and the general flow of the conversation. It keeps the language fluent and logical.
The Co-Pilot (Old Clinical Model): The older, specialized medical AI sits in the passenger seat. It doesn't touch the steering wheel. Instead, it constantly whispers corrections to the driver.
The Whisper (Contrastive Decoding): When the driver is about to say something, the Co-Pilot checks: "Is this the right medical term? Is this the safe way to phrase this?"
- If the driver says, "Check the heart," and the Co-Pilot knows that for this specific surgery, you should check "perfusion" (blood flow), it whispers, "No, say 'perfusion'."
- The system then swaps the driver's choice for the Co-Pilot's suggestion only for that specific word, then lets the driver continue.

The Magic: This happens instantly, word-by-word, without ever needing to retrain the new car or rebuild the engine. It works even if the driver and the Co-Pilot speak slightly different "languages" (different technical vocabularies), because the system translates the Co-Pilot's advice on the fly.

What They Found: The "Best of Both Worlds"

The researchers tested this system on six different medical tasks, like classifying patient notes or writing treatment plans. Here is what happened:

Beating the Competition: The new car with the Co-Pilot (CAPT) performed significantly better than the new car alone, the old car alone, and even other methods that tried to combine models. On average, it was 17.6% better than the next best method.
Safety First: In terms of generating "risk-free" medical advice, CAPT was much safer than the other methods. It made fewer dangerous mistakes.
The Division of Labor: When they looked closely at the words the system chose, they saw a perfect split:
- The New Model controlled the grammar, the sentence structure, and the overall flow (making it sound natural).
- The Old Model controlled the specific medical jargon, the safety warnings, and the style of the clinical note (making it sound like a real doctor wrote it).

Real-World Examples from the Paper

The authors showed two specific examples where this "Co-Pilot" made a huge difference:

The Forearm Graft:
- Without CAPT: The new model suggested monitoring "cardiac" (heart) symptoms for a forearm surgery. This was a generic guess.
- With CAPT: The Co-Pilot corrected it to "perfusion" (blood flow) and "distal limb circulation." It also changed a vague timeline of "24-72 hours" to the more precise "24-48 hours" that doctors actually use.
The C-Section:
- Without CAPT: The new model suggested watching for "infection" as a reason to get up and walk around early.
- With CAPT: The Co-Pilot corrected this to "thromboembolism" (blood clots), which is the actual medical reason for early walking after this surgery. It also added specific checks for "fundal height" (a measure of the uterus) and "lochia" (post-birth bleeding), which are critical for safety but were missing from the generic model.

The Bottom Line

This paper proves that hospitals don't need to spend millions of dollars and years of time retraining their AI every time a new, smarter model is released.

By using CAPT, they can take a brand-new, powerful AI and instantly "teach" it the specific language and safety habits of their older, specialized medical AI. It's like giving a brilliant new intern a veteran mentor who whispers the right answers in their ear, resulting in a team that is both highly intelligent and clinically precise, all without a single day of extra training.

1. Problem Statement

The rapid advancement of general-domain Large Language Models (LLMs) has outpaced their adaptation to the clinical domain. While state-of-the-art general models possess superior reasoning and fluency, they often lack specific clinical knowledge, exhibit hallucinations, and fail to adhere to clinical documentation styles.

Current Limitation: Traditional domain adaptation (continued pretraining and instruction tuning) is computationally expensive and resource-intensive. It requires retraining for every new model generation (e.g., moving from MedPalm 1 to MedPalm 2), creating a lag where healthcare institutions cannot leverage new model capabilities without significant reinvestment.
Gap in Existing Solutions: Existing model ensembling methods (like UNiTE) and proxy tuning techniques generally assume shared tokenizers or overlapping capabilities. They struggle to combine models with disjoint vocabularies (different tokenizers) and asymmetric capabilities (e.g., a new general model vs. an older, specialized clinical model).

2. Methodology: Cross-Architecture Proxy Tuning (CAPT)

The authors propose CAPT, a training-free, inference-time model ensembling approach that combines a new-generation general-domain model ( $M_{new}$ ) with a legacy clinical model ( $M_{old-clin}$ ) and its base counterpart ( $M_{old}$ ).

Core Mechanism

CAPT operates at the decoding step using contrastive decoding to selectively inject clinical signals while preserving the general model's reasoning.

Candidate Selection: At each decoding step, the system selects the top- $k$ candidate tokens (default $k=20$ ) from the new-generation general model ( $M_{new}$ ).
Cross-Architecture Mapping: Since $M_{new}$ $M_{n e w}$ and the clinical models have disjoint vocabularies, CAPT employs a decode–retokenize mapping ( $f$ $f$ ):
- A token from $M_{new}$ 's vocabulary is decoded to its string form.
- The string is re-tokenized using the clinical model's tokenizer ( $T_{old}$ ) to find the corresponding token in the clinical vocabulary.
Log-Probability Adjustment: The log-probability of each candidate token is adjusted using a "clinical knowledge offset." This offset is the difference between the log-probabilities of the clinical model ( $M_{old-clin}$ $M_{o l d - c l in}$ ) and its untrained base ( $M_{old}$ $M_{o l d}$ ):
$s(i) = \log p_{new}(i | x_{1:t}) + \alpha \left( \log p_{old-clin}(f(i) | x_{1:t}) - \log p_{old}(f(i) | x_{1:t}) \right)$
- Where $\alpha$ is a scaling factor (set to 1.0).
- This delta isolates the knowledge learned during clinical fine-tuning, removing the base model's general language bias.
Selection: The token with the highest adjusted score is selected and appended to the context.

Implementation Details

Models Used:
- $M_{new}$ : Qwen3-30B (New-generation general model).
- $M_{old-clin}$ : MeLLaMA-13B-chat (Legacy clinical model).
- $M_{old}$ : LLaMA-2-13B-base (Base counterpart).
Constraints: The method supports black-box APIs where only top- $k$ log-probabilities are exposed, making it practical for deployment.

3. Key Contributions

Training-Free Adaptation: Introduces CAPT, enabling the reuse of legacy, domain-adapted models to adapt new general models without any additional training or fine-tuning.
Disjoint Vocabulary Support: Solves the architectural barrier of combining models with different tokenizers via the decode–retokenize mapping, allowing the integration of older clinical models with modern architectures.
Selective Integration: Demonstrates that CAPT does not merely average probabilities but selectively amplifies tokens related to clinical decision-making and documentation style while letting the general model control linguistic structure and fluency.

4. Results

The method was evaluated on six clinical tasks (3 classification, 3 text generation) using metrics like Macro-F1, Accuracy, MedHELM LLM-jury scores, and MedVAL risk-free percentages.

Performance Gains:
- CAPT consistently outperformed both the individual models and state-of-the-art baselines (UniTE and Proxy Tuning).
- Average Improvement: +17.6% over UniTE and +41.4% over Proxy Tuning across all metrics.
- Safety: Significant improvements in "risk-free" outputs (e.g., +13.94% over UniTE on text generation), indicating better clinical safety.
Token-Level Analysis:
- Clinical Model Influence: Strongly influenced tokens related to clinical decision-making (e.g., headers, assessment terms) and documentation style (e.g., hedging, reporting verbs).
- General Model Influence: Retained control over linguistic structure, formatting, and general morphemes.
- Medical Knowledge: Found near-zero shifts for core medical terminology, suggesting both models agree on facts, but CAPT improves how that knowledge is expressed to match clinical conventions.
Physician Case Studies:
- Two board-certified physicians reviewed outputs, noting CAPT improved clinical specificity (e.g., changing "agents" to "analgesics"), contextual accuracy (e.g., correcting monitoring timelines from 24-72h to 24-48h), and safety-critical recommendations (e.g., emphasizing postpartum hemorrhage signs over generic infection signs).

5. Significance and Impact

Resource Efficiency: CAPT offers a viable path for healthcare institutions with limited computational resources to adopt cutting-edge LLMs without the prohibitive cost of retraining.
Modular Adaptation: It transforms legacy clinical models into reusable "adapters" that can be paired with future foundation models, accelerating the translation of AI advances into clinical workflows.
Bridging the Gap: It effectively addresses the "lag" between general model capabilities and clinical applicability, ensuring that new models inherit the safety and stylistic nuances of established clinical AI without sacrificing their reasoning power.
Limitations & Future Work: The method increases inference compute (requiring three model passes) and may inherit over-explanatory styles from general models. Future work involves adaptive scaling of the adjustment coefficient ( $\alpha$ ) based on user expertise and exploring smaller clinical models to reduce latency.

In conclusion, CAPT represents a paradigm shift from "retraining for every model" to "adapting via ensembling," providing a practical, high-performance solution for clinical NLP deployment.