🤖 machine learning

AdaPCLA: Adaptive Prior-Calibrated Logit Adjustment for Long-Tailed Longitudinal EHR Generation

The paper proposes AdaPCLA, an adaptive prior-calibrated logit adjustment framework that leverages simulated annealing and data distribution-aware training to significantly improve the fidelity of rare event generation and zero-shot cross-population adaptation in longitudinal Electronic Health Record modeling.

Original authors: Shuai Cui, Chen Wenxuan, Wenjie Du, Jian Lou, Dan Li, Wenjie Feng

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuai Cui, Chen Wenxuan, Wenjie Du, Jian Lou, Dan Li, Wenjie Feng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to write a cookbook that perfectly mimics the eating habits of a whole city. You have millions of recipes for pizza and burgers (the "head" events), but only a handful of notes on how to cook a rare, exotic mushroom dish (the "tail" events).

If you just let your cooking AI learn by tasting everything, it will get really good at making pizza. But when it tries to make that rare mushroom dish, it will likely mess up, because the mushroom recipes are so few and far between that the AI barely notices them. It might even decide mushrooms don't exist at all! This is exactly the problem doctors face with Electronic Health Records (EHR). Most patients have common conditions like colds or high blood pressure, but a few have rare, complex diseases. Standard AI models tend to ignore these rare cases, making the fake medical records they generate useless for studying the patients who need it most.

Enter AdaPCLA, a new method that acts like a very smart, patient cooking instructor.

The Problem: The "Pizza Bias"

In the world of medical data, common diseases drown out rare ones. When an AI tries to learn from this data, it gets overwhelmed by the "loud" signals of common conditions. The quiet, rare signals get lost. Previous attempts to fix this were like shouting at the AI to "pay attention to the mushrooms!" during the whole cooking class. While this helped a bit, it meant the AI never actually learned how to cook the mushrooms on its own; it just kept relying on the instructor's shouting. If you took the instructor away (or if you tried to cook for a different city with different mushroom habits), the AI would fail.

The Solution: The "Training Scaffold"

The authors of this paper propose a clever trick called AdaPCLA. Think of it as a temporary training scaffold, like the wooden supports builders put up while constructing a tall tower.

The Scaffold (Early Training): At the start of training, the system adds a "bias" or a helpful nudge. It's like the instructor whispering, "Hey, remember, mushrooms are rare but important!" This nudge makes the AI pay extra attention to those rare medical codes, ensuring it doesn't ignore them.
The Slow Pull-Down (Annealing): Here is the magic part. As the AI gets better, the instructor slowly stops whispering. The "scaffold" is gradually removed. The goal is for the AI to internalize the lesson. By the time the training is done, the AI has learned the structure of rare diseases inside its own brain, not just by following an external rule.
The Result (Inference): When the AI is finally ready to cook (generate data) on its own, the scaffold is gone. It doesn't need any external help to remember the rare mushrooms. It just knows.

Why This Matters: The "Zero-Shot" Magic

The paper shows that this method doesn't just make the AI better at the specific city it trained in; it makes it adaptable. Imagine you trained your chef in New York (where they eat a lot of bagels). With AdaPCLA, you can send that chef to Tokyo (where they eat a lot of rice) without retraining them. You just give them a quick note: "In Tokyo, rice is the main dish." Because the chef learned the structure of cooking, not just the specific ingredients, they can instantly adjust to the new population.

The paper calls this zero-shot distribution control. It means the model can adapt to a new group of patients with different disease frequencies without needing to see a single new patient record first.

What the Numbers Say

The researchers tested this on real-world data from two massive hospital databases, MIMIC-III and MIMIC-IV. They found that AdaPCLA was significantly better at generating realistic rare events than previous methods.

On MIMIC-III, it improved the ability to generate rare pairs of conditions (TailPairSeen) by 114.2% compared to the previous best method (HALO).
On MIMIC-IV, that improvement was 65.1%.
When trying to adapt from one population to another without retraining, it beat standard "GPT-style" generation by 3.5% in F1 score, a measure of accuracy.

What It's NOT

It's important to note what this paper doesn't claim. The authors explicitly argue against using a fixed, permanent bias. They showed that if you keep the "scaffold" (the external nudge) on forever, the AI becomes dependent on it and fails when that help is removed. They also note that while their method works great for diagnosis codes, they haven't yet tested it on other types of medical data like images or free-text notes.

The Bottom Line

The paper suggests that by treating the "help" as a temporary tool that gets slowly removed, we can teach AI to truly understand the rare and complex parts of medical history. It's not just about making the AI smarter; it's about making sure it doesn't forget the patients who are often invisible in the data. The results show that this approach creates more realistic, useful, and adaptable synthetic medical records, which could be a huge step forward for privacy-preserving medical research.

Technical Summary: AdaPCLA for Long-Tailed Longitudinal EHR Generation

Problem Statement

Longitudinal Electronic Health Records (EHR) are critical for clinical applications such as risk prediction and medication recommendation. However, the sharing of real-world EHR data is hindered by privacy regulations, necessitating high-quality synthetic data generation. Existing generative models (e.g., GANs, VAEs, autoregressive models) struggle with the long-tailed distribution inherent in medical codes. While common clinical patterns are frequent, rare but clinically significant conditions (tail events) provide sparse learning signals. Standard autoregressive models tend to optimize for dominant "head" patterns, causing them to underrepresent the co-occurrence structure of rare events. This results in synthetic data that appears realistic on average but fails to preserve the fidelity and faithfulness required for rare subpopulations, limiting downstream utility and cross-population adaptability.

Methodology: AdaPCLA

The authors propose AdaPCLA (Adaptive Prior-Calibrated Logit Adjustment), a curriculum-based framework designed to internalize data distribution knowledge into model parameters through a simulated annealing process. The framework consists of three core components:

EHR Generator Backbone: An autoregressive Transformer that models patient visit sequences as multi-hot vectors of medical codes. It captures temporal and cross-event dependencies via shared hidden states.
Statistical Prior Modeling (The Scaffold): To address the sparsity of rare codes, the method constructs a label-wise bias term ( $b_{stat}$ ) derived from empirical marginal frequencies. This bias shifts the effective logits of rare codes toward zero (the high-sensitivity region of the sigmoid function) during early training, amplifying the gradient updates from sparse positive occurrences relative to dominant head codes.
Curriculum Prior Injection (Annealing): Unlike static logit adjustment methods that rely on external corrections during inference, AdaPCLA treats the prior bias as a temporary optimization scaffold. A schedule coefficient $\alpha(\tau)$ $α (τ)$ gradually anneals the bias from 1 to 0 over the training steps.
- Training: The model optimizes using effective logits $\tilde{z} = z + \alpha(\tau)b_{stat}$ .
- Inference: The bias is completely removed ( $\alpha=0$ ), forcing the model to rely solely on its learned internal parameters to generate rare events, ensuring bias-free inference.

Zero-Shot Cross-Population Adaptation:
AdaPCLA supports training-free adaptation to new populations with different code frequencies. By applying a target-prior correction difference ( $\Delta b$ ) to the logits at inference time, the model can adjust generation probabilities to match a target distribution without retraining, provided the conditional dependencies remain consistent.

Theoretical Analysis

The paper provides theoretical justification for the method's efficacy:

Gradient Transfer: The authors analyze how the prior scaffold affects intrinsic logit updates through the label-wise empirical Neural Tangent Kernel (NTK). The scaffold reshapes Binary Cross-Entropy (BCE) residuals, and these effects are transferred to intrinsic logits via the NTK structure, strengthening early rare-code learning signals.
Prior-Internalization Bound: The authors derive a bound showing that the error between the learned intrinsic logits and the moving target (as the scaffold is removed) depends on the annealing speed and the minimum eigenvalue of the empirical NTK. A smoother annealing schedule and a well-conditioned NTK ensure that the prior signals are successfully internalized into the model parameters rather than being lost when the scaffold is removed.

Key Contributions

Framework Proposal: Introduction of AdaPCLA, a prior-guided curriculum framework that strengthens rare-code learning while enabling bias-free inference and zero-shot distribution control.
Theoretical Insight: Characterization of rare-code logit updates via label-wise empirical NTK dynamics and the derivation of a prior-internalization bound linking annealing speed and NTK conditioning to signal retention.
Empirical Validation: Demonstration of significant gains in tail plausibility, downstream utility, and cross-population adaptation on real-world datasets.

Experimental Results

Experiments were conducted on MIMIC-III and MIMIC-IV datasets, comparing AdaPCLA against baselines including GPT-style models, LSTM, EVA, SynTEG, and HALO.

Tail Clinical Plausibility: AdaPCLA achieved the strongest performance on tail-focused metrics.
- On MIMIC-III, it improved TailPairSeen by 114.2% over HALO (0.1114 vs. 0.0520).
- On MIMIC-IV, it improved TailPairSeen by 65.1% over HALO (0.0822 vs. 0.0498).
- It also demonstrated superior tail-context consistency (lower TailCtxJSD) compared to baselines.
Downstream Utility: Using the Train-on-Synthetic, Test-on-Real (TSTR) protocol, AdaPCLA achieved the highest Accuracy, F1, and AUPRC scores for 25-label diagnosis classification on both datasets, outperforming the GPT-style baseline by 3.5% F1 in zero-shot cross-population adaptation.
Ablation Studies:
- Static Bias vs. Annealing: Models using a fixed bias during training but removing it at inference (Static-PCLA Train-only) failed completely (TailPairSeen dropped to near zero), confirming that the bias alone does not teach the structure.
- Internalization: AdaPCLA's annealing approach successfully internalized the prior, achieving the best metrics without external bias at inference.
- Zero-Shot Control: The model successfully adapted from MIMIC-IV to MIMIC-III using only target prior corrections, achieving an AUPRC of 0.922 without retraining.

Significance and Claims

The paper claims that AdaPCLA resolves the trade-off between fitting the overall data distribution and preserving rare-event dependencies. By internalizing prior knowledge through a curriculum annealing process, the method generates high-fidelity synthetic EHRs that are reliable for rare subpopulations. This is significant because it enables privacy-preserving research and facilitates the transfer of generative models to new clinical settings (different hospitals or populations) without the need for expensive retraining, addressing a critical bottleneck in medical AI development.

Limitations: The authors note that current evaluation is limited to diagnosis-code sequences from two public benchmarks. Future work is expected to validate the method across diverse hospitals and richer EHR modalities beyond diagnosis codes.