🤖 machine learning

TEDDY: A Pediatric Foundation Model for Risk Forewarning from ICD-Coded Diagnostic Histories

The paper introduces TEDDY, a compact 1.84-million-parameter decoder transformer trained on pediatric ICD-10 data that significantly outperforms larger and more complex models in forecasting disease onset and visit timing, demonstrating that pediatric diagnostic histories can effectively support generative foundation models for rare and long-horizon risk prediction without requiring population-scale datasets.

Original authors: Matthew Brady Neeley, Jorge Botas, Johnathan Jia, Lin Yao, Daniel Palacios, Benjamin Choi, Zhandong Liu, Hyun-Hwan Jeong

Published 2026-07-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Matthew Brady Neeley, Jorge Botas, Johnathan Jia, Lin Yao, Daniel Palacios, Benjamin Choi, Zhandong Liu, Hyun-Hwan Jeong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future of a story, but the characters are growing up, changing their voices, and learning new things every single day. This is the challenge of pediatric medicine: a child is not just a small adult. Their bodies, their diseases, and even how they get sick change rapidly as they develop from a newborn into a teenager. For a long time, doctors and computer programs have mostly learned from stories about grown-ups, which often leads to wrong guesses when applied to kids. It's like trying to use a map of a city to navigate a forest; the rules are different, and the landmarks don't match.

To make better predictions, scientists are building "foundation models." Think of these as super-smart computers that read millions of medical records to learn the patterns of health and sickness. Usually, these models are huge, like giant libraries of knowledge, and they are trained on data from adults. But what if we could build a smaller, specialized model that understands the unique, fast-paced story of childhood? That is the big question this paper tackles: Can a computer learn to spot the early signs of a child's future illness just by reading their past medical visits, even for rare diseases that haven't happened yet?

Meet TEDDY (Temporal Event Decoder for Disease in Youth). You can think of TEDDY as a super-attentive pediatrician with a perfect memory of every child it has ever seen. Instead of just looking at a child's current symptoms, TEDDY reads their entire medical history like a novel, paying close attention to the order of events and the time between them. Its job is to guess what might happen next: Will this child develop asthma? Could they have a rare genetic condition? And crucially, when might that happen?

The team trained TEDDY on a massive collection of medical records from 1.6 million children at Texas Children's Hospital. They fed the model about 73 million diagnosis codes (the standard medical shorthand for illnesses) to learn the patterns. TEDDY is surprisingly small for a computer brain of this power, with only 1.84 million "parameters" (the internal knobs and dials that make it smart). To put that in perspective, it is much smaller than the giant language models you might have heard of, which often have billions of parameters.

So, what did TEDDY find?

It's a Great Predictor, Even for Rare Things
TEDDY is really good at its job. When tested on 797 different diseases, it correctly ranked children who would get a specific illness higher than those who wouldn't 72% of the time (a score known as AUC). This is a big deal because most computer models struggle with rare diseases, often getting confused or giving up. But TEDDY actually got better at predicting the rarest conditions. For the 225 rarest diseases (those affecting fewer than 5 out of every 10,000 children), the model was still confident and accurate, with its predictions being far better than random guessing.

It Sees the Future Years in Advance
One of the most exciting discoveries is how far ahead TEDDY can see. The model can spot the "warning signs" of a disease more than two years before a doctor officially writes down the diagnosis. While the predictions are strongest right before the diagnosis happens, the signal remains detectable years earlier. This suggests that a child's medical history holds clues about their future health long before the illness fully takes hold.

It Beats the Competition
The researchers didn't just build TEDDY; they put it in a race against other computer models. They compared it to older types of AI (like RNNs and LSTMs) and even a massive, general-purpose language model that is 1,000 times larger than TEDDY.

For predicting asthma, TEDDY scored 79.3%, while the best competitor (a giant language model) only scored 62.7%.
For predicting ADHD, TEDDY scored 84.7%, beating the next best model which scored 71.7%.

This proves that you don't need a gigantic, billion-dollar computer to understand children's health. A model built specifically for kids, trained on their unique stories, works much better than a giant model built for adults.

How It Avoids Cheating
The scientists were very careful to make sure TEDDY wasn't "cheating." In medical records, a doctor might write down several diagnoses on the same day. A tricky AI might look at the first diagnosis to guess the second, which isn't fair. TEDDY was designed to make its prediction before seeing any of the diagnoses for that day, forcing it to rely only on the child's past history. They also made sure to compare sick children only with other sick children of the same age and gender, so the model couldn't just guess based on whether a child was a boy or a girl.

It's Not Perfect (Yet)
TEDDY is great at guessing what might happen, but it's a bit less precise about exactly when. It can estimate the time until the next doctor's visit with an average error of just 3 days, which is impressive. However, it sometimes struggles with very long gaps between visits, overestimating how long a child might wait before returning. The researchers admit this is a limitation of their current design, which uses a simple mathematical rule to guess time.

The Bottom Line
This paper shows that we can build compact, powerful AI tools specifically for children that outperform massive, adult-focused models. By treating a child's medical history as a unique, unfolding story, TEDDY can spot risks for common and rare diseases years in advance. While it's not a crystal ball that tells us everything perfectly, it's a powerful new tool that could help doctors catch illnesses earlier, especially the rare ones that often go undiagnosed for years. The authors suggest that the next step is to test this model across different hospitals and to add more types of data, like growth charts and lab results, to make the predictions even sharper.

Technical Summary: TEDDY – A Pediatric Foundation Model for Risk Forewarning

Problem Statement

Pediatric electronic health records (EHRs) capture developmentally structured clinical trajectories that differ fundamentally from adult data in anatomy, physiology, and disease presentation. However, existing clinical prediction tools and AI/ML models are predominantly trained on adult populations, leading to unreliable predictions for children. This gap is particularly acute for rare and developmental conditions (e.g., congenital disorders, autism spectrum disorder) where adult records offer no relevant priors.

Current generative foundation models for EHRs (e.g., Delphi, ReClaim, Curiosity) are trained on adult or mixed-age cohorts and often suffer from two methodological pitfalls when applied to pediatric data:

Within-encounter leakage: Models may inadvertently predict a diagnosis using other codes from the same visit day, inflating accuracy because EHR systems record diagnoses by calendar day without internal ordering.
Demographic confounding: Models may achieve high performance by learning demographic correlations (e.g., sex-specific diseases) rather than genuine clinical signals.

Furthermore, pediatric data is scarce, fragmented, and often lacks the population-scale datasets required to train billion-parameter models. There is a critical need for a generative model specifically designed for children that can anticipate incident diagnoses and visit timing without relying on massive data scales or adult-derived priors.

Methodology

Data and Cohort

The model, TEDDY (Temporal Event Decoder for Disease in Youth), was trained on data from Texas Children's Hospital (TCH).

Cohort: 1.6 million pediatric patients (age ≤25 years).
Data Volume: Approximately 73 million ICD-10 diagnosis events.
Vocabulary: A closed vocabulary of 1,849 tokens, including 1,841 ICD-10-derived codes (truncated to chapter/subcategory) and 8 special tokens (e.g., VISIT_START, VISIT_END, NO_EVENT, BIRTH, SEX_*).
Preprocessing:
- Gap-filling: NO_EVENT tokens are inserted between real events based on the American Academy of Pediatrics (AAP) well-child visit schedule to define time intervals during periods of no care.
- Visit Bracketing: All codes from the same calendar day are grouped between VISIT_START and VISIT_END tokens.
- De-identification: Temporal data is stored solely as age in days from birth; no calendar dates or protected health information (PHI) are retained.
- Exclusions: Mother-infant dyads were excluded from pretraining to allow for future intergenerational transfer experiments.

Model Architecture

TEDDY is a decoder-only transformer with 1.84 million parameters (8 layers, 8 attention heads, embedding dimension 128).

Inputs: Sequences of (token, age) pairs. Age is encoded via sinusoidal features mapped to years, summed with learned token embeddings. No learned positional embeddings are used; temporal ordering is derived from the age channel and causal attention masks.
Training Objective: A joint next-token and time-to-event objective:
1. Categorical Cross-Entropy ( $L_{CE}$ ): Predicts the next token (diagnosis code, NO_EVENT, or visit bracket) over the 1,849-token vocabulary.
2. Time-to-Event Loss ( $L_{\Delta t}$ ): Predicts the rate ( $\lambda_t$ ) of an exponential distribution for the time until the next visit boundary. NO_EVENT targets are treated as right-censored observations.
Loss Function: $L = L_{CE} + L_{\Delta t}$ .

Evaluation Protocol

The evaluation was designed to eliminate common sources of inflated performance:

Visit-Boundary Scoring: Predictions are made at the VISIT_START token, before any codes from that specific day are revealed. This prevents within-encounter leakage.
First-Occurrence Only: Models are scored only on the first appearance of a diagnosis in a child's record, focusing on anticipating new conditions rather than re-predicting known ones.
Demographic Matching: For each condition, cases are matched with controls of the same sex and similar age bands (using a cascading fine-to-coarse matching strategy) to ensure performance reflects clinical signal rather than demographic stratification.
Benchmarks: TEDDY was compared against:
- Same-data baselines: DenseNet, CNN, RNN, LSTM (all size-matched).
- A general-purpose language model: Gemma-3-4b (4 billion parameters, ~2000x larger), prompted on the same patient histories.

Key Results

Diagnostic Risk Prediction

Across 797 disease-onset prediction tasks spanning 16 ICD-10 chapters:

Overall Performance: TEDDY achieved a median AUC of 72.0%.
Comparison: It outperformed same-data baselines (DenseNet: 50.0%, CNN: 57.2%, RNN: 60.1%, LSTM: 62.7%) on 96–99% of tasks.
Rare Disease: Performance was strongest for rare conditions. Of the 225 rarest conditions (prevalence < 5 per 10,000), 90% (202 conditions) had 95% confidence intervals entirely above chance.
Demographics: Performance remained consistent across sex and age strata, indicating the signal is not driven by demographic separability.
Specific Benchmarks:
- Asthma (J45): TEDDY AUC = 79.3% vs. Gemma-3-4b (62.7%) and LSTM (60.0%).
- ADHD (F90): TEDDY AUC = 84.7% vs. Gemma-3-4b (51.2%) and LSTM (71.7%).
- Note: Gemma-3-4b, despite being 2000x larger, performed near chance for ADHD and significantly worse than TEDDY for both conditions.

Temporal Forewarning

The model's ability to anticipate diagnoses was evaluated at various lead times:

Signal Persistence: Predictive signal remained detectable more than two years before the first recorded diagnosis.
- Median AUC at >2 years lead time: 59.7% (unrestricted analysis) and 64.4% (fixed-cohort sensitivity analysis).
- Median AUC on the same day: 71.8%.
This suggests the model captures long-horizon trajectory structures rather than just short-term correlations.

Visit Timing Prediction

TEDDY estimates the time to the next visit using a restricted mean survival time (RMST) metric:

Accuracy: Mean absolute error of 3.0 days over a 365-day horizon.
Calibration: The model captures child-specific visit rates (Pearson $r=0.54$ for log-transformed times) but exhibits drift in the long tail, overestimating the probability of no return visit by one year (predicted 0.399 vs. observed 0.296). This is attributed to the simplicity of the single-exponential rate head.

Significance and Claims

The paper claims that TEDDY establishes pediatric diagnostic histories as a viable substrate for compact generative models capable of broad, rare-disease, and long-horizon risk forecasting.

Key contributions and claims include:

Pediatric-Specific Foundation Model: TEDDY is the first generative EHR foundation model built specifically for children, addressing the "population gap" where adult models fail to generalize to pediatric physiology and disease presentation.
Data Efficiency: The model demonstrates that faithful representation of clinical trajectories (via visit bracketing and age encoding) can yield clinically meaningful predictions even with small data (1.6M patients) and compact models (1.84M parameters), challenging the paradigm that EHR performance requires billion-parameter models and population-scale data.
Rigorous Evaluation: By implementing strict safeguards against within-encounter leakage and demographic confounding, the paper argues that the reported performance reflects genuine predictive structure rather than artifacts.
Clinical Utility: The ability to detect signals years in advance, particularly for rare conditions where diagnostic odysseys are common, suggests potential for earlier intervention and referral in routine pediatric care.

The authors conclude that while the model is a first step, future work should expand inputs to include measurements and medications, generalize across institutions, and explore intergenerational (mother-child) trajectories. They explicitly note limitations regarding single-institution data and the exclusion of non-diagnostic structured data (labs, vitals).