💻 computer science

Adaptive Temporal Latent Architecture using Structured Electronic Health Records: A Temporal Physiologic-State Centric Foundation Model

The paper introduces ATLAS-EHR, a self-supervised foundation model that learns reusable, temporally-aware representations from structured electronic health records to outperform task-specific models across diverse clinical prediction tasks and enable robust imputation and physiological signal analysis.

Original authors: David Chen, Xiaotan Sun, H.W. Wilson Tang, Po-Hao Chen, Christopher Nguyen, Xiaofeng Wang, Deborah Kwon

Published 2026-11-05

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: David Chen, Xiaotan Sun, H.W. Wilson Tang, Po-Hao Chen, Christopher Nguyen, Xiaofeng Wang, Deborah Kwon

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Hospitals generate a constant, flowing river of data about every patient who walks through their doors. From the moment a person arrives, machines and nurses record thousands of measurements: heart rates, blood pressure, chemical levels in the blood, and the timing of every medication. This digital record, known as an electronic health record, is a massive archive of human biology in motion. However, for decades, the way doctors and computers have used this information has been clumsy. To make sense of the data, researchers often forced these complex, irregular streams into rigid, static boxes. They would take a patient's entire history and flatten it into a single snapshot, or they would fill in missing numbers with simple guesses, like assuming a patient's condition today is exactly the same as it was yesterday. In doing so, they threw away the rich, messy, and vital details of how a patient's body actually changes over time. The field of medical artificial intelligence has long sought a way to understand this data not as a list of facts, but as a continuous story of a living person.

A team of researchers at the Cleveland Clinic has taken a significant step toward solving this problem with a new system they call ATLAS-EHR. Instead of trying to force patient data into a fixed table, they built a model that learns to understand the patient's state as it evolves, moment by moment. Imagine a patient's health as a complex, shifting landscape where different measurements rise and fall together. The researchers trained their system on millions of real patient records from a public database called MIMIC-IV. The system did not memorize specific diseases or outcomes. Instead, it learned to look at a collection of measurements taken at the same time, understand how they relate to each other, and then use that understanding to predict what a missing measurement might be. It is like learning the grammar of a language by reading thousands of sentences, rather than memorizing a dictionary of words. By focusing on the relationships between numbers rather than the numbers themselves, the model learned a deep, flexible representation of what a patient is experiencing at any given second.

The results of this approach were striking. When the researchers tested the model by hiding random numbers from the patient records and asking the system to fill them in, it performed far better than standard methods. While older techniques that simply copied the last known value failed to capture the nuance of a changing condition, ATLAS-EHR used the surrounding context to make a smart, educated guess. It achieved a level of accuracy that reduced the error rate by more than half compared to the best existing methods. Crucially, the system also knew when it was unsure. It could signal that a missing piece of information was too difficult to guess because the surrounding data was too sparse, offering a built-in warning that prevented it from making up false facts. This ability to quantify its own uncertainty is a vital safety feature for any tool used in medicine.

Perhaps the most powerful finding was that this single, learned understanding of patient states could be applied to many different medical questions without needing to be retrained from scratch. The researchers took the core "brain" of the model and froze it, then attached simple, lightweight tools to it to solve seven different problems, ranging from predicting if a patient would die in the hospital to estimating how long they would stay. In six of these seven tasks, the model outperformed the best specialized tools currently used by hospitals. It was able to distinguish between different types of illness and predict outcomes with greater precision than models that had been built specifically for those single tasks. This suggests that the model had learned a universal language of human physiology that applies across many different diseases.

The team also tested whether this knowledge could travel to a different hospital. They applied the model trained on data from one major medical center to a completely different set of patients from a network of hospitals across the United States. While the model could not predict outcomes perfectly on its own in this new setting, it retained its core understanding of patient states. When the researchers simply retrained the final step of the process to match the new hospital's specific patterns, the system's performance jumped dramatically, matching or exceeding the best local tools. This indicates that the fundamental patterns of human physiology learned by the model are portable and robust, even when the way data is collected changes from one place to another.

Digging deeper into the model's internal logic, the researchers found that it had organized the data into distinct clusters that mirrored real human biology. The system naturally grouped patients into states that looked like specific medical conditions, such as kidney failure or severe infection, without ever being told to look for those diseases. It identified patterns where high levels of certain chemicals in the blood consistently appeared together, or where a drop in consciousness was linked to specific changes in breathing. These groupings were not random; they corresponded to coherent physiological states that doctors would recognize. Furthermore, when the researchers tested the model on patients with chronic kidney disease or severe sepsis, they found that the system relied on the correct medical evidence. For kidney disease, it focused heavily on the specific chemical markers of kidney function. For sepsis, it drew on a broader mix of signals involving the heart, lungs, and metabolism. This proved that the model was not just finding shortcuts or memorizing patterns of hospital paperwork, but was actually learning the biological signals of the diseases themselves.

The study also highlighted what the model does not do, which is just as important as what it does. The researchers deliberately excluded information about what doctors decided to do, such as the names of medications prescribed or the specific diagnoses written in the chart. They wanted the model to learn only from the raw observations of the patient's body. By doing this, they ensured that the model learned the state of the patient, rather than the state of the hospital's response to the patient. This distinction is critical for understanding the true progression of illness, separate from the treatments applied to it. The model learned to see the patient's physiology clearly, even when the medical record was incomplete or irregular.

In the end, this work suggests a new way to think about medical data. Instead of treating a patient's history as a static list of facts to be mined for specific answers, the researchers showed that it is possible to build a system that understands the continuous, dynamic story of a human life. The model learned to see the connections between a heart rate, a blood test, and a breathing pattern as a single, evolving picture of health. It demonstrated that a machine can learn to fill in the gaps of a medical record with intelligent, context-aware guesses, and that this same understanding can be reused to solve a wide variety of medical problems. While the system is not yet a replacement for a doctor, it offers a powerful new lens through which to view the complex, messy, and beautiful reality of human health data.

Technical Summary: Adaptive Temporal Latent Architecture using Structured Electronic Health Records (ATLAS-EHR)

1. Problem Statement

Electronic Health Records (EHRs) constitute a vast, longitudinal archive of human health data characterized by high dimensionality, asynchrony, and heterogeneity. Despite this abundance, clinical modeling often reduces these complex records into static, fixed-feature tables for task-specific prediction, discarding rich temporal and multivariable information. Existing deep learning approaches for structured EHRs typically rely on one-hot encoding, simple imputation, or fixed-window aggregation, which impose arbitrary assumptions about event alignment and ordering. Furthermore, many current models focus on reconstructing the record itself or predicting the next event, rather than learning a shared representation of the underlying patient state. A critical challenge remains in distinguishing between signals reflecting the evolving physiological state of a patient versus signals reflecting clinical decisions, interventions, or documentation workflows, which can obscure the true state representation.

2. Methodology

The authors introduce ATLAS-EHR (Adaptive Temporal Latent Architecture using Structured Electronic Health Records), a foundation model based on a Joint-Embedding Predictive Architecture (JEPA). The model is designed to learn reusable representations from routinely collected structured EHR data without optimizing for specific downstream endpoints during pretraining.

Core Design Principles:

Permutation Invariance: Concurrent observations at the same timestamp are integrated without imposing an intrinsic order, utilizing a SetEncoder.
Irregular Temporal Context: Successive clinical states are contextualized over irregular time intervals using signed relative-time attention rather than absolute positional embeddings.
Latent Space Prediction: Instead of reconstructing raw input values (which may emphasize noise or institution-specific patterns), the model predicts the latent representation of a withheld observation based on its multivariable and longitudinal context.

Architecture Details:

Input Processing: Structured observations (numeric and categorical) are encoded using variable identity, type, and standardized value. Numeric values are expanded to include unsigned quadratic magnitude and sign indicators.
Hierarchical Encoding:
- SetEncoder: Uses multi-head self-attention to exchange information among concurrently observed events at a single timestamp, producing a local timestamp-level state ( $z_{set}$ ).
- Temporal Transformer: A bidirectional Transformer processes the sequence of timestamp states, incorporating temporal information via signed relative-time embeddings ( $\Delta \tilde{t}$ ).
Self-Supervised Pretraining (JEPA):
- Masking: Selected numeric events are masked from the online context pathway.
- Target Generation: An Exponential Moving Average (EMA) target encoder generates the latent representation of the masked event directly from the true value, bypassing the SetEncoder and temporal Transformer.
- Prediction: A value-blind predictor queries the latent target using only the variable identity, combined with local and global contextual representations. The objective is to minimize the L1 distance between the predicted latent representation and the stop-gradient EMA target.
Data Constraints: The model was trained exclusively on "patient state" observations (demographics, labs, vitals, bedside assessments), deliberately excluding diagnostic codes, medications, procedures, and free-text to prevent the model from learning downstream clinical decisions rather than physiological states.

Datasets:

Training/Validation: MIMIC-IV v3.1 (Beth Israel Deaconess Medical Center).
External Validation: eICU-CRD v2.0 (Multi-center ICU data).

3. Key Contributions

Foundation Model for Structured EHRs: ATLAS-EHR is the first JEPA-style foundation model specifically designed for structured, irregular, multivariable EHR streams, moving beyond token-based or purely reconstruction-based approaches.
State-Centric Representation: By predicting latent states rather than raw values and excluding intervention codes, the model aims to capture the evolving physiological state of the patient.
Unified Imputation and Prediction: The architecture enables masked-observation imputation and uncertainty estimation through a non-parametric query over the learned latent manifold, without requiring variable-specific decoders.
Comprehensive Evaluation Framework: The study evaluates the model not just on downstream task performance, but also on imputation quality, uncertainty calibration, cross-institutional transfer, robustness to data loss, and the interpretability of the latent space.

4. Results

Imputation and Uncertainty:

On 1.7 million masked observations across 828 variables, ATLAS-EHR achieved a standardized Mean Squared Error (MSE) of 0.322, outperforming Last Observation Carried Forward (LOCF) with training-mean fallback (MSE 0.717) and training-set mean (MSE 1.342).
The model demonstrated superior performance on low-persistence variables (e.g., blood gases, electrolytes) compared to LOCF, while LOCF remained competitive for high-persistence variables (e.g., infusion rates).
Uncertainty Calibration: The model's predicted uncertainty was well-calibrated; the nominal 95% prediction interval achieved 94.8% empirical coverage. Higher predicted uncertainty strongly correlated with higher actual imputation errors.

Downstream Clinical Tasks:

A single frozen backbone supported seven heterogeneous tasks (mortality, length of stay, readmission, ICD/HCUP/DRG classification).
Task-specific probes trained on the frozen representation outperformed the best logistic regression on all seven tasks and the best XGBoost on six of seven primary metrics.
The mean relative improvement over the best tabular model was 5.0%.

Cross-Institutional Transfer:

Direct zero-shot transfer of MIMIC-trained probes to eICU was suboptimal. However, local adaptation (retraining only the task-specific probe while keeping the backbone frozen) significantly improved performance, achieving the highest AUPRC for length-of-stay endpoints and approaching fully retrained XGBoost performance for mortality.
The backbone retained task-relevant information even after random deletion of up to 50% of events for several outcomes, though robustness varied by task.

Latent Space Analysis:

Clustering revealed 30 recurrent states. Some clusters corresponded to coherent physiological patterns (e.g., renal dysfunction, neurologic-metabolic disturbance, hepatic-muscle injury), while others reflected clinical observation processes (e.g., nursing assessments).
Cluster membership was associated with heterogeneous mortality risks, and the timing of state occupancy within an admission influenced outcome probabilities.

Disease-Specific Perturbation:

Perturbation analyses confirmed that the model's predictions for Chronic Kidney Disease (CKD) and Severe Sepsis were grounded in clinically concordant evidence.
CKD sensitivity was concentrated on renal function measurements (creatinine, urea), while Sepsis sensitivity was distributed across hemodynamic, metabolic, and inflammatory systems.
Neutralizing quantitative values significantly weakened predictions, indicating the model relies on actual physiological values rather than just observation patterns.

5. Significance and Claims

The paper claims that ATLAS-EHR demonstrates the feasibility of learning a broadly reusable, information-rich representation of longitudinal structured EHRs that captures the asynchronous, high-dimensional nature of patient data.

Contextual Learning: The results support the hypothesis that patient state is best represented as a joint function of concurrent and longitudinal context, rather than independent variable trajectories. The model learns that the value of contextual modeling is variable-specific.
Uncertainty as a Safety Mechanism: The ability to provide calibrated uncertainty estimates suggests the model can identify when the available record provides insufficient evidence, potentially preventing unjustifiably confident clinical inferences.
Separation of State and Intervention: By excluding explicit diagnostic and management codes, the model learns a representation of the patient state that is distinct from clinical decisions, potentially improving scalability and interpretability.
Practical Deployment: The findings suggest a realistic deployment architecture where a shared, pre-trained representation is reused across institutions, but task-specific prediction heads are locally adapted to account for institutional differences in patient populations and documentation practices.

The authors acknowledge limitations, including the use of a single institution for pretraining, the restriction to harmonizable variables for external validation, and the fact that latent states are descriptive rather than validated causal phenotypes. They posit that future work could extend this framework to model transitions between latent states and incorporate interventions as actions for treatment-effect estimation.