📊 statistics

EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins

This paper introduces EHR-MPC, a framework that leverages generative patient digital twins to enable inference-time model predictive control for sepsis treatment, thereby overcoming the adaptability limitations of fixed reinforcement learning policies by optimizing decisions through real-time simulation over learned clinical dynamics.

Original authors: Joshua Pickard, Wei Qi, Na Li, Ann Woolley, Lisa Cosimi, Roy Kishony, Deborah Hung

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joshua Pickard, Wei Qi, Na Li, Ann Woolley, Lisa Cosimi, Roy Kishony, Deborah Hung

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a robot how to treat a patient with sepsis, a dangerous infection that affects over 48 million people and causes 11 million deaths every year. For decades, doctors have been arguing about the best way to handle it, especially regarding a specific medicine: corticosteroids. Some studies say they help, others say they might hurt, and the debate has been raging since the 1970s.

Most computer scientists have tried to solve this by teaching a robot a "fixed rulebook." They feed the robot thousands of past patient records and say, "Learn the pattern of what worked best." This is like training a dog to sit by giving it a treat every time it sits; eventually, the dog learns to sit, but if you suddenly change the rules and say, "Now, stand up!" the dog is confused. It has to start over. In the medical world, this means if a doctor wants to change the goal—say, from "keep the patient alive for 28 days" to "keep their organs working better"—the old robot has to be completely retrained. It's rigid, slow, and often gets it wrong when the situation changes.

The New Idea: The "Digital Twin" Simulator

The authors of this paper, Joshua Pickard and his team, propose a different approach called EHR-MPC. Instead of teaching the robot a fixed rulebook, they teach it how to be a Digital Twin.

Think of a Digital Twin like a hyper-realistic video game character that perfectly mimics a real patient. You feed the game the patient's current health data (their vitals, lab results, and history), and the game learns exactly how that specific person's body reacts to different things.

Here is the magic trick: The Digital Twin doesn't just guess what happens next; it can run simulations. Before a doctor gives a real drug, the Digital Twin runs a "what-if" scenario in its head.

"What if we give 50mg of steroids?" -> Simulates the next 24 hours.
"What if we give 100mg?" -> Simulates a different 24 hours.
"What if we give nothing?" -> Simulates another path.

The computer then looks at all these simulated futures and picks the one that leads to the best outcome based on the doctor's current goal. If the doctor wants to lower the risk of death, the computer picks the path that saves the most lives. If the doctor wants to improve organ function, it picks that path instead.

The "Inference-Time" Superpower

The coolest part is that the computer doesn't need to be retrained to change its mind. Because the "brain" (the Digital Twin) knows how the body works, and the "goal" (what the doctor wants) is separate, the doctor can just say, "Hey, let's try to minimize organ failure today," and the computer instantly recalculates the best plan. This is called inference-time control. It's like having a GPS that doesn't just give you one route, but instantly recalculates the best path the moment you say, "I want to avoid traffic" or "I want to see the ocean."

What the Numbers Say

The team tested this on a massive dataset from 8 hospitals in the Mass General Brigham system, looking at 36,930 ICU patients collected between 2022 and 2026. They didn't just guess; they ran two different types of tests:

The "Backwards Look" (Off-Policy): They checked how their new system would have done on past patient records compared to what the doctors actually did and compared to other AI models (called Q-networks). The results showed that EHR-MPC performed just as well as the other AI models and better than the doctors' actual past choices.
The "Forward Look" (Simulation): They let the Digital Twin run new simulations. Here, EHR-MPC consistently beat the other AI models. It improved patient outcomes (measured by a score called SOFA, which tracks organ failure) and reduced the risk of death more effectively than the standard AI approaches.

What It Does NOT Do (and What It Rules Out)

It's important to know what this paper doesn't claim.

It's not a magic cure-all: The authors are very clear that this is a simulation and a proposal. They haven't tested this on real patients in a live hospital yet. The results are based on computer models and historical data.
It doesn't prove steroids are the answer: While the system suggested using more steroids when the goal was to reduce death (which matches some recent clinical trial evidence), the paper doesn't say steroids are definitely the right choice for everyone. It just says the system found that path to be optimal in its simulations.
It's not perfect: The authors admit their "Digital Twin" isn't a perfect replica of reality. It's trained on data, and if the data is messy or the model is slightly off, the simulation might be wrong. They explicitly state that validating these models in the real world is still a huge, open challenge.

The Bottom Line

This paper suggests a new way to think about medical AI. Instead of building a robot that memorizes a single "best" way to treat a disease, we should build a robot that understands how the human body works and can play out different scenarios in real-time to find the best solution for the specific patient in front of it.

The authors suggest that by separating the "learning how the body works" part from the "deciding what to do" part, we can create medical tools that are flexible, adaptable, and ready to change their strategy the moment a doctor's goals change. It's a promising step toward smarter, more personalized care, but as the authors remind us, we are still in the "simulation" phase, and the real test is yet to come.

Technical Summary: EHR-MPC: Inference-Time Control for Sepsis Treatment with Generative Patient Digital Twins

Problem Statement

Sepsis remains a leading cause of global mortality, yet optimal treatment policies—particularly regarding corticosteroid administration—remain contested. Existing reinforcement learning (RL) approaches for sepsis management typically learn fixed policies by entangling patient dynamics and clinical objectives during the training phase. This coupling limits adaptability; changing clinical goals (e.g., prioritizing organ preservation over immediate hemodynamic stability) or deployment settings often necessitates retraining the entire model. Furthermore, learned RL policies are often difficult to interpret, validate, or adapt to individualized clinical contexts, leading to a disconnect between retrospective performance and clinical practice. The core challenge is to develop a decision-making framework that can decouple the learning of patient physiology from the optimization of treatment strategies, allowing for inference-time adaptation to evolving objectives without retraining.

Methodology: EHR-MPC Framework

The authors propose EHR-MPC (Electronic Health Record Model Predictive Control), a two-stage framework that separates the learning of patient dynamics from the optimization of treatment decisions.

Generative Patient Digital Twin (Learning Dynamics):
- Data Representation: The system ingests tokenized Electronic Health Record (EHR) data from a multi-center ICU cohort. Clinical events (vitals, labs, interventions, demographics) are discretized into a sequence of tokens $x_{1:k}$ .
- Model Architecture: A generative sequence model (implemented as a transformer) is trained to predict the next token in the sequence, $p_\theta(x_{t+1} | x_{\le t})$ . This model serves as a "digital twin," learning the conditional distribution of patient trajectories under observed history.
- Counterfactual Simulation: To evaluate interventions, the model performs "token-forced rollouts." Candidate treatment sequences (e.g., specific dosages of corticosteroids) are injected as action tokens into the input. The model then autoregressively generates future patient states, simulating counterfactual trajectories under these specific interventions.
Model Predictive Control (Optimizing Decisions):
- Inference-Time Planning: Unlike RL, which outputs a fixed policy, EHR-MPC performs planning at inference time. At each decision step, the controller simulates multiple candidate intervention sequences over a finite horizon $h$ using the digital twin.
- Objective Specification: A reward function $R$ is defined to encode clinical objectives (e.g., minimizing SOFA scores or mortality risk). Crucially, this reward function is not embedded in the model during training. It is specified at inference time, allowing the same digital twin to be optimized for different, potentially competing, clinical goals without retraining.
- Action Selection: The controller selects the action sequence that maximizes the expected reward over the simulated trajectories. Only the first action is executed, after which the system observes new data and repeats the planning process (receding-horizon control).

Key Contributions

Decoupling Dynamics and Policy: The paper introduces a paradigm shift where reusable models of patient dynamics replace fixed policies as the primary object of learning. Treatment decisions are derived via inference-time control over these models.
Inference-Time Adaptability: The framework enables the specification of new clinical objectives and the adaptation to changing standards of care without retraining the underlying generative model.
Interpretable Counterfactuals: By explicitly simulating patient trajectories under candidate interventions, the system provides clinicians with mechanistic forecasts of patient evolution rather than opaque scalar reward estimates.
Generalizable Framework: The approach extends the concept of foundation models from prediction to planning, utilizing learned patient simulators for sequential decision-making.

Experimental Results

The framework was evaluated on a cohort of 36,930 ICU patients across 8 hospitals in the Mass General Brigham system.

Digital Twin Fidelity: The generative model demonstrated strong performance in next-token prediction and outcome forecasting (e.g., mortality, length of stay). It exhibited dose-response sensitivity, showing monotonic decreases in predicted mortality risk with increased corticosteroid tokens in counterfactual simulations.
Off-Policy Evaluation: Using Weighted Importance Sampling (WIS), EHR-MPC achieved performance comparable to Q-network baselines and significantly outperformed observed clinician policies in minimizing SOFA scores and mortality risk.
Simulation-Based (On-Policy) Evaluation: When evaluated via rollouts on the digital twin simulator, EHR-MPC consistently outperformed Q-network baselines in both SOFA improvement and mortality reduction.
Robustness: The performance advantage of EHR-MPC persisted even when the planning model was decoupled from the evaluation model (using different training checkpoints), suggesting the results were not merely due to exploiting a shared dynamics model.
Policy Divergence: EHR-MPC showed greater divergence from observed clinical policies compared to Q-networks. Notably, when optimized for mortality reduction, EHR-MPC recommended increased corticosteroid use, aligning more closely with recent randomized clinical trial evidence (e.g., Annane et al., 2025) than the Q-network, which tended to reduce steroid use.

Significance and Claims

The paper claims that EHR-MPC establishes a general framework for decision-making with generative clinical models. Its primary significance lies in shifting the optimization burden from training-time policy fitting to inference-time planning. This allows for:

Flexibility: Adapting to new clinical objectives without retraining.
Reusability: A single dynamics model serving multiple, potentially conflicting, clinical goals.
Safety and Interpretability: Enabling the explicit evaluation of counterfactual trajectories and the incorporation of safety constraints during the planning phase.

The authors acknowledge limitations, particularly regarding the fidelity of the learned digital twin and the challenges of off-policy evaluation in healthcare. They posit that while the current results are promising, the ultimate value of the framework depends on the continued improvement of digital twin fidelity and the development of principled evaluation methods for simulation-based policies. The work suggests a path toward clinical machine learning systems that move beyond static prediction to dynamic, objective-aware planning.