🤖 machine learning

KMGen: A Skill-based Approach for Synthetic Individual Patient Data Generation

KMGen is an open-source, end-to-end framework that fully automates the extraction of Kaplan-Meier curves from published plots and generates synthetic individual patient data, including adverse event trajectories, by combining agentic code generation with a mechanistic sampling approach that preserves marginal survival distributions and demographic correlations.

Original authors: Jalen Jiang, Chufan Gao, Ethan Rasmussen, Stephen Z. Xie, Jimeng Sun

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jalen Jiang, Chufan Gao, Ethan Rasmussen, Stephen Z. Xie, Jimeng Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of medical research, the most valuable resource is often the raw, individual record of a patient's journey through a clinical trial. These records contain the precise timing of when a treatment began, when side effects appeared, and exactly how long a patient lived. Scientists call this individual patient data, and it is the gold standard for understanding how well a new therapy works. However, for reasons of privacy and legal protection, these detailed records are almost never released to the public. Instead, researchers are usually left with only the final summary: a graph showing the percentage of patients who survived over time, and a list of how many people experienced specific side effects. This summary is useful, but it is like trying to understand a complex story by reading only the final chapter; the nuances of how the story unfolded are lost. Without the full data, it is difficult to build better computer models to predict outcomes or to rigorously test new safety monitoring systems.

A team of researchers has developed a new tool called KMGen to bridge this gap. Their work addresses a long-standing limitation in how medical data is shared. While previous methods could sometimes reconstruct the survival graph from a published picture, they could not create the missing half of the patient story: the timeline of side effects. KMGen is the first system designed to do both. It takes a published survival graph and a public trial registry entry, and it automatically generates a complete, synthetic set of individual patient records that look and behave like real data. This allows scientists to test their models and safety algorithms on data that mimics real trials, without ever needing access to the private, protected records of actual patients.

The process begins with a computer program that acts as a digital analyst. Its first job is to look at a published image of a survival curve—a line graph that shows how many patients are still alive at different points in time—and turn that picture back into numbers. This is a difficult task because these graphs often have overlapping lines, different colors, or distracting grid marks that can confuse a computer. The system uses a smart software agent to inspect the image, decide which technique is best for that specific picture, and then write code to extract the data with pixel-perfect precision. The researchers tested this extraction tool on thirty-two different types of graphs, ranging from clean, simple images to messy, distorted ones designed to trick the software. The tool succeeded in almost every case, producing a digital version of the curve that was nearly identical to the original, with errors so small they were barely noticeable.

Once the survival curve is digitized, the system moves to the second, more creative stage: generating the individual patient records. Instead of guessing or randomly inventing patient histories, the system follows a strict, logical recipe. First, a software agent reads the public trial registry to understand the basic facts of the study: the number of patients, their average age, their gender, and the types of side effects that were reported. It organizes this information into a structured plan. Then, a deterministic sampler—a computer program that follows fixed mathematical rules—uses this plan to create thousands of fake patients.

The way this generation works is carefully designed to be both realistic and reliable. The system creates different "archetypes" of patients based on their risk factors, such as age, gender, and overall health. It then assigns each fake patient a survival time that matches the curve extracted from the graph, ensuring that the overall pattern of survival is exactly preserved. For the side effects, the system schedules them based on the typical rhythm of a clinical trial, where patients are checked at regular intervals. It ensures that patients who are sicker or older are more likely to experience side effects, just as they would in a real trial, and that the timing of these events follows the natural patterns of treatment cycles. Crucially, every step of this process is transparent and based on explicit rules rather than hidden guesses, meaning scientists can trace exactly how a specific patient record was created.

The researchers tested this entire pipeline on three real-world cancer trials involving hundreds of patients. They used the system to generate synthetic data and then compared it to the actual, hidden data from those trials. The results were strikingly close. The synthetic survival curves matched the real ones with a high degree of accuracy, and the distribution of side effects in the fake data closely mirrored the real data. For example, the system correctly identified the most common side effects in three out of four cases and reproduced the timing of when those side effects occurred with an average error of less than one month. Even the demographic details, such as the mix of ages and genders, were nearly indistinguishable from the real trials.

This work does not claim to replace real clinical trials or to generate data that can be used to make regulatory decisions about new drugs. The synthetic records are not real people, and they do not capture the full complexity of human biology. However, the study demonstrates that it is possible to create high-quality, realistic data for the specific purpose of testing and improving the tools scientists use to analyze health. By providing a way to generate individual patient data from public summaries, KMGen offers a new resource for researchers to validate their methods and explore safety questions in a setting where real data is unavailable. The system is now available as open-source software, allowing the broader scientific community to use and refine this approach for future studies.

Technical Summary: KMGen – A Skill-based Approach for Synthetic Individual Patient Data Generation

1. Problem Statement

Individual Patient Data (IPD) from clinical trials is the gold standard for survival modeling, meta-analysis, and safety research. However, access to IPD is frequently restricted due to privacy regulations, proprietary interests, and data governance. Consequently, secondary analyses often rely on aggregated summary statistics (e.g., hazard ratios, median survival), which lack the granular event timing and censoring patterns required for rigorous machine learning model development.

Existing methodological advances have addressed only half of this gap:

Reconstruction: Methods like IPDfromKM and KM-GPT attempt to reconstruct IPD from published Kaplan–Meier (KM) curves. While effective, these typically require manual digitization or human-in-the-loop correction and offer no mechanism for generating the adverse event (AE) streams that constitute the other half of a patient record.
Generation: Purely generative approaches (e.g., end-to-end LLM samplers) often hallucinate clinically implausible event sequences or suffer from mode collapse on the long tail of rare events.

There is a need for a unified framework that can both automate the extraction of survival data from published plots and generate high-fidelity synthetic IPD, including treatment-emergent adverse events, without requiring access to the original restricted datasets.

2. Methodology

KMGen is a dual-purpose framework that decouples interpretive tasks (assigned to a multimodal Large Language Model agent) from computational tasks (assigned to deterministic code). This separation ensures that approximation is confined to strategy selection, while exactness and reproducibility are maintained in data processing.

2.1. KM Plot Extraction Pipeline

The extraction component is a fully automated, agentic pipeline designed to recover patient-level survival data from published KM plots.

Agent Role: An LLM analyst inspects the input image, identifies challenges (e.g., overlapping curves, color schemes, reference lines), and selects applicable techniques from a "toolbox."
Deterministic Execution: The agent generates Python code to execute pixel-precise extraction. Techniques include:
- Bounding-box detection via tick marks (verified by linear regression).
- Color/Grayscale separation using HSL hue matching or intensity thresholds.
- Reference-line scrubbing to remove non-curve gridlines.
- Curve tracing with continuity enforcement and dashed-line gap bridging.
Self-Verification Loop: The agent performs five independent checks (e.g., $S(0) \approx 1.0$ , monotonicity, median annotation alignment) and re-extracts until all anchors clear.
Output: A JSON document containing reconstructed coordinates $(t_i, S_i)$ for each treatment arm.

2.2. Synthetic IPD Generation Methodology

The generation component converts public trial registry records (e.g., ClinicalTrials.gov) and extracted KM data into synthetic IPD cohorts through three stages:

Configuration Extraction (LLM Agent):
- The agent parses trial records into a structured, MedDRA-compliant JSON configuration.
- Strict Source Priority: Data is extracted in a hierarchy: (1) ClinicalTrials.gov structured results (primary), (2) Reconstructed KM CSVs, (3) Primary publications, and (4) Statistical Analysis Plans (SAP).
- The output includes per-arm demographics, treatment timing (cycle length, induction/maintenance split), and adverse event incidence probabilities.
Patient Archetype Extraction:
- The configuration is expanded into discrete "archetypes" based on the cross-product of four risk dimensions: Age, Sex, ECOG performance status, and Comorbidity (Charlson Comorbidity Index).
- Each archetype is assigned risk multipliers ( $m_{OS}$ for survival, $m_{AE}$ for adverse events) derived from clinical priors (literature reviews on geriatric oncology and toxicity).
Statistical Sampling (Deterministic Sampler):
- Survival Assignment: Uses bootstrap rank-correlation coupling. The sampler draws bootstrap samples from the empirical KM curve and assigns them to patients based on their "frailty" scores (derived from archetype risk multipliers). This preserves the marginal survival distribution exactly while inducing within-cohort risk stratification.
- Adverse Event Scheduling: Events are generated on a cycle-based time grid (matching trial visit schedules).
  - Phase Split: Events are scheduled with an induction/maintenance split ( $\alpha$ vs. $1-\alpha$ ), reflecting higher event rates during early, high-dose cycles.
  - Shared Frailty: A patient-level propensity modifier ( $\eta_i$ ) is applied multiplicatively across all organ systems to induce positive within-patient correlation among AEs, mimicking the "toxicity burden" of real patients.

3. Key Contributions

Unified Framework: KMGen is the first end-to-end system combining adaptive hybrid extraction with deterministic synthetic generation for comprehensive IPD accessibility.
Automated Extraction Benchmark: Evaluation on a 32-plot synthetic benchmark (spanning clean, edge-case, and adversarial conditions) achieved a mean Integrated Absolute Error (IAE) of 0.0151, competitive with human-guided tools but fully automated.
High-Fidelity Generation: The framework generates synthetic IPD that includes both survival and adverse event trajectories. It achieves a mean integrated KM absolute difference ( $\Delta_{KM}$ ) of $\le 0.051$ and recovers $\ge 71\%$ of the top-15 adverse events by exact MedDRA term across three held-out oncology trials.
Interpretability and Open Source: The pipeline is interpretable end-to-end, with clinical priors as explicit parameters rather than latent weights. The code is released as open source.

4. Results

The framework was evaluated on three oncology trials (NCT03041311, NCT02499770, NCT00844649) with cohort sizes ranging from 37 to 430 patients, using a single fixed parameter set.

Extraction Performance:
- Synthetic Benchmark: Mean IAE = 0.0151. Performance degraded slightly on combination stress tests (Mean IAE = 0.0283) but remained robust.
- Real Trials: Mean IAE against sponsor-delivered IPD was 0.0262.
Generation Performance (Stage B):
- Survival: Mean $\Delta_{KM} \le 0.051$ . Median OS relative errors ranged from 7.0% to 19.4%. The Mann–Whitney U test failed to reject the null hypothesis for smaller trials but rejected for the largest trial (NCT00844649) due to a 1.3-month median shift driven by statistical power, not structural mismatch.
- Demographics: Sex JSD $\le 0.006$ and ECOG JSD $\le 0.013$ (where support matched).
- Adverse Events: Top-15 AE overlap was $\ge 71\%$ . Cosine similarity for AE frequency reached 0.940 on the largest trial.
- Timing: Mean onset error was $\le 0.71$ months. Synthetic onsets exhibited cycle-length striping (an artifact of the cycle-based sampling model), which is consistent with the visit-driven nature of real trial data.
- Burden: The default parameter set slightly undershot real AE burden; tuning the propensity variance ( $\sigma_\eta$ ) from 0.4 to 0.8 narrowed this gap significantly without affecting survival metrics.

5. Significance and Claims

The paper claims that KMGen addresses a critical bottleneck in clinical research by enabling the creation of synthetic IPD for methodological development in settings where real data is restricted.

Interpretability: By decoupling LLM-driven archetype extraction from deterministic statistical sampling, the framework ensures that clinical priors (e.g., induction/maintenance splits, risk multipliers) are explicit and citable, making the synthetic data auditable.
Marginal Preservation: The use of bootstrap rank-correlation coupling guarantees that the marginal survival distribution is preserved exactly by construction, independent of noise parameters.
Scope and Limitations: The authors explicitly state that the framework is currently validated on oncology trials (three distinct drug classes) and is intended for methodological applications (e.g., stress-testing survival models, validating safety algorithms). They do not claim the synthetic data is a substitute for randomized controlled trials or suitable for regulatory claims. The framework models structural correlations (e.g., shorter survival limits event opportunity) but does not simulate causal pathways where toxicity induces mortality.

The work demonstrates that a skill-based approach, where LLMs handle strategy and code generation while deterministic algorithms handle precision, can effectively bridge the gap between published summary statistics and the granular data required for advanced machine learning in healthcare.