💬 NLP

Reconstructing Sepsis Trajectories from Clinical Case Reports using LLMs: the Textual Time Series Corpus for Sepsis

This paper presents a pipeline using large language models to extract and time-localize clinical findings from medical case reports, resulting in an open-access textual time series corpus for Sepsis-3 that demonstrates high accuracy in event recovery and temporal ordering compared to physician annotations.

Original authors: Shahriar Noroozizadeh, Jeremy C. Weiss

Published 2026-05-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Shahriar Noroozizadeh, Jeremy C. Weiss

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a patient's journey through a hospital as a complex movie. The structured data (like vital signs and lab results) are the special effects and the soundtrack—easy to read by computers, but they only show you the scenes in order, often missing the plot details. The clinical case reports (the doctors' written stories) are the full movie script. They contain the rich details of what happened, why it happened, and the emotional context. However, there's a catch: the script is usually written after the movie is finished. It's a retrospective summary, and the "timestamps" in the story are often fuzzy or out of order (e.g., the doctor writes, "The patient died," at the top, even though that happened at the very end).

This paper, "Reconstructing Sepsis Trajectories from Clinical Case Reports using LLMs," is about teaching a super-smart AI (a Large Language Model or LLM) to watch that finished movie script and rewrite it into a perfectly timed, minute-by-minute timeline.

Here is the breakdown of their work using simple analogies:

1. The Problem: The "Rewind" Button is Broken

Sepsis is a life-threatening reaction to an infection. To understand how to treat it better, doctors need to know exactly when things went wrong.

The Issue: Hospital computers have data streams, but they are often incomplete (like a movie with missing frames). The written case reports have all the frames, but they are written after the fact. A doctor might write, "The patient had a fever yesterday," but the computer doesn't know if that was 2 hours ago or 2 days ago.
The Goal: The authors wanted to take these written stories and use AI to extract a "Textual Time Series." Think of this as turning a paragraph of text into a precise calendar of events, where every symptom and treatment is pinned to a specific hour relative to when the patient arrived.

2. The Solution: The AI "Time-Traveler"

The researchers built a pipeline using powerful AI models (like GPT-5 and Llama 3.3) to act as a forensic time-traveler.

The Task: They fed the AI thousands of medical case reports from the internet (PubMed).
The Magic: The AI was instructed to read the story, find every clinical event (like "fever," "diagnosis," "surgery"), and assign it a timestamp.
- Example: If the text says, "Three days before admission, the patient had a fever," the AI calculates: "Admission is Hour 0. Three days ago is Hour -72."
- It also splits complex sentences. If a text says "metastases in liver and pancreas," the AI treats this as two separate events: one for the liver, one for the pancreas, both at the same time.

3. The "T2S2" Corpus: A New Library of Timelines

The result of this work is a new, open-access library called T2S2 (Textual Time Series for Sepsis).

Size: It contains 2,139 detailed case reports.
Content: Each report has been transformed from a block of text into a list of 3,000+ specific events with precise times.
Why it matters: This is the first time such a large collection of "time-stamped" sepsis stories has been created from free-text reports. It's like turning a library of novels into a database of precise historical timelines.

4. Did the AI Get it Right? (The Quality Control)

The authors didn't just trust the AI; they put it through a rigorous "exam" to see if it was hallucinating (making things up) or getting the timing wrong.

The Human Exam: They had real doctors annotate 40 of these stories manually to create a "Gold Standard" answer key.
The Results:
- Event Recovery: The best AI (GPT-5) found 93% of the events the doctors found. It was very good at spotting the "what."
- Timing Accuracy: The AI was excellent at ordering events correctly (96.5% agreement on the sequence).
- The "Hallucination" Check: They checked if the AI made up events that weren't in the text. In a sample of 1,500 events, zero were found to be unsupported hallucinations. The AI was sticking to the script.
The Weakness: The AI sometimes struggled with very long time gaps. If a story said, "He died 6 months later," the AI was good at the order but sometimes guessed the exact hour wrong. It's like knowing the movie ends in the third act, but guessing the exact minute the credits roll.

5. What This Is (and What It Isn't)

The authors are very clear about the boundaries of their work:

It IS: A powerful research tool. It allows scientists to study the narrative of sepsis with fine-grained timing. It helps build better models to predict risk or understand how the disease progresses over time.
It IS NOT: A tool to be used in a hospital right now to make real-time decisions. The data comes from published stories (which are edited and polished), not the messy, real-time notes doctors write while a patient is currently in the ICU.
The Limitation: Because these are published stories, they might focus on rare or interesting cases (publication bias). They aren't a perfect representation of every sepsis patient in the world.

Summary Analogy

Imagine you have a pile of old, handwritten diaries from a war. They tell you everything that happened, but the dates are messy, and some pages are torn.

Structured Data is a spreadsheet of troop movements that is missing half the columns.
This Paper is hiring a team of AI historians to read every diary, figure out the exact date and time of every battle, and organize them into a single, perfect timeline.
The Result: A massive, searchable timeline of the war that researchers can use to understand the flow of battle, even though the AI isn't a general commanding troops on the front lines today.

The paper proves that AI can effectively turn messy medical stories into precise, time-ordered data, opening the door for better research into how diseases like sepsis unfold over time.

Technical Summary: Reconstructing Sepsis Trajectories from Clinical Case Reports using LLMs

Problem Statement
Clinical case reports and discharge summaries offer the most complete and accurate summarization of patient encounters; however, they are finalized only after the encounter concludes, resulting in a coarse temporal resolution compared to the granular events they describe. Conversely, structured data streams (e.g., vital signs, labs) are available in real-time but often suffer from significant incompleteness, making it difficult to render or confirm diagnoses like sepsis. While structured data is the primary resource for current early-warning systems and phenotyping studies, its missingness leads to suboptimal modeling. There is a critical need for a dataset that combines the narrative completeness of clinical text with fine-grained temporal granularity to enable better temporal analyses, causal reasoning, and forecasting of sepsis trajectories. Existing efforts have largely focused on temporal relations between clinical concepts rather than event timing, or have utilized small sample sizes and excerpted summaries.

Methodology
The authors introduce a pipeline to construct the Textual Time Series for Sepsis-3 (T2S2) corpus, comprising 2,139 open-access case reports from the PubMed Open Access (PMOA) subset. The methodology involves a four-stage process:

Case Report Detection & Phenotyping: The pipeline filters PMOA reports using regex and Large Language Models (LLMs) to identify reports meeting Sepsis-3 criteria (suspected/confirmed infection plus qSOFA abnormalities).
Clinical Finding Extraction: Using a structured prompt with few-shot examples, six LLMs (including GPT-5, DeepSeek-R1, and various Llama 3.x models) extract clinical findings. The extraction rules require splitting conjunctions (e.g., "liver and pancreas metastases" becomes two events) and preserving specific text spans.
Temporal Assignment: The models assign timestamps relative to admission ( $t=0$ ) in hours. Pre-admission events receive negative timestamps. The pipeline handles duration events by assigning the start time and infers times for events lacking explicit temporal markers using contextual reasoning.
Evaluation Framework: The system is validated against a gold-standard set of 40 clinician-annotated reports (sepsis-40) and a bronze-standard proxy set (sepsis-100). Evaluation metrics include:
- Event Match Rate: Recursive best-matching using cosine distance of PubMedBERT embeddings (threshold $\theta=0.1$ ).
- Temporal Ordering: Concordance index (c-index) for event pairs.
- Time Accuracy: Median Absolute Error (MAE) and Area Under the Log-Time Curve (AULTC), which integrates discrepancies across time scales to prevent long-horizon errors from dominating the metric.

Key Contributions

T2S2 Corpus: The first open-access textual time series corpus for Sepsis-3, containing 2,139 case reports with extracted event-timestamp pairs.
Annotation Pipeline: A novel LLM-as-annotator approach specifically designed to extract time-associated clinical findings from free text, distinct from prior work focusing on i2b2-style concept relations.
Gold-Standard Annotations: 3,229 manually annotated reference events across 40 PMOA case reports, providing a rigorous benchmark for evaluation.
Evaluation Criteria: Adapted metrics for textual time series where matched findings need not be identical strings, utilizing semantic embedding distances and log-time scaling for error measurement.
Multi-Layer Validation: A comprehensive framework including bootstrap uncertainty quantification, inter-annotator agreement (clinician vs. clinician), hallucination auditing, self-consistency analysis, and full-corpus cross-model agreement.

Results

Extraction Performance: On the clinician-annotated sepsis-40 set, GPT-5 demonstrated the strongest overall performance with an event match rate of 0.93, a concordance of 0.965, and a median absolute error of 4 hours. DeepSeek-R1 and Llama 3.3 70B provided competitive open-model baselines, with DeepSeek-R1 achieving a 0.79 match rate and Llama 3.3 achieving 0.76.
Temporal Ordering: Large models exhibited strong temporal ordering capabilities. GPT-5 and Llama 3.3 achieved concordance scores of 0.965 and 0.908, respectively, significantly outperforming smaller models (e.g., Llama 3.1 8B at 0.856) and the BERT time-regression baseline.
Validation Findings:
- Hallucination Audit: In a manual audit of 1,517 events from DeepSeek-R1 and Llama 3.3, zero unsupported hallucinated events were identified, though timestamp errors remained possible even for grounded events.
- Self-Consistency: Multi-pass generation showed high stability across random seeds, indicating the extraction process is not driven by stochastic noise.
- Human Variability: Clinician-to-clinician agreement on the same reports was imperfect (match rate 0.681), suggesting that the gap between model performance and human performance is narrowing and that model errors are often comparable to human annotation variability.
- Limitations: Performance degraded on the i2m4 dataset (legacy i2b2 concepts) due to ontology mismatches (fragmented concepts vs. compositional findings) and under-identification biases. Additionally, long-horizon events (months/years) showed higher timestamp discrepancies.

Significance and Claims
The paper characterizes the ability of LLMs to time-localize clinical findings, demonstrating that foundation models with instruction tuning can effectively reconstruct retrospective clinical trajectories from text. The authors claim that T2S2 serves as a high-fidelity retrospective research resource for downstream tasks such as risk forecasting, causal reasoning, and disease trajectory characterization.

The authors explicitly state that T2S2 is not a deployment-ready clinical extraction system. They highlight limitations including publication bias inherent in case reports, the lack of exhaustive clinician validation for the full corpus, and the difficulty of translating this pipeline to prospective, real-time clinical settings where data is noisier and continuously evolving. The work suggests that while LLMs can recover temporal structures with high fidelity, the primary failure mode is often the assignment of incorrect times to real findings rather than the fabrication of events. Future utility is envisioned in integrating these narrative-derived timelines with structured physiological data to build comprehensive multimodal representations of patient trajectories.