Temporally Phenotyping GLP-1RA Case Reports with Large Language Models: A Textual Time Series Corpus and Risk Modeling

Imagine you are trying to understand the life story of a patient with Type 2 diabetes who is taking a popular weight-loss and blood-sugar medication called a GLP-1RA.

In a perfect world, a doctor's notes would look like a spreadsheet: a neat list of dates, events, and outcomes. But in reality, medical case reports are written like novels. They are full of sentences like, "Three days after starting the drug, the patient felt nauseous," or "Two weeks later, they were admitted to the hospital."

For computers, this is like trying to build a train schedule using a storybook. The information is there, but it's buried in paragraphs, making it hard to see the sequence of events or predict what might happen next.

The Mission: Turning Stories into Timelines

This paper is about a team of researchers who decided to teach Artificial Intelligence (AI) to read these medical "novels" and turn them into structured timelines.

Think of it like this:

The Problem: You have 136 messy, handwritten diaries of patients. You want to know exactly when they took their medicine, when they got sick, and when they got better.
The Solution: They used a super-smart AI (a Large Language Model, or LLM) to read these diaries and extract a "timeline" for each patient. The AI acts like a detective, spotting clues in the text (like "two days later") and organizing them into a neat list of events with timestamps.

The "Gold Standard" Test

To make sure the AI wasn't just guessing, the researchers hired two human medical experts to create the "correct" timelines manually. This is the Gold Standard.

They then asked the AI to do the same job and compared its work to the humans.

The Result: The best AI model (called GPT-5) was incredibly good. It found about 87% of the important events and got the order of those events right about 84% of the time.
The Analogy: Imagine a student taking a history test. The human experts wrote the answer key. The AI student took the test and got an "A" on the timeline section, proving it can read complex medical stories and understand the sequence of time almost as well as a human doctor.

Why Does This Matter? (The "So What?")

Once they had these clean, organized timelines, they used them to ask a big question: "Does taking this diabetes drug change a patient's risk of getting sick in the lungs, heart, or kidneys?"

Usually, to answer this, you need massive databases of hospital records. But those records often miss the "story" of how a patient felt or when they started a new drug. By using these AI-extracted timelines, the researchers could look at the "story" of the patients.

What they found:

Lungs: Patients who took the GLP-1 drug seemed to have a much lower risk of developing respiratory (lung) problems compared to those who didn't. This matches what other studies have suggested.
Heart & Kidneys: The results were mixed or unclear. The data wasn't strong enough to say for sure if the drug helped or hurt the heart or kidneys in this specific group of stories.

The Big Picture

This paper is a proof-of-concept. It shows that we can use AI to turn messy, unstructured medical stories into clean, usable data.

Before: Medical stories were like a pile of loose puzzle pieces. You could see the picture, but you couldn't build it.
After: The AI acts as the puzzle sorter, organizing the pieces by time so we can finally see the full picture of how these drugs affect patients over the long term.

The Catch

The researchers are honest about the limitations:

Selection Bias: Case reports are like "highlight reels" of medicine. They often feature rare or dramatic cases, not the average patient. So, the results might not apply to everyone.
AI Mistakes: While the AI is great, it's not perfect. It might occasionally misread a date or miss a subtle detail.
Timing: Sometimes the AI records when a symptom was written down, not necessarily when it first happened in real life.

Conclusion

In short, this paper is a bridge. It connects the rich, detailed world of human-written medical stories with the rigid, data-driven world of computer modeling. By teaching AI to understand the "when" and "what" of medical narratives, we can get better at predicting risks and understanding how life-saving drugs really work over time.

1. Problem Statement

Type 2 Diabetes (T2D) and the long-term effects of Glucagon-like peptide-1 receptor agonists (GLP-1RAs) are critical areas of study. However, existing data sources present significant limitations for longitudinal modeling:

Structured Data Limitations: Electronic Health Records (EHRs) and claims data provide timestamps but often lack the narrative context required to reconstruct medication-centered disease dynamics (e.g., indications, tolerability, specific adverse events). They are also often biased toward short-term inpatient trajectories.
Unstructured Data Challenges: Clinical case reports contain rich, detailed narratives of long-term treatment courses. However, event timing is expressed in relative, free-text terms (e.g., "two weeks after starting semaglutide") rather than absolute timestamps.
The Gap: There is a scarcity of large, richly annotated corpora that align narrative clinical events with relative time points, hindering the ability to perform time-to-event analyses and risk forecasting for GLP-1RA therapies.

2. Methodology

The authors developed a pipeline to convert unstructured case reports into structured, time-stamped clinical timelines using Large Language Models (LLMs).

A. Data Extraction & Cohort Selection

Source: PubMed Open Access (PMOA) repository (1.48M manuscripts).
Filtering:
1. Identified candidate case reports using regex patterns (e.g., "case report," "patient").
2. Filtered for single-patient reports using an LLM.
3. Selected the GLP-1RA cohort (136 reports) via keyword matching against a curated lexicon of class-level terms (e.g., "GLP-1 receptor agonist") and specific drug names (e.g., semaglutide, liraglutide).

B. Textual Time Series (TTS) Annotation

LLM-as-Annotator: Multiple LLMs (DeepSeek R1, Llama3.3, GPT5, O1, O3, O4mini) were prompted to extract TTS from the 136 reports.
Definition of TTS: A sequence $S = \{(e_1, t_1), ..., (e_n, t_n)\}$ $S = {(e_{1}, t_{1}), ..., (e_{n}, t_{n})}$ where $e_i$ $e_{i}$ is a clinical finding and $t_i$ $t_{i}$ is the time in hours relative to a reference point ( $t=0$ $t = 0$ ).
- Reference Point ( $t=0$ ): Hospital admission (if stated) or the earliest documented clinical encounter.
- Timestamping: Events before $t=0$ are negative; events after are positive. Relative expressions (e.g., "3-day history") are normalized to hour offsets.
Event Scope: Includes symptoms, diagnoses, procedures, treatments, outcomes, and pertinent negatives. Demographics are anchored at $t=0$ .

C. Evaluation Framework

Gold Standard: Two clinically trained experts manually annotated the same 136 reports to create a reference timeline.
Metrics:
- Event Matching: Recursive best-match procedure using PubMedBERT cosine similarity (threshold $\le 0.1$ ) to align predicted events with reference events.
- Temporal Concordance: C-index (probability that event pairs maintain the same temporal order).
- Timestamp Accuracy: Area Under the Log-Time CDF (AULTC), which measures how concentrated timestamp errors are near zero, handling errors spanning orders of magnitude.

D. Downstream Analysis

Survival Modeling: A Cox proportional hazards model was used to analyze time-to-onset for kidney, cardiovascular, and respiratory outcomes.
Cohorts:
- Treatment: GLP-1RA users with diabetes, initiating treatment within 72 hours of $t=0$ (to reduce immortal time bias).
- Control: Diabetes cases without GLP-1RA exposure or with delayed initiation.
Outcome Ascertainment: Keyword-based matching against the extracted TTS events.

3. Key Contributions

Novel Corpus: Creation of a GLP-1RA textual time-series corpus (136 reports) converting unstructured narratives into structured, time-stamped timelines.
Benchmarking: A rigorous evaluation of multiple LLMs against a dual-expert gold standard, establishing performance baselines for temporal extraction in clinical narratives.
Methodological Pipeline: A scalable framework for "phenotyping" case reports that can be adapted to other disease areas beyond GLP-1RAs.
Clinical Utility Demonstration: Successful application of the extracted timelines to perform time-to-event survival analysis, revealing outcome-specific associations with GLP-1RA exposure.
Open Science: Release of the LLM-extracted timelines, expert annotations, and code as a pilot benchmark corpus.

4. Key Results

A. Cohort Characteristics

Demographics: Median age 49; nearly balanced sex distribution (49% male, 49% female). Ethnicity was rarely reported.
Clinical Profile: Dominated by cardiometabolic comorbidities (Hypertension, Obesity, T2D). High prevalence of kidney disease (45.7%) and digestive disorders, reflecting case-report selection bias toward complex/severe cases.
Temporal Density: Timelines vary widely (50–200+ events). Mean duration is 11 years, with 24% of events having negative timestamps (pre-admission history).

B. LLM Performance Evaluation

Best Performer: GPT5 achieved the best trade-off, with high event coverage (Match Rate: 0.871) and reliable temporal sequencing (Concordance: 0.843).
Comparison: GPT5 outperformed other models (O3, O4mini, Llama variants) and even showed better temporal ordering than the second human annotator (A2) when compared against the first (A1).
Human Baseline: Inter-annotator agreement was 0.811 (match), 0.798 (concordance), and 0.702 (AULTC), providing a ceiling for automated performance.

C. Survival Modeling Findings

Respiratory Outcomes: GLP-1RA exposure was associated with a significantly lower risk of respiratory sequelae (Hazard Ratio [HR] = 0.259, $p=0.040$ ).
Cardiovascular Outcomes: No significant association found (HR = 0.927, $p=0.835$ ).
Kidney Outcomes: Point estimate suggested higher risk (HR = 1.675), but results were not statistically significant ( $p=0.239$ ) and likely influenced by selection bias and keyword-based ascertainment.

5. Significance and Future Directions

Bridging the Data Gap: This work demonstrates that LLMs can effectively transform unstructured clinical narratives into structured temporal data, enabling longitudinal modeling where structured EHR data is insufficient or unavailable.
Risk Modeling: The study validates the utility of these timelines for downstream risk prediction, specifically identifying a potential protective effect of GLP-1RAs on respiratory outcomes.
Scalability: The pipeline is disease-agnostic and can be extended to other conditions, though it requires careful handling of varying time scales (acute vs. chronic).
Limitations: The study acknowledges publication bias in case reports, the high cost of expert annotation, and the potential for LLM extraction errors. It emphasizes that the results are associative, not causal.
Future Work: The authors suggest integrating multimodal data (labs, imaging) to reconcile "perceived" event timing with "documented" timing and refining temporal extraction for different clinical domains.

In conclusion, this paper establishes a foundational methodology for temporal phenotyping using LLMs, offering a scalable solution to extract longitudinal clinical insights from the vast but unstructured repository of medical literature.