Technical Summary: Memorization in Large Language Models in Medicine

Problem Statement

While Large Language Models (LLMs) have demonstrated potential in medicine through continued pretraining and fine-tuning, a critical gap exists in understanding the extent to which these models memorize their training data. Memorization, defined as the recall or regeneration of content seen during adaptation, presents a dual-edged sword in the medical domain. On one hand, it may facilitate the retention of valuable domain knowledge (e.g., clinical guidelines, biomedical concepts). On the other, it poses significant risks, including the inadvertent reproduction of sensitive Protected Health Information (PHI), the reliance on surface-level copying rather than deep reasoning, and the potential for hallucinated or repetitive outputs that undermine safety and generalizability. Current literature lacks a systematic assessment of memorization prevalence, characteristics, and downstream impacts specifically within medical adaptation scenarios.

Methodology

The study systematically investigates memorization across three common adaptation scenarios:

1. Continued Pretraining: Adapting general-purpose LLMs on large-scale medical corpora (biomedical literature and clinical notes).
2. Fine-tuning on Standard Benchmarks: Instruction tuning on medical question-answering (QA) datasets.
3. Fine-tuning on Real-World Clinical Data: A case study using 13,000 unique inpatient records from the Yale New Haven Health System for disease diagnosis.

Models Evaluated:
The analysis covers both medical foundation models (PMC-LLaMA, Meditron, Me-LLaMA, Med-Llama-3) and general-purpose LLMs (Llama 2/3 variants, gpt-oss).

Evaluation Metrics:
The authors employ a multi-faceted evaluation pipeline:

• Exact Measures: Consecutive token matches (30 and 50 tokens) to detect verbatim regeneration.
• Answer Option Regeneration: Specifically for QA benchmarks, measuring the regeneration of removed multiple-choice options.
• Approximate & Semantic Measures: ROUGE-L, BLEU, BERT, and BART scores to capture non-verbatim but semantically similar memorization.
• Manual Review: Human evaluation of model outputs to categorize memorized content into beneficial, uninformative, and harmful types, and to verify PHI detection tools against ground truth.
• Training Dynamics: Analysis of memorization ratios, validation loss, and test accuracy across training epochs to determine when memorization emerges.

Key Results

1. Prevalence of Memorization

Memorization is prevalent and significantly higher in medical adaptation compared to general-domain LLMs.

• Continued Pretraining: Models exhibited memorization ratios of 10%–22% for consecutive 30-token spans on specific medical corpora (e.g., Clinical guidelines), compared to ~0.1% in general domains.
• Fine-tuning on Benchmarks: Models regenerated 14%–21% of removed answer options in medical QA tasks. Notably, the original release of PMC-LLaMA showed exceptionally high memorization (18.2% on MedQA), which was not replicated in a reproduced version, suggesting overfitting in the original release.
• Fine-tuning on Clinical Data: Fine-tuned models showed 2%–30% memorization ratios for consecutive 30-token spans. Crucially, in a case study with 10,000 training records, fine-tuned models regenerated 3,192 PHI instances. Manual review of 200 outputs revealed 98 additional instances of sensitive information (e.g., family relationships, specific diagnoses) missed by automated de-identification tools.

2. Characteristics and Persistence

• Persistence: Memorization acquired during continued pretraining is highly persistent. Up to 87% of content memorized during pretraining remained after subsequent fine-tuning on new tasks.
• Distinct Patterns:
  • Continued Pretraining: Tends to produce higher rates of verbatim (exact) memorization.
  • Fine-tuning: Tends to produce more task-specific memorization (e.g., regenerated answer options) with lower exact token match rates but significant semantic overlap.
• Factors Influencing Memorization: Memorization increases with model size (e.g., 70B > 7B) and input length. Decoding parameters like temperature had limited influence. Memorization often increases early in training (by epoch 3–4), even while validation loss is still decreasing, suggesting it is not solely a result of late-stage overfitting.

3. Categorization of Memorization

The study classifies memorization into three types based on impact:

• Beneficial: Accurate recall of domain-specific knowledge (e.g., drug instructions, clinical guidelines, gene-disease associations) that supports reasoning and accuracy.
• Uninformative: Regeneration of templated language, standard disclaimers, and repetitive statements that reflect surface-level copying rather than deep understanding.
• Harmful: Regeneration of data-specific content, including PHI, sensitive diagnoses, and dataset artifacts (e.g., specific answer options), which poses privacy risks and limits generalization.

Significance and Contributions

The paper claims the following contributions to the field of medical AI:

1. Systematic Benchmarking: It provides the first comprehensive assessment of memorization prevalence across the full pipeline of medical LLM adaptation (pretraining to fine-tuning), establishing that medical LLMs exhibit significantly higher memorization rates than their general-domain counterparts.
2. Risk Identification: It highlights that standard de-identification processes are insufficient, as models can regenerate PHI and sensitive information beyond standard definitions. It demonstrates that memorization can emerge early in training, challenging the assumption that it is solely a late-stage overfitting phenomenon.
3. Practical Recommendations: The authors propose a framework to manage memorization:
   • Facilitate Beneficial Memorization: Monitor the preservation of domain knowledge across adaptation stages.
   • Minimize Uninformative Memorization: Use diverse training data (deduplication) and reasoning-focused post-training.
   • Mitigate Harmful Memorization: Employ privacy-preserving techniques (e.g., differential privacy) and retrieval-augmented generation (RAG) to reduce reliance on training data regeneration.
4. Community Call to Action: The paper advocates for integrating memorization metrics into standard reporting guidelines for medical LLMs, moving beyond accuracy-only evaluations to ensure safety and accountability in downstream deployment.

The study concludes that while memorization is an inherent and sometimes beneficial trait of LLM adaptation in medicine, its unmanaged prevalence poses critical risks to patient privacy and model reliability, necessitating new evaluation standards and mitigation strategies.