A PubMed-Scale Dataset of Structured Biomedical Abstracts

Technical Summary: Structured PubMed

Problem Statement
Structured abstracts, which partition text into labeled sections (e.g., Background, Objective, Methods, Results, Conclusions), are critical for biomedical information retrieval, text mining, and knowledge synthesis. While the US National Library of Medicine (NLM) encourages this format based on the IMRAD convention, a substantial portion of PubMed abstracts remain unstructured. This lack of explicit section labels creates a bottleneck for section-aware retrieval and large-scale biomedical NLP, as methods developed on structured abstracts cannot be directly applied to the larger body of unstructured abstracts without additional labeling. Existing resources for structured abstracts are often limited in scale (tens to hundreds of thousands of records) or restricted to specific study types.

Methodology
The authors introduce Structured PubMed, a corpus of 23.3 million section-labeled biomedical abstracts derived from the complete NLM PubMed XML distribution (accessed May 3, 2026). The dataset construction involved the following steps:

1. Data Collection and Preprocessing:

   • The raw PubMed XML files (baseline snapshot and update files) contained 40,451,610 citation records.
   • Records with empty abstracts were excluded.
   • Non-research article types (e.g., letters, case reports, editorials, reviews) were removed, retaining 23,360,953 research article records.
   • Records were classified as either "structured" (containing NLMCategory or Label attributes in the XML) or "unstructured."

2. Dataset Composition:
   The corpus is divided into two subsets:

   • Author-Structured Subset: 5,993,846 records where section labels were parsed directly from the original XML files using NLM-assigned categories.
   • LLM-Labeled Subset: 17,287,922 records with originally unstructured abstracts. These were processed using a Large Language Model (LLM) pipeline to assign section labels.

3. LLM-Based Structuring Pipeline:

   • Model Selection: The authors evaluated GPT-4.1-nano, GPT-4.1-mini, and a fine-tuned BERT-based sentence classification model (BERT CLS). GPT-4.1-mini was selected for production due to superior performance.
   • Prompt Engineering: A "verbatim-extraction" protocol was employed. The system prompt constrained the model to extract text exactly as written (no paraphrasing or typo correction), remove explicit headers, and output a valid JSON object. If a section was absent, the model returned null.
   • Boundary Rules: Specific linguistic cues were provided to distinguish sections (e.g., OBJECTIVE as the statement of intent; METHODS as technical steps).

4. Technical Validation:

   • Evaluation Sample: 30,000 records (10,000 each from Research Articles, Observational Studies, and Randomized Clinical Trials) with known gold-standard structured abstracts were used. The structured sections were concatenated to form unstructured input, simulating a realistic upper-bound test.
   • Metrics: Performance was measured using ROUGE-1, ROUGE-2, ROUGE-L, and sentence-level BLEU scores.

Key Contributions

• Scale: Structured PubMed is the largest corpus of section-labeled biomedical abstracts released to date, approximately 4.09 times the size of the previous largest resource (MedConclusion).
• Coverage: By applying LLM-based labeling to unstructured abstracts, the dataset extends section labels to a significantly larger portion of PubMed than resources restricted to natively structured abstracts.
• Standardization: All records are harmonized under a unified five-section schema (Background, Objective, Methods, Results, Conclusions) and mapped to their original PubMed ID, publication type, and date, ensuring traceability and integration with other PubMed-derived datasets.

Results

• Model Performance: GPT-4.1-mini achieved the highest performance across all metrics and article types. Its aggregate ROUGE-1, ROUGE-2, ROUGE-L, and BLEU scores remained stable across Research Articles, Observational Studies, and Randomized Clinical Trials.
• Comparison:
  • GPT-4.1-mini: Maintained exceptionally high precision across all sections (scores never dipping below 0.90 in the Research Article subset). It demonstrated robust boundary detection unaffected by varying linguistic styles.
  • GPT-4.1-nano: Underperformed relative to GPT-4.1-mini, particularly in delineating core body and data-heavy sections (Methods, Results, Conclusions), though it performed better than BERT CLS on introductory elements.
  • BERT CLS: Achieved high overall averages but showed uneven performance, struggling significantly with the OBJECTIVE section (ROUGE-1 = 0.74) due to confusion between background context and research intent.
• Error Profile: The high ROUGE and BLEU scores for GPT-4.1-mini reflect the verbatim-extraction design; deviations from the gold standard primarily arose from boundary placement errors rather than lexical substitutions, ensuring faithful attribution to the original text.

Significance and Applications
The authors claim that Structured PubMed supports reuse across several settings:

• Model Training and Evaluation: The author-provided subset offers human-curated data for training and evaluating abstract-structuring models, including sentence-classification approaches.
• Extended Coverage: The LLM-labeled subset enables section-specific retrieval, summarization, and evidence extraction over records that prior labeled resources did not cover.
• Benchmarking: The dataset allows users to compare model behavior across natively structured and retrospectively structured abstracts.
• Downstream Tasks: The corpus can be utilized to train sentence-classification models, benchmark text-segmentation architectures, and perform large-scale, section-specific information extraction at an unprecedented PubMed-wide scale.