💬 NLP

Learning from Lost Provenance: Multiple Instance Learning for Cancer Registry Tumor Group Classification

This paper proposes an Attention-Based Multiple Instance Learning (ABMIL) framework that leverages routine, patient-level cancer registry labels to recover lost report-level provenance, distilling a high-quality training dataset that enables deep learning models to achieve high-performance tumor group classification without requiring additional manual annotation.

Original authors: Leonard Ruocco, Jonathan Simkin, Lovedeep Gondara, Gregory Arbour, Raymond Ng

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Leonard Ruocco, Jonathan Simkin, Lovedeep Gondara, Gregory Arbour, Raymond Ng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you have a massive pile of clues (pathology reports) and only one final verdict for the whole case (the tumor group). The problem? You don't know which specific clue led to that verdict. In a cancer registry, doctors (called Oncology Data Specialists) read dozens of reports for a single patient and then assign one label to the whole patient. But the computer doesn't know which report was the "smoking gun" and which was just a red herring.

For a long time, researchers tried to solve this by either throwing away most of the clues (keeping only patients with one report) or by assuming every clue was the smoking gun. The paper argues that the second idea is a trap. If you tell the computer that every single report for a patient is the reason for the diagnosis, you end up training it on noise. It's like telling a student that every page in a textbook explains the answer, even the pages about unrelated topics. The paper explicitly rules out this "label propagation" method as a good way to train models because it confuses the system, especially for rare types of cancer.

Instead, the authors propose a clever new way to learn called Attention-Based Multiple Instance Learning (ABMIL). Think of this like a smart detective who looks at the whole pile of clues at once. The model doesn't try to guess the answer from just one report. Instead, it looks at the entire "bag" of reports for a patient and asks, "Which of these reports actually matters?" It assigns a score of importance to each one.

Here is the magic trick: The model learns to ignore the boring, irrelevant reports and focus only on the ones that actually explain the diagnosis. Once it figures out which reports are the "stars of the show," it can do two cool things:

Direct Detective Work: It can use that "bag of clues" approach to classify new cases directly, without needing to know exactly which report is which.
The Great Filter: It can act as a sieve. It takes the huge pile of reports, throws away the ones it thinks are irrelevant, and keeps only the high-scoring ones. Then, it takes the patient's label and sticks it onto those few, high-quality reports. Suddenly, you have a clean, perfect dataset where every single report has a correct label, all without a human having to read and label them one by one.

The researchers tested this on data from the British Columbia Cancer Registry. They looked at 82,855 unique patients and 359,714 pathology reports. They split this data up, keeping 90% for training and 10% for checking their work. They also tested their final system on a fresh batch of 2,057 reports that had been manually checked by experts (the "ODS-annotated dataset").

The results were promising. The model that used this filtering trick (called ABMIL-DISTIL) managed to get a score of 0.83 (a macro F1 score) on the new test data. This was better than the old methods that just assumed every report was important. The paper suggests that this approach is a practical way to turn messy, real-world hospital records into high-quality training data without needing a massive army of humans to label everything or super-computers to process it all.

It's important to note that while the results are strong, the paper treats this as a measured success on specific data, not a magic bullet that solves everything forever. The authors point out that their test data used a slightly different labeling system than their training data, which adds a tiny bit of "noise" to the final score, meaning the real-world performance might vary slightly. But overall, they suggest this is a very accessible and effective route to automating cancer registry work, turning a mountain of unorganized paperwork into a sharp, focused tool.

Technical Summary: Learning from Lost Provenance via Attention-Based Multiple Instance Learning

Problem Statement
Modernizing cancer registries with deep learning faces a critical bottleneck: the scarcity of individual pathology reports labeled at the document level. While Population-based Cancer Registries (PBCRs) like the British Columbia Cancer Registry (BCCR) generate vast amounts of expert-assigned tumor group labels, these labels exist at the patient level (aggregated over a patient's history) rather than the report level. Crucially, the structural link between a specific patient-level label and the individual pathology reports that informed it is lost in routine operations.

Existing approaches to this "lost provenance" problem have significant limitations:

Filtering for single-report patients: Discards the majority of available data and introduces selection bias.
Label Propagation: Assigns the patient-level label to every associated report. This creates noisy training data by labeling clinically irrelevant reports (e.g., incidental findings, workup biopsies) as positive, degrading model performance, particularly for minority tumor groups.
Text Concatenation: Merging all reports into a single text block exceeds computational context windows and prevents the model from isolating relevant documents.

Methodology
The authors propose a two-stage framework leveraging Attention-Based Multiple Instance Learning (ABMIL) to recover the link between patient-level labels and individual reports without requiring manual per-report annotation.

Data Structure: Patients are treated as "bags" of "instances." Each bag corresponds to a Patient Health Number (PHN) with a single registered tumor (the bag label), and the instances are the associated pathology reports.
Stage 1: Weakly Supervised Classification (ABMIL-WS):
- Encoder: Reports are encoded using BCCRTron, a domain-adapted clinical language model (based on GatorTron-base) pretrained on 1.1 million BCCR pathology reports. The [CLS] token serves as the 1024-dimensional report embedding.
- Attention Mechanism: The model computes gated attention weights ( $a_i$ ) for each report within a bag using a learnable network (Equation 1). This allows the model to identify which specific reports contribute to the bag-level label without instance-level supervision.
- Pooling: Report embeddings are pooled into a single bag-level representation ( $z$ ) weighted by their attention scores, which is then passed to a classification head to predict the tumor group.
- Training: Only the attention pooling layer and classification head are trained; the encoder remains frozen.
Stage 2: Dataset Distillation (ABMIL-DISTIL):
- The attention scores from the trained ABMIL-WS model are repurposed as relevance scores.
- The corpus is filtered by retaining only reports with attention scores above specific percentile thresholds (e.g., 85th, 95th).
- The patient-level label is propagated only to these high-confidence, filtered reports.
- A standard supervised classifier is then fine-tuned on this distilled, high-quality dataset. This stage allows for end-to-end fine-tuning (with partially frozen layers) on a computationally manageable subset of the data.

Key Contributions

Framework Application: This is the first application of ABMIL to cancer-registry tumor-group classification over free-text pathology reports using operationally generated bag labels.
Noise Reduction via Distillation: The paper demonstrates a method to transform a large, noisy, patient-level labeled corpus into a compact, high-quality per-report training dataset without additional human annotation.
Practical Accessibility: The approach avoids the need for large-scale computing infrastructure required for full end-to-end training on unfiltered data, making it accessible for institutions with varying resource capacities.
Dual-Pathway Evaluation: The study evaluates both the direct use of the ABMIL model for classification and the utility of ABMIL as a filtering mechanism to improve downstream supervised classifiers.

Results

Performance: A classifier fine-tuned on the distilled dataset (ABMIL-DISTIL) achieved a macro F1 score of 0.83.
Comparison: This performance outperformed established baselines, including:
- Label Propagation: Training on all reports with propagated labels (equivalent to 0% attention threshold).
- Concatenated Bag: Training on concatenated text sequences using a model with a larger context window (BCCRTron-2k).
Threshold Analysis: The study explored various filtering thresholds (0% to 99%). While higher thresholds reduced dataset size, they increased the likelihood that retained reports were diagnostically relevant, generally improving model performance up to an optimal point before data scarcity became a limiting factor.
Evaluation Benchmark: Performance was validated on an independent, ODS-annotated dataset of 2,057 reports collected in 2025. The authors note a modest degree of label noise in this test set due to differences between the training (CAIS mapping) and testing (SEER-based mapping) schemas.

Significance
The paper claims that ABMIL offers a practical and accessible route to automating cancer registry workflows. By leveraging routine operational labels to distill high-quality training data, the method addresses the primary bottleneck of supervised NLP in clinical settings: the lack of document-level annotations. The authors posit that this approach can be generalized to virtually all PBCRs facing the same structural challenge of patient-level labels without recorded links to individual reports, thereby democratizing cancer registry NLP.

Technical Summary: Learning from Lost Provenance via Attention-Based Multiple Instance Learning

More like this