💬 NLP

ELM: A Hybrid Ensemble of Language Models for Automated Tumor Group Classification in Population-Based Cancer Registries

ELM is a novel hybrid ensemble system combining fine-tuned encoder-only models and a large language model arbitrator that significantly improves automated tumor group classification accuracy in population-based cancer registries, reducing manual review requirements by 60–70% while maintaining high data quality standards.

Original authors: Lovedeep Gondara, Jonathan Simkin, Shebnum Devji, Gregory Arbour, Raymond Ng

Published 2026-03-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lovedeep Gondara, Jonathan Simkin, Shebnum Devji, Gregory Arbour, Raymond Ng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive library where every book is a medical report describing a patient's cancer. These reports are written by doctors in complex, technical language, and they are messy—some are short, some are long, and the most important information might be hidden in the first paragraph or buried at the very end.

The job of the Cancer Registry is to sort these thousands of books into the correct "shelves" (categories like "Lung Cancer," "Skin Cancer," or "Leukemia") so researchers and doctors can study them.

The Problem: The Overworked Librarian

For years, human librarians (called registrars) had to read every single report manually.

The Struggle: There are about 90,000 reports a year. Reading them all takes about 900 hours of human work every year.
The Old Robot: They tried using a simple computer program (called eMaRC) to do the sorting. Think of this old robot as a person who only knows how to look for specific keywords. If it sees the word "breast," it assumes it's breast cancer. But if the report says "skin cancer on the breast," the robot gets confused and sorts it wrong. It fails about 40% of the time, so the humans still had to check everything anyway.

The Solution: Meet ELM (The Super-Team)

The authors created a new system called ELM (Ensemble of Language Models). Instead of one robot, they built a team of specialists who work together like a high-stakes jury.

Here is how the team works, using a simple analogy:

1. The Six "Quick Readers" (The Encoder Models)

Imagine you have six very fast, smart readers.

The Strategy: Because medical reports can be long, no single reader can hold the whole thing in their memory at once. So, the team splits the work:
- Three readers only read the top half of the report (where the final diagnosis usually lives).
- Three readers only read the bottom half (where the doctor's detailed reasoning and comments live).
The Vote: Each reader guesses the cancer category. If 5 out of 6 readers agree on the answer (e.g., "It's Skin Cancer"), the team accepts that answer immediately. This is fast and cheap.

2. The "Expert Judge" (The Large Language Model)

Sometimes, the six readers disagree, or the case is tricky (like distinguishing between two very similar types of blood cancer).

The Handoff: Instead of guessing, the system sends these difficult cases to a Super-Expert (a Large Language Model, or LLM).
The Trick: The Super-Expert isn't asked to guess from all 19 possible cancer types (which would be overwhelming and prone to mistakes). Instead, the system tells the Expert: "The six readers think it's either Skin Cancer or Melanoma. You are the judge: which one is it?"
The Result: The Expert uses its deep understanding of language to make the final call, explaining why it chose that answer.

Why This Team is a Game-Changer

Speed & Cost: The six "Quick Readers" are very fast and cheap. They handle about 80-85% of the work in less than a second. The expensive "Super-Expert" only steps in for the tough 15-20% of cases.
Accuracy: By combining the speed of the small readers with the deep reasoning of the Expert, the system is much smarter than any single part.
- Analogy: It's like having a team of detectives who quickly scan a crime scene, and when they find a confusing clue, they call in a forensic specialist to solve it.
Real-World Impact: At the British Columbia Cancer Registry, this system has reduced the human workload by 60-70%. The human librarians no longer have to read every single report; they only review the tricky ones the computer flagged. This saves about 900 hours of work a year, allowing humans to focus on complex tasks that actually require human empathy and judgment.

The Bottom Line

ELM is a hybrid team that combines the speed of small, specialized tools with the brainpower of a giant AI. It solves the problem of "too much text, too many confusing words" by dividing and conquering. It proves that you don't need one giant, expensive brain to do everything; sometimes, a well-organized team of smaller brains plus a smart referee works best.

1. Problem Statement

Population-Based Cancer Registries (PBCRs) rely on extracting structured data from unstructured pathology reports to monitor cancer trends. A critical step in this workflow is Tumor Group Classification, which categorizes cancers into broad anatomical and histological groups (e.g., breast, lung, leukemia).

Current Bottleneck: This process is traditionally manual, requiring highly trained registrars. For a medium-sized registry processing ~100,000 reports annually, this consumes approximately 900 person-hours.
Limitations of Existing Automation: Current rule-based systems (e.g., eMaRC) fail to handle the linguistic complexity of clinical text. They rely on pattern matching, struggle with long-range dependencies, and cannot distinguish context (e.g., differentiating a skin biopsy on breast tissue from breast cancer). At the British Columbia Cancer Registry (BCCR), eMaRC fails to assign a tumor group to ~40% of cases and has high error rates, necessitating full manual review regardless of its output.

2. Methodology: The ELM Architecture

The authors propose ELM (Ensemble of Language Models), a hybrid architecture that strategically combines the computational efficiency of small, encoder-only models with the reasoning capabilities of Large Language Models (LLMs).

A. The Initial Ensemble (Small Models)

Architecture: The system employs six fine-tuned, encoder-only language models (based on GatorTron, BCCRTron, and ClinicalBERT).
Token Segmentation Strategy: To overcome the 512-token input limit of BERT-based models while ensuring full report coverage, the pathology reports are split into two segments:
- Top 512 tokens: Contains the "Final Diagnosis" or explicit tumor information.
- Bottom 512 tokens: Contains pathologist interpretations, comments, and impressions.
- Three models process the top; three process the bottom.
Voting Mechanism: The models generate independent predictions. A consensus is reached if at least 5 out of 6 models agree on a tumor group.

B. LLM Arbitration (Large Models)

Trigger Conditions: Cases proceed to the LLM if:
1. The ensemble agreement is low (<5 votes).
2. The predicted tumor group belongs to a pre-defined set of "hard-to-classify" categories (e.g., skin, cervix, multiple myeloma, primary unknown).
LLM Configuration: The system uses Mistral Nemo Instruct-2407 (12B parameters), deployed locally for privacy compliance.
Prompt Engineering: The LLM is not asked to choose from all 19 possible groups. Instead, the prompt:
- Constrains the output to a small set (2–4 options) derived from the ensemble's top predictions and domain expert knowledge.
- Provides specific guidance on difficult distinctions (e.g., Leukemia vs. Lymphoma, Melanoma vs. Non-melanoma).
- Requires a JSON output containing the prediction and reasoning.

C. Workflow Algorithm

Inference: Six small models process the report segments.
Aggregation: Votes are summed.
Decision:
- If High Confidence (≥5 votes) AND Easy Category: Output the majority vote.
- If Low Confidence OR Hard Category: Route to LLM with a constrained prompt.
Final Output: The LLM's decision is accepted as the final classification.

3. Key Contributions

Novel Hybrid Architecture: This is the first successful deployment of a hybrid small-model-LLM architecture for tumor group classification in a real-world PBCR setting.
Strategic Routing: The system optimizes cost and latency by routing only ~15–20% of cases (the ambiguous ones) to the expensive LLM, while the majority (80–85%) are resolved instantly by the ensemble.
Domain-Specific Prompting: The paper demonstrates how constraining LLM output spaces based on ensemble votes and expert rules significantly reduces hallucinations and improves accuracy compared to zero-shot LLM approaches.
Operational Validation: Unlike many academic papers, this work includes a full production deployment at BCCR with validated operational metrics.

4. Results

The model was evaluated on a held-out test set of 2,058 pathology reports spanning 19 tumor groups.

Overall Performance: ELM achieved a Weighted F1-score of 0.94 (Precision: 0.94, Recall: 0.94).
Comparative Analysis:
- vs. Rule-based (eMaRC): ELM significantly outperformed eMaRC (F1 0.94 vs. 0.64). eMaRC failed to predict 40% of cases.
- vs. Encoder-Only Ensemble: The hybrid approach improved the F1-score from 0.91 (ensemble only) to 0.94 (ELM), a statistically significant gain ( $p < 0.001$ ).
- vs. LLM-Only: A pure LLM approach (choosing from all 19 groups without guidance) scored 0.88 F1, proving the necessity of the ensemble's guidance.
Specific Gains on Challenging Categories:
- Leukemia: F1 improved from 0.76 to 0.88.
- Lymphoma: F1 improved from 0.76 to 0.89.
- Skin Cancer: F1 improved from 0.44 to 0.58 (with strategic routing, non-melanoma skin cancer improved from 0.58 to 0.74).
Efficiency:
- Latency: The ensemble processes reports in 600ms. With LLM arbitration for 15-20% of cases, the average processing time is **850ms**.
- Cost: A pure LLM approach would take 2–3 seconds per report (3–4x slower/more expensive).

5. Significance and Impact

Operational Efficiency: Deployed at BCCR, ELM reduced manual review requirements by 60–70%, saving an estimated 900 person-hours annually. This allows registrars to focus on complex cognitive tasks (staging, multi-primary reconciliation) rather than routine classification.
Data Quality: Post-deployment audits confirmed that the automated system maintained data quality standards equivalent to fully manual processes.
Scalability: The architecture is designed to be replicable for other registries processing 50,000–200,000 reports, potentially saving thousands of person-hours across the global cancer surveillance ecosystem.
Future Directions: The authors suggest extending the system to multi-label classification for multi-primary cancers, implementing active learning for rare tumor types, and exploring smaller LLMs (3B–7B parameters) for further cost optimization.

In conclusion, ELM demonstrates that a carefully designed hybrid approach, leveraging the speed of small models and the reasoning of large models, can solve complex clinical NLP tasks with high accuracy and operational feasibility.