CaresAI at SMM4H-HeaRD 2026: Predicting TNM Staging

Imagine you are a detective trying to solve a complex medical mystery. The clues are hidden inside long, messy, handwritten-style reports written by pathologists (doctors who examine tissue samples). These reports describe a patient's cancer, but they don't always say the answer clearly. Your job is to figure out three specific things about the cancer:

1. T (Tumor): How big is the main lump, and how deep has it burrowed into the nearby tissue?
2. N (Node): Has the cancer spread to the nearby lymph nodes (the body's security checkpoints)?
3. M (Metastasis): Has the cancer escaped to other parts of the body entirely?

This is called TNM Staging, and it's the "scorecard" doctors use to decide how to treat a patient. The team at CaresAI entered a competition (SMM4H-HeaRD 2026) to build a computer program that can read these messy reports and automatically guess the T, N, and M scores.

Here is how they did it, explained simply:

1. The Challenge: Reading the "Fine Print"

The reports they used came from a massive database called TCGA. Think of these reports as thousands of different novels written by different authors. Some are short, some are huge, some use fancy medical jargon, and some are vague.

• The Problem: The data was "unbalanced." Imagine a bag of marbles where 93% are white (no cancer spread) and only 7% are black (cancer spread). If you just guess "white" every time, you'd be right most of the time, but you'd miss the dangerous black marbles. The computer had to learn to spot the rare, dangerous clues.

2. The Tools: Two Different Ways to "Read"

The team tried two main ways to teach the computer how to understand the text:

• Method A: The "Keyword Counter" (TF-IDF & LightGBM)
  Think of this like a librarian who counts how often specific words appear. If a report mentions "metastasis" or "spread" a lot, the computer flags it. They used a smart algorithm called LightGBM (which is like a super-fast, organized decision tree) to look at these word counts.

  • Result: This was the champion. It was very good at spotting the patterns in the text, almost like a seasoned detective who knows exactly which words to look for.

• Method B: The "Deep Thinker" (BERT Models & Neural Networks)
  This approach used advanced AI models (ClinicalBERT, BioBERT) that have already read millions of medical books. Instead of just counting words, these models try to understand the meaning and context of the sentences. They then fed this "understanding" into a Wide Residual Network (WRN), which is a type of brain-like computer network designed to spot complex patterns.

  • Result: This was a bit more hit-or-miss. While it understood the "vibe" of the text well, it sometimes got confused by the specific details compared to the keyword counter.

3. The Results: How Well Did They Do?

The team tested their "detectives" on two different sets of new, unseen reports (Test Set 1 and Test Set 2).

• The Good News: On the first test set, the computer was incredibly accurate. It got a score of 0.978 (out of 1.0) for the Tumor stage and 0.957 for the Node stage. It was like a detective who solved almost every case perfectly.
• The Bad News: When they moved to the second test set, the scores dropped (down to around 0.80).
  • Why? The paper explains that the computer got "spoiled" by the first set of data. It learned the specific patterns of the training data too well (a problem called overfitting). When it saw a report that was slightly different or longer than usual, it got confused. Also, because the "dangerous" cases (like cancer spreading) were so rare in the training data, the computer struggled to recognize them when they appeared in new, tricky reports.

4. The Limitations: Why It's Not Ready for the Hospital Yet

The authors are very honest about what their tool cannot do yet:

• The "Page Limit" Problem: The AI models they used can only read about 512 words at a time. Many of these medical reports are much longer. It's like trying to read a whole novel but only being allowed to read the first few pages; the computer misses the ending and important clues.
• The "Rare Event" Problem: Because the "cancer spread" cases were so rare in the training data, the computer is still a bit biased. It's safer to say "no spread" than to guess "spread," which means it might miss some real cases.
• Not a Doctor: The paper explicitly states that while this is a great "baseline" (a starting point), it is not reliable enough to be used in a real hospital to make life-or-death decisions yet. It needs more training and better handling of long documents.

The Bottom Line

The CaresAI team built a computer program that can read cancer reports and guess the stage of the disease with impressive accuracy on familiar data. They found that a simple "keyword counting" method (LightGBM) actually worked better than the fancy "deep thinking" AI for this specific task.

However, just like a student who aces a practice test but struggles on the real exam because the questions were slightly different, this AI needs more work to handle the messy, long, and rare cases found in the real world before it can be trusted to help doctors. For now, it's a powerful tool for research and a great starting point for future improvements.