CrcBiomeScreen: a reproducible workflow for class imbalance and cross-cohort generalisability in colorectal cancer microbiome prediction

Colorectal cancer is a leading cause of death worldwide, often because it is discovered too late for treatment to be fully effective. Current screening methods, such as tests that look for hidden blood in stool, are helpful but imperfect; they can miss early signs of disease or flag healthy people for unnecessary and invasive follow-up procedures. In recent years, scientists have turned their attention to the trillions of tiny organisms living in our guts, known as the microbiome. These microbial communities leave distinct chemical signatures that change when cancer or pre-cancerous growths develop. The hope is that by analyzing these microbial patterns, doctors could identify high-risk individuals more accurately than blood tests alone. However, turning these biological signals into a reliable medical tool is fraught with difficulty. The data is messy, the number of healthy people vastly outnumbers those with cancer, and the way researchers process the data can drastically change the results, making it hard to know which methods actually work.

To navigate this complexity, a team of researchers at the University of Leeds developed a new, open-source tool called CrcBiomeScreen. Rather than simply proposing a single "best" way to predict cancer, they built a flexible framework that allows scientists to test different strategies side-by-side to see which ones hold up under pressure. They used this system to analyze stool samples from over 2,200 people involved in a real-world screening program in the UK, as well as nine other independent datasets from around the world. Their goal was to find out which specific steps in the analysis—how the data is cleaned, how the different types of bacteria are counted, and how the computer models handle the fact that cancer cases are rare—lead to the most accurate and reliable predictions.

The researchers discovered that the choices made before the computer even starts learning are critical. They tested several ways to normalize the data, a process that adjusts for differences in how much genetic material was collected from each sample. They found that a specific method called GMPR, which accounts for the unique way microbial communities are structured, consistently produced better results than older, simpler methods. They also examined whether it mattered to include bacteria that have never been successfully grown in a lab, often labeled as "uncultured." The study showed that keeping these mysterious, uncultured bacteria in the analysis did not hurt the model's performance and might actually preserve valuable clues about the disease that would otherwise be lost. Furthermore, they determined that looking at the bacteria at the genus level—a broad category that groups related species together—provided the best balance. This level of detail was specific enough to be biologically meaningful but broad enough to be recognized consistently across different laboratories and sequencing technologies.

A major hurdle in cancer screening is the sheer imbalance between healthy people and those with the disease. In a real-world screening program, for every person with cancer, there are hundreds or thousands of healthy individuals. If a computer model is trained on a dataset that has too many cancer cases, it might perform well in the lab but fail miserably when applied to the general public. The team simulated this extreme imbalance by creating test scenarios where healthy controls vastly outnumbered cancer cases. They found that a technique called class weighting was essential. This approach tells the computer to pay extra attention to the rare cancer cases during training, effectively making the model more sensitive to the disease without losing its ability to correctly identify healthy people. When they applied this weighting, the model maintained its ability to spot cancer cases while significantly reducing the number of healthy people who were incorrectly flagged as sick. This improvement in precision is vital for screening, as it helps avoid unnecessary colonoscopies and the anxiety they cause.

The researchers then tested how well these models worked when moved from the UK dataset to entirely different populations from nine other countries. This is where the choice of computer algorithm made a surprising difference. One type of model, known as Random Forest, proved to be very steady and reliable, performing consistently well even when the training data was limited or the populations were very different. Another type, called XGBoost, was more powerful when trained on large, diverse datasets, achieving higher accuracy in those specific conditions but showing more variability when the data was smaller. The study suggests that there is no single "magic bullet" algorithm that works best in every situation. Instead, the best approach depends on the size and nature of the data available.

Ultimately, the value of CrcBiomeScreen lies in its transparency and flexibility. It does not force researchers to use a single, rigid method. Instead, it provides a structured way to compare different options, ensuring that the final tool chosen is the one that works best for the specific dataset at hand. By demonstrating that careful attention to data processing, the inclusion of uncultured bacteria, and the use of class weighting can significantly improve performance, the study offers a roadmap for developing more reliable microbiome-based screening tools. While these models are not yet ready to replace existing tests, they represent a significant step toward a future where a simple stool test could more accurately identify who needs urgent medical attention, helping to catch colorectal cancer earlier and save more lives.