Prediction of HER2 FISH result from registered H&E and IHC slides in breast cancer
This study demonstrates that a multimodal deep learning model trained on registered paired H&E and IHC slides can accurately predict HER2 FISH results, potentially eliminating the need for costly FISH testing in approximately one-third of equivocal breast cancer cases.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Breast cancer is not a single disease but a collection of different conditions, each driven by unique molecular signals that dictate how the tumor grows and how it should be treated. One of the most critical signals to identify is a protein called HER2. When this protein is present in high amounts, it acts like a gas pedal for the cancer, causing it to grow rapidly. Fortunately, doctors have developed targeted therapies that can turn off this gas pedal, but these treatments only work if the patient's cancer actually has the HER2 signal. To find out, pathologists examine tissue samples under a microscope. They first look at a standard stained slide, which shows the general shape of the cells, and then they often use a special chemical stain to highlight the HER2 protein itself. If the protein levels are clearly high or clearly low, the diagnosis is straightforward. However, about one in seven patients falls into a gray area where the protein levels are ambiguous. For these patients, the standard of care requires a third, more complex test that uses light to count the actual genes inside the cells. This additional test is expensive, time-consuming, and not available to everyone, particularly in regions with fewer medical resources.
A team of researchers from Israel, Portugal, and the United States has explored a way to bypass this difficult third step for many patients. They asked whether a computer, trained on thousands of microscope images, could look at the standard tissue slide and the special protein stain together to predict the result of the complex gene test. To do this, they first had to solve a physical puzzle: the two slides come from the same piece of tissue but are cut at different times and stained differently, meaning they do not line up perfectly. The researchers developed a method to digitally align these two images, matching the specific patterns of cells on one slide to the corresponding patterns on the other. They then fed these aligned pairs into a deep learning system, a type of artificial intelligence that learns by analyzing vast amounts of visual data. The system was trained to recognize the subtle visual cues in the tissue that indicate whether the cancer cells are likely to have the gene amplification that the complex test usually detects.
The study, which analyzed data from nearly 1,500 patients, found that this combined approach works with remarkable accuracy. When the researchers tested the system on cases where the initial protein stain was ambiguous, the computer model was able to predict the gene test result with a high degree of certainty. In roughly one-third of these difficult cases, the model was so confident in its prediction that it could effectively rule out the need for the expensive gene test. For about eight percent of these ambiguous cases, the model was certain the cancer was positive for the gene, and for another twenty-one percent, it was certain the cancer was negative. In these specific groups, the model's predictions were so reliable that they matched the gold-standard test perfectly in the study's validation. This suggests that for a significant portion of patients, the complex gene test could be skipped without compromising the accuracy of the diagnosis.
The researchers also compared their new method against using just the protein stain or just the standard tissue slide. They found that while the protein stain alone was good, adding the standard tissue slide significantly improved the results. The standard slide provided context, helping the computer focus specifically on the cancerous areas and ignore the surrounding healthy tissue, which led to better predictions. The study suggests that this technology could be a powerful tool for triage, helping doctors decide which patients truly need the expensive gene test and which ones can be treated based on the simpler images alone. This could be especially valuable in countries where the complex test is hard to obtain, potentially allowing more patients to receive the right life-saving treatment sooner.
However, the authors are careful to note that this is not a replacement for the gene test in every situation. The model still makes mistakes in a small number of cases, and it was not designed to detect very low levels of the protein that are becoming relevant for new treatments. The study was conducted on a specific set of data, and the researchers acknowledge that the system needs to be tested in other hospitals and with different patient groups before it can be widely adopted. They also point out that their method currently requires a human to mark a few points on the slides to help the computer align them, a step that is being refined to become fully automatic. Despite these limitations, the work demonstrates that combining two common types of microscope images with artificial intelligence can extract information that was previously thought to require a much more complex and costly procedure.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.