Interpretable Concept-Bottleneck Vision-Language Model Predicts Pathologic Complete Response From Longitudinal Breast DCE-MRI

In the fight against breast cancer, doctors often begin with a powerful strategy called neoadjuvant therapy, where patients receive treatment before surgery to shrink tumors. The ultimate goal is a pathologic complete response, a state where no cancer cells can be found in the tissue removed during surgery. This outcome is a strong sign that a patient will live longer and stay cancer-free. However, predicting who will achieve this result is difficult. For years, scientists have tried to use artificial intelligence to look at MRI scans and make these predictions, but most of these systems act like black boxes. They produce an answer, but they cannot explain how they reached it. Because doctors cannot see the reasoning behind the machine's decision, they are often hesitant to trust it, especially when a patient's life is on the line. To be useful in a hospital, an AI tool needs to be transparent, showing its work in a way that a human expert can verify.

A team of researchers has developed a new kind of artificial intelligence that solves this problem by forcing the machine to think in terms of recognizable medical features. Instead of looking at a scan and instantly guessing the outcome, this system first identifies eleven specific characteristics of the tumor, such as its shape, its size, and how it looks on the scan. These features are based on standard medical descriptions that radiologists use every day. The system then uses these identified features, along with the patient's age and the specific type of cancer they have, to generate a written report. This report explains the reasoning behind the prediction, much like a doctor writing a note to a colleague. The researchers tested this system on data from hundreds of patients and found that it could accurately predict who would respond well to treatment, while also providing a clear, auditable explanation for every single prediction.

The study focused on patients from two large, multi-center clinical trials, involving a total of 753 individuals who had undergone dynamic contrast-enhanced MRI scans. This type of imaging involves taking pictures of the breast before and after injecting a dye, which helps doctors see how blood flows through the tumor. The researchers trained their model on data from the I-SPY2 trial and then tested it on a completely separate group of patients from the I-SPY1 trial to see if it would work on different machines and with different scanning protocols. The model was designed to look at two sets of images for each patient: one taken before treatment began and another taken early in the treatment process, usually after about three weeks. By comparing these two time points, the system could see how the tumor was changing.

The core of this new approach is a "concept bottleneck." Before the model makes a final prediction about whether a patient will be cured, it must first identify specific, measurable concepts. These include things like the functional tumor volume, which is a measure of how much of the tumor is actively growing, and the sphericity, which describes how round or irregular the tumor is. It also looks at whether the tumor appears as a solid mass or as a more diffuse area of enhancement, and whether it has sharp or fuzzy edges. The system predicts these eleven concepts with a high degree of accuracy, matching the results that would be obtained by a computer program designed specifically to measure them. Once these features are identified, they are fed into a language model that acts like a radiologist. This language model reads the list of features and writes a structured report, explaining which factors are pushing the prediction toward a cure and which are pushing it away.

When the researchers tested the model, it performed well. On the group of patients used for testing, the system correctly distinguished between those who would achieve a complete response and those who would not with an accuracy score of 0.76. When tested on the completely independent group of patients from the other trial, the accuracy remained strong at 0.71. This is significant because it shows the model can work across different hospitals and with different types of MRI machines, a common hurdle for medical AI. The study also revealed that the most important factor in predicting a cure was the change in the functional tumor volume. In other words, the patients whose tumors shrank the most in the first few weeks of treatment were the ones most likely to have no cancer left at the end. This finding aligns with what doctors already know, but the new system confirmed it without being explicitly told to look for it.

The researchers also found that the model's confidence was a useful guide. When the system was very sure of its prediction, it was correct 88 percent of the time. However, when the prediction was borderline or uncertain, the accuracy dropped to 58 percent. This suggests that the tool could be used to help doctors prioritize their time. If the model says a patient is a clear responder or a clear non-responder, the doctor can trust that assessment. If the model is unsure, the doctor knows to look more closely at the case or rely on other methods. This ability to say "I am not sure" is a crucial feature for medical tools, as it prevents the system from making dangerous guesses when the data is ambiguous.

One of the most interesting findings was how the model handled different types of cancer and treatments. It worked well for most groups, but it struggled slightly with patients receiving a specific type of immunotherapy. In these cases, the usual signs of tumor shrinkage on an MRI did not always match the actual biological response. The model correctly identified that the standard measures of volume change were less reliable for this group, highlighting a known limitation in how we currently measure treatment success for immune-based therapies. This nuance, which a black-box model might have missed or glossed over, was clearly visible in the system's reasoning, showing that it was paying attention to the specific biology of the disease.

The study also addressed the issue of trust by making the entire process transparent. Because the model's reasoning is built on standard medical concepts, a radiologist can look at the same scan and check the model's work. They can see if the system correctly identified the tumor's shape or its growth rate. If the model makes a mistake, the doctor can see exactly which concept was misidentified. This level of detail is something that traditional deep learning models cannot provide, as they often rely on patterns that are invisible to the human eye. By grounding the prediction in concepts that doctors understand, the researchers have created a tool that can be audited and verified, rather than just accepted on faith.

Despite these successes, the researchers are careful to note the limitations of their work. The model was trained on data from specific clinical trials, and while it worked well on an independent set, it has not yet been tested in a real-world hospital setting with all the variability that comes with everyday practice. Additionally, the system relies on algorithmic definitions for some of its measurements, rather than having every single concept checked by a human expert. This means that while the model is interpretable, the ground truth it was trained on is still a computer's best guess rather than a human's final judgment. The researchers also point out that the model predicts short-term outcomes, such as whether the cancer will be gone at the time of surgery, but it does not yet predict long-term survival or the risk of the cancer returning years later.

The work represents a shift in how artificial intelligence is built for medicine. Instead of trying to create a system that is simply the most accurate, regardless of how it works, the researchers prioritized clarity and accountability. They showed that it is possible to build a system that is both accurate and understandable. By forcing the AI to speak the language of radiology and to justify its conclusions with specific, observable features, they have created a bridge between the power of machine learning and the judgment of human clinicians. This approach does not replace the doctor; rather, it gives the doctor a new way to see the data, one that is clear, logical, and open to inspection. In a field where the stakes are incredibly high, this kind of transparency may be the key to finally bringing these powerful tools into the clinic.