BrainFusionNet: a deep learning and XAI model to understand local, global, and sequential features of MRI images for improved brain tumour detection

The Big Picture: A Detective Team for Brain Scans

Imagine you are trying to find a tiny, hidden treasure (a brain tumor) inside a very noisy, foggy room (an MRI scan). Sometimes the fog is so thick you can't see the edges of the treasure. Sometimes the treasure looks exactly like the furniture around it.

For a long time, computers (Deep Learning) have tried to solve this puzzle. But they often get confused because:

1. The noise hides the edges.
2. The complexity makes it hard to tell the difference between healthy tissue and sick tissue.
3. The "forgetting" problem: As the computer looks deeper into the image to find details, it sometimes forgets the big picture, or vice versa.

The authors of this paper built a new tool called BrainFusionNet. Think of it not as a single detective, but as a specialized three-person detective team working together to solve the case.

The Three-Person Detective Team

Instead of using just one type of computer brain, this model combines three different "brains" (algorithms) to look at the MRI from different angles:

1. The "Local" Detective (CNN):

   • What it does: This detective uses a magnifying glass. It looks at tiny, specific spots in the image to find small details, textures, and edges.
   • The Analogy: Imagine looking at a painting and focusing on the individual brushstrokes to see if the paint is cracked. This part of the model is great at spotting the "grain" of the tumor.

2. The "Global" Detective (Vision Transformer or ViT):

   • What it does: This detective steps back and looks at the whole room at once. It connects the dots between different parts of the image to understand the context.
   • The Analogy: Imagine stepping back from the painting to see the whole picture. It helps the model understand how the tumor relates to the rest of the brain, ensuring it doesn't get lost in the tiny details. It acts like a "self-attention" mechanism, deciding which parts of the image are important to focus on.

3. The "Storyteller" (GRU):

   • What it does: This detective takes the notes from the Local and Global detectives and puts them in order. Even though an MRI is a static picture, the features inside it have a sequence (like a story).
   • The Analogy: Imagine a detective who listens to the first two witnesses and then says, "Okay, based on what they said in order, here is the conclusion." This part helps the model understand the relationship between the local details and the global picture before making a final decision.

The Result: By fusing these three approaches, the model can see the small details and the big picture simultaneously, leading to a much more accurate diagnosis.

The "Black Box" Problem: Making the Computer Explain Itself

Usually, when a computer says, "This is a tumor," doctors can't ask, "Why?" The computer just gives an answer, which is like a "black box."

To fix this, the authors added Explainable AI (XAI) tools. They used three different "flashlights" (LIME, SHAP, and Grad-CAM) to shine on the MRI image and show exactly where the computer was looking when it made its decision.

• Green highlights: "This part of the image made me think it's a tumor."
• Red highlights: "This part made me think it's not a tumor."

This allows doctors to verify if the computer is looking at the actual tumor or just guessing based on a random shadow.

The Experiment: How Well Did It Work?

The team tested their new "Three-Person Team" against six other famous computer models (like VGG16 and ResNet) using two different sets of brain scan data.

• The Score: BrainFusionNet got 98% accuracy.
• The Competition: The best of the other models got around 96%.
• The Secret Sauce: The paper found that the new model was especially good at finding small tumors that other models missed. It also handled the "noise" in the images better.

A Surprising Discovery: The Brightness of the Image

The researchers did something unique: they analyzed the pixel intensity (how bright or dark the pixels are) in the images.

• The Finding: They discovered that images with higher brightness (higher pixel intensity) were much easier for the computer to classify correctly.
• The Analogy: It's like trying to read a book. If the lights are bright (high intensity), you can read the words easily. If the lights are dim (low intensity), you might misread a word. The computer struggles more when the "lighting" in the MRI is poor or inconsistent.

What the Paper Doesn't Say

It is important to stick to what the paper actually claims:

• It is not a cure: The paper is about detecting and classifying tumors, not treating them.
• It is not a hospital tool yet: While the authors mention they built a prototype app (a web interface and an Android app) to show how it could work, the paper focuses on the research and testing of the model itself, not its deployment in real hospitals.
• It is limited to MRI: The model was tested only on MRI scans. The authors admit they haven't tested it on CT scans, X-rays, or ultrasound images yet.
• Transfer Learning failed: The paper notes that when they tried to use "Transfer Learning" (taking a model trained on general photos and applying it to brain scans), it actually performed worse than training from scratch. This suggests that brain scans are so unique that general knowledge doesn't help much here.

Summary

BrainFusionNet is a new, hybrid computer model that combines three different ways of looking at brain scans (tiny details, the big picture, and the sequence of features) to find tumors. It is more accurate than current top models, especially for small tumors, and it includes a "flashlight" feature that shows doctors exactly why it made its decision. The study also revealed that the brightness of the MRI image itself plays a huge role in how well the computer can see the tumor.