💻 computer science

Swin-TCN-XAI: A Hybrid End-to-End Framework for Explainable Multi-Class Brain Tumor MRI Classification

This paper introduces Swin-TCN-XAI, a hybrid deep learning framework that integrates Swin Transformers and Temporal Convolutional Networks with multi-level explainability techniques to achieve state-of-the-art accuracy and clinical interpretability in multi-class brain tumor MRI classification.

Original authors: Umme Sara, MSTMd. Mamu, MD IRFANUL KABIR HIRA, MD SOHAG HOSSAIN, Md. Kowsar Ahmed, Md. Mamun Ur Rashid

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Umme Sara, MSTMd. Mamu, MD IRFANUL KABIR HIRA, MD SOHAG HOSSAIN, Md. Kowsar Ahmed, Md. Mamun Ur Rashid

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective, the Detective's Notebook, and the Brain's Secret Code

Imagine you are a detective trying to solve a mystery inside a complex, winding city. This city is the human brain, and the mystery is a hidden tumor. To see the city, you don't use a flashlight; you use a special camera called an MRI (Magnetic Resonance Imaging) that takes hundreds of thin, slice-by-slice pictures, like pages in a book. For a long time, doctors had to read every single page of this book by hand, looking for tiny clues that something was wrong. It was tiring work, and sometimes, even the best detectives missed a clue because the pictures were so similar or the tumor looked tricky.

Recently, scientists started teaching computers to be detectives. They built "AI" (Artificial Intelligence) models that could look at these brain pictures and say, "Ah, I see a tumor here!" But there was a catch. These computer detectives were often "black boxes." They would give you an answer, but they couldn't explain why they thought that. It was like a friend guessing the answer to a riddle without showing you their thinking. In medicine, knowing why is just as important as knowing what. If a computer says a patient has a tumor, a doctor needs to see exactly which part of the brain the computer is looking at to trust the answer. This paper introduces a new kind of detective team designed to not only solve the mystery with incredible speed and accuracy but also to show its work, step-by-step, so doctors can verify the clues.

The Team: A Hybrid Detective Squad

The researchers behind this study created a new system they call Swin-TCN-XAI. Think of this system as a two-part detective squad working together to solve the case of brain tumors.

Part 1: The Super-Observer (The Swin Transformer)
First, imagine a very sharp-eyed observer who looks at a single page of the brain's picture book. This observer is the Swin Transformer. Its job is to look at one slice of the brain and find all the tiny details—shapes, textures, and patterns—that might look like a tumor. It's really good at spotting things in a single image, kind of like how you might spot a specific character in a crowded photo.

Part 2: The Storyteller (The TCN)
But a brain isn't just one picture; it's a whole book of pages stacked together. A tumor might look weird on one page but normal on the next, or it might stretch across several pages. This is where the second detective, the Temporal Convolutional Network (TCN), comes in. While the first detective looks at one page, the TCN reads the story of the pages. It looks at the sequence of images, understanding how one slice connects to the next. It's like reading a comic strip instead of just looking at a single panel; it understands the flow and the relationships between the pages.

By teaming up, the Super-Observer finds the details, and the Storyteller understands how those details fit together in 3D space. The paper shows that this team works much better than using just one or the other.

The Big Test: No Overfitting Allowed!

One of the biggest problems with previous computer detectives was that they were sometimes "overfitting" during their training. Imagine if you were studying for a test, and your teacher gave you the exact same questions on the test that you had already practiced with. You would get a perfect score, but you wouldn't actually know the material; you'd just have memorized the answers.

In the world of brain scans, this happens if the computer sees slices from the same patient in both its training and its testing. It might memorize that "Patient A's brain looks like this" rather than learning what a tumor actually looks like.

The authors of this paper were very strict about this. They made sure that if a patient's brain slices were in the training group, none of that patient's slices were allowed in the testing group. It was a "patient-wise" split. This means the computer had to prove it could recognize a tumor on a brand new patient it had never seen before. This is the real test of whether a detective is actually smart or just a memorizer.

The Results: A Near-Perfect Score

When they put their new Swin-TCN-XAI team to the test on this strict, no-overfitting challenge, the results were stunning.

Accuracy: The system got it right 99.41% of the time.
The "No Tumor" Case: It was perfect at spotting healthy brains. It never falsely accused a healthy person of having a tumor (100% precision) and never missed a healthy brain (100% recall).
The Tricky Cases: Even for the hardest type of tumor to spot (glioma), the system was incredibly accurate, with a score of 0.9925 (a measure of how well it balances finding tumors and not making mistakes).

The paper compared their team against five other top-tier detective teams (using different AI methods like LSTMs or standard CNNs). The Swin-TCN team beat them all on every single metric. It wasn't just a little better; it was consistently the best.

The "Explainable" Magic: Showing the Work

The most exciting part of this paper isn't just the high score; it's the XAI (Explainable Artificial Intelligence) part. The authors didn't just want a black box; they wanted a transparent one. They added three different tools to help the computer explain its thinking:

Grad-CAM (The Highlighter): This tool takes the brain image and highlights the exact spots the computer is looking at in bright colors. If the computer says "Tumor," the highlighter shows a glowing red blob right over the tumor. The paper shows that the computer is actually looking at the tumor, not just random noise or the edge of the skull.
SHAP (The Scorecard): This tool breaks down the decision like a scorecard. It tells you which specific parts of the image contributed the most to the final guess. It's like the computer saying, "I said 'Tumor' because of these specific textures here, and these specific shapes there."
LIME (The Local Detective): This tool looks at one single picture and asks, "What if we covered up this tiny part? Would the answer change?" It helps verify that the decision is based on the actual lesion and not a weird artifact in the image.

The paper found that all three tools agreed with each other. They all pointed to the same anatomical locations, confirming that the computer was making its decisions based on real medical clues, not random guesses.

What This Means (And What It Doesn't)

The authors are very clear about what they have achieved and what they haven't. They have proven that this specific hybrid team (Swin + TCN) is highly accurate and trustworthy for classifying four types of brain conditions: Glioma, Meningioma, Pituitary tumors, and "No Tumor." They have shown that looking at the sequence of slices (the story) is better than looking at them one by one.

However, the paper also notes that this is a "slice-by-slice" approach. The computer reads the pages of the book one at a time, even though it understands the order. It hasn't yet learned to look at the whole 3D brain volume all at once, which might be even better for complex shapes. Also, while the system is great at explaining where it looked, the authors admit it doesn't fully explain the causal reason (the deep "why") in a way that replaces a doctor's judgment.

The paper concludes that this framework is a powerful tool for doctors. It's not here to replace the radiologist but to act as a super-smart second opinion that shows its work, helping doctors make faster and more confident decisions. It's a step toward a future where AI doesn't just give answers, but helps us understand the mystery of the brain alongside us.

Technical Summary: Swin-TCN-XAI for Explainable Multi-Class Brain Tumor MRI Classification

Problem Statement
Automated brain tumor classification from Magnetic Resonance Imaging (MRI) faces three primary limitations that hinder real-world clinical utility. First, many existing methods treat individual MRI slices as independent 2D images, ignoring the intrinsic sequential ordering and inter-slice dependencies inherent in volumetric scans. Second, when temporal modeling is attempted, it often relies on recurrent architectures (e.g., LSTMs, GRUs) that suffer from sequential constraints, computational inefficiency, and vanishing gradient issues. Third, and most critically for clinical adoption, high-performing models often operate as "black boxes," lacking the interpretability required for radiologists to verify the anatomical basis of a prediction. Current explainable AI (XAI) approaches frequently provide only single-modality insights (either spatial heatmaps or feature rankings) rather than a comprehensive, multi-level explanation.

Methodology
The paper proposes Swin-TCN-XAI, a hybrid end-to-end deep learning framework designed to jointly learn complementary spatial and sequential feature representations while ensuring clinical transparency.

Architecture: The framework integrates a Swin Transformer (specifically the Swin-Tiny backbone pretrained on ImageNet-1K) with a Temporal Convolutional Network (TCN).
- Spatial Extraction: The Swin Transformer processes input MRI slices (resized to 224×224) using a shifted-window self-attention mechanism. It tokenizes the image into non-overlapping 4×4 patches, generating a hierarchical 7×7 grid of 768-dimensional tokens. This captures both local and global spatial context.
- Temporal Modeling: The flattened token sequence (49 tokens) is fed into the TCN. Unlike recurrent models, the TCN utilizes dilated causal convolutions with residual connections to model ordered inter-token dependencies across the slice sequence. This allows for parallel training and stable gradient flow while capturing long-range sequential relationships.
- Fusion Strategy: The components are jointly optimized via end-to-end fine-tuning using differential learning rates: a conservative rate ( $2 \times 10^{-5}$ ) for the pretrained Swin backbone to preserve learned representations, and a higher rate ( $3 \times 10^{-4}$ ) for the TCN classification head to facilitate rapid domain adaptation.
Explainability (XAI): To address the "black box" issue, the framework incorporates a three-level explainability pipeline:
1. Grad-CAM: Applied to the final TCN convolutional block to generate class-discriminative spatial heatmaps, visualizing lesion localization.
2. SHAP (SHapley Additive exPlanations): Used to quantify the marginal contribution of each Swin token to the classification decision at global and class-specific levels.
3. LIME (Local Interpretable Model-Agnostic Explanations): Provides instance-level superpixel attributions for individual MRI slices, validating decision boundaries.
Evaluation Protocol: The model was evaluated on a four-class dataset (Glioma, Meningioma, Pituitary Tumor, No Tumor) comprising 11,148 slices. Crucially, the study employed a strict patient-wise train-test split (80/20) to prevent data leakage, ensuring that no slices from the same patient appeared in both training and testing sets.

Key Contributions

Hybrid Framework: The proposal of Swin-TCN-XAI, which uniquely combines the hierarchical spatial feature extraction of Swin Transformers with the efficient sequential modeling of TCNs, avoiding the optimization instability of recurrent networks.
Multi-Level Explainability: The integration of Grad-CAM, SHAP, and LIME into a single pipeline, providing spatial, feature-level, and instance-level interpretations simultaneously to support clinical auditing.
Rigorous Validation: The demonstration of high performance under a strict patient-wise evaluation protocol, which offers a more realistic estimate of generalization compared to random slice-level splitting used in prior literature.

Results
The proposed framework achieved state-of-the-art performance on the test set:

Overall Accuracy: 99.41%
Macro-Averaged Metrics: Precision (0.9940), Recall (0.9948), and F1-Score (0.9944).
Class-Specific Performance:
- No Tumor: Perfect precision and recall (1.0000).
- Glioma: F1-Score of 0.9925, addressing the challenge of this invasive tumor type.
- Meningioma & Pituitary: F1-Scores of 0.9897 and 0.9953, respectively.
Comparative Performance: The model outperformed five temporal and hybrid baselines (including CNN+LSTM, Swin+LSTM, and various CNN/Transformer variants) on all metrics. It also demonstrated robust generalization on external datasets (Figshare and Mendeley), maintaining accuracy above 99%.
Ablation Study: The full Swin+TCN architecture outperformed variants using only CNN, only TCN, only Swin, or Swin+LSTM, confirming the efficacy of the specific fusion of hierarchical spatial features with dilated causal temporal modeling.

Significance and Claims
The paper claims that Swin-TCN-XAI provides an accurate, robust, and interpretable approach for automated brain tumor diagnosis. By combining high classification performance with a multi-level explainability strategy, the framework supports trustworthy clinical decision-making. The authors emphasize that the model's ability to focus on anatomically relevant regions (verified by Grad-CAM, SHAP, and LIME) and its rigorous patient-wise validation make it suitable for integration into Computer-Aided Diagnosis (CAD) systems as a second-reader tool. The study concludes that this framework addresses the critical gap between high-performing deep learning models and the transparency required for regulatory and clinical adoption in neuro-oncology.