Enhancing Brain Tumor Classification Using Vision Transformers with Colormap-Based Feature Representation on BRISC2025 Dataset

1. Problem Statement

Accurate classification of brain tumors from Magnetic Resonance Imaging (MRI) is critical for early diagnosis and treatment planning. While Convolutional Neural Networks (CNNs) like ResNet and EfficientNet have achieved success in medical image analysis, they face specific limitations:

• Local Receptive Fields: CNNs struggle to capture long-range dependencies and global contextual information, which are essential for understanding complex anatomical structures in MRI scans.
• Feature Discriminability: Subtle intensity variations and structural patterns in grayscale MRI scans can be difficult for models to distinguish, leading to potential misclassification.
• Data Dependency: Many deep learning models require extensive data augmentation or large annotated datasets to generalize well, which can be a bottleneck in clinical settings.

The study aims to address these challenges by proposing a framework that combines the global modeling capabilities of Vision Transformers (ViT) with colormap-based feature enhancement to improve multi-class brain tumor classification without relying on heavy data augmentation.

2. Methodology

The proposed framework integrates a pre-trained Vision Transformer with a novel preprocessing pipeline. The workflow consists of the following stages:

A. Data Preparation & Colormap Transformation

• Dataset: The study utilizes the BRISC2025 dataset, containing approximately 6,000 T1-weighted contrast-enhanced MRI scans across four classes: Glioma, Meningioma, Pituitary Tumor, and Non-tumor.
• Preprocessing:
  1. Images are resized to a uniform resolution of 224 × 224.
  2. Pixel intensities are normalized to the range $[0, 1]$.
  3. Colormap Enhancement: A nonlinear mapping function (e.g., Jet colormap) is applied to the normalized grayscale images. This transforms the single-channel intensity data into a pseudo-color representation ($I_{color}$), expanding the intensity distribution into a higher-dimensional color space to highlight subtle structural patterns and intensity variations.

B. Vision Transformer (ViT) Architecture

• Model Backbone: The study employs a pre-trained ViT-Base-Patch16-224 model (pre-trained on ImageNet).
• Patch Embedding: The color-enhanced image is divided into non-overlapping patches of size 16 × 16, resulting in 196 patches.
• Sequence Processing:
  • Patches are flattened and linearly projected into an embedding space.
  • A learnable classification token ($x_{cls}$) is prepended to the sequence.
  • Positional encodings are added to retain spatial information.
• Transformer Encoder: The sequence passes through multiple encoder layers containing Multi-Head Self-Attention (MSA) mechanisms and Feed-Forward Networks (FFN). The self-attention mechanism allows the model to capture global dependencies across the entire MRI scan.
• Classification: The final representation of the $x_{cls}$ token is passed through a Multi-Layer Perceptron (MLP) head to generate probability distributions over the four tumor classes.

C. Training Strategy

• Loss Function: Categorical Cross-Entropy Loss.
• Optimizer: Adam optimizer with a learning rate of $1 \times 10^{-4}$.
• Regularization: The model uses Early Stopping (patience = 15) based on validation accuracy to prevent overfitting.
• Data Augmentation: Notably, the study does not use traditional data augmentation (rotation, flipping, scaling). The authors argue that the combination of pre-trained ViT representations and colormap enhancement provides sufficient discriminative power.

3. Key Contributions

1. Novel Framework: Integration of Vision Transformers with colormap-based feature enhancement specifically for brain tumor classification.
2. Dataset Utilization: Comprehensive evaluation on the BRISC2025 dataset for a four-class classification task (Glioma, Meningioma, Pituitary, Non-tumor).
3. Performance Benchmarking: Demonstration that the proposed method outperforms state-of-the-art CNN baselines (ResNet50, ResNet101, and EfficientNetB2).
4. Interpretability & Generalization: The use of colormap visualization aids in feature discrimination, while the model demonstrates strong generalization without artificial data augmentation.

4. Experimental Results

The model was evaluated on the BRISC2025 dataset using standard metrics (Accuracy, Precision, Recall, F1-score, and AUC).

Performance Comparison:

Model	Accuracy	Precision	Recall	F1-Score	AUC
ResNet50	98.20%	98.36%	98.36%	98.34%	-
ResNet101	98.09%	98.20%	98.26%	98.21%	-
EfficientNetB2	98.37%	98.51%	98.54%	98.52%	-
Proposed ViT	98.90%	98.99%	98.98%	98.98%	99.97%

• Accuracy: The proposed ViT model achieved 98.90%, surpassing the best CNN baseline (EfficientNetB2) by ~0.53%.
• Discriminative Power: The model achieved an exceptional AUC of 99.97%, indicating near-perfect separability between the four classes.
• Confusion Matrix: Analysis showed minimal misclassification across all tumor types, confirming robustness.

5. Significance and Conclusion

The study highlights the potential of Vision Transformers in medical imaging, particularly when combined with domain-specific preprocessing.

• Global Context: The self-attention mechanism successfully captures long-range dependencies in MRI scans, overcoming the locality limitations of CNNs.
• Feature Enhancement: The colormap transformation effectively amplifies subtle intensity variations, making structural patterns more distinguishable for the model.
• Clinical Applicability: The high accuracy and AUC, achieved without complex data augmentation, suggest the model is computationally efficient and robust enough for integration into real-world clinical decision support systems.

Future Work: The authors suggest extending the framework to multi-modal imaging data, exploring lightweight transformer architectures to reduce computational costs, and validating the model on diverse external clinical datasets.