Adaptive Self-Attention for Enhanced Segmentation of Adult Gliomas in Multi-Modal MRI

Every year there are an estimated 80,000–90,000 new glioma cases, highlighting the need for reliable imaging-based decision support. Although deep learning has improved tumor sub-region segmentation, many state-of-the-art models fail to fully capture complementary information across T1, T1Gd, T2, and FLAIR MRI modalities and often operate as “black boxes,” limiting physician trust when precise delineation is critical for surgical planning, radiation targeting, and treatment monitoring. To address these limitations, we propose AIMS, an Adaptive Integrated Multi-Modal Segmentation framework that maintains modality-specific feature streams and employs adaptive self-attention within a hierarchical CNN-Transformer architecture to prioritize and fuse multi-modal MRI features. We evaluated AIMS on the BraTS 2019 adult glioma dataset using five-fold cross-validation and compared it against strong hybrid baselines with paired statistical testing; generalization was assessed on an independent BraTS 2021 cohort without fine-tuning. AIMS achieved high ensemble Dice Similarity Coefficients of 0.936 for enhancing tumor, 0.942 for tumor core, and 0.931 for whole tumor on BraTS 2019, with statistically significant improvements over competing methods, and maintained strong performance on BraTS 2021 despite protocol and scanner variability. Finally, Grad-CAM-based explanations applied to adaptive attention and fusion layers, together with quantitative sanity checks, provided modality-aware and spatially meaningful visualizations that support clinical interpretation. By improving both segmentation accuracy and model transparency relative to strong baselines, AIMS advances multi-modal glioma segmentation and helps bridge human–machine teaming by enabling faster, clinician-aligned tumor delineation without sacrificing reliability.