NeuroCAM-X: An Explainable Hybrid AI Framework for Brain Tumor Classification Using MRI and Clinical Reports with Advanced Tumor Analytics

Abstract

Brain tumor classification from Magnetic Resonance Imaging (MRI) is a critical task in medical diagnostics that demands both high accuracy and clinical interpretability. This research presents NeuroCAM-X, a novel explainable hybrid artificial intelligence framework that integrates deep learning-based MRI image analysis with Optical Character Recognition (OCR)-enabled clinical report interpretation for comprehensive brain tumor diagnosis. The system employs an EfficientNet-B0 architecture achieving 97.0% classification accuracy on a dataset of 7,023 MRI images. Unlike conventional approaches that rely solely on imaging data, NeuroCAM-X implements a hybrid decision engine that cross-validates MRI predictions with OCR-extracted clinical findings, achieving an 87.5% agreement rate for diagnostic consistency. The framework incorporates multiple Explainable AI (XAI) techniques—Grad-CAM, SHAP, and LIME—to provide complementary visual interpretations of model predictions with 94.3% alignment to expert-identified tumor regions. In addition, the system includes automated tumor analytics for quantitative assessment and staging to support clinical decision-making. A production-ready web application provides patient management, interactive diagnostic visualization, and automated report generation. Preliminary clinical evaluation demonstrated high physician trust (4.2/5.0) and satisfaction (4.4/5.0), indicating the framework\'s potential for clinical deployment. This work addresses critical gaps in medical AI by combining accurate classification, multimodal data integration, explainable AI, quantitative analytics, and clinical decision support within a unified framework suitable for real-world healthcare applications.

Introduction

The paper introduces NeuroCAM-X, an explainable multimodal AI system designed to improve brain tumor diagnosis from MRI images and clinical reports. The motivation is that existing deep-learning systems often achieve high accuracy but lack interpretability, ignore complementary clinical information, and provide limited quantitative tumor characterization.

Main problem

Brain tumors cause substantial global mortality, and MRI is the preferred imaging method because of its strong soft-tissue contrast. Although CNNs have achieved high classification accuracy, current systems have three major weaknesses:

• Limited explainability and clinical transparency
• Reliance mainly on MRI images, without integrating clinical reports
• Limited quantitative tumor measurements for standardized reporting

Proposed NeuroCAM-X framework

The system combines six major components:

1. MRI classification: EfficientNet-B0 classifies MRI scans into glioma, meningioma, pituitary tumor, or no tumor.
2. Clinical report analysis: OCR extracts text from scanned/PDF reports, followed by NLP-based identification of relevant biomarkers and diagnostic information.
3. Hybrid decision fusion: MRI predictions and report-based predictions are compared and classified as Match, Partial Match, or Mismatch.
4. Explainable AI:
   • Grad-CAM identifies important regions in MRI images.
   • SHAP measures feature contributions.
   • LIME identifies locally important image regions.
5. Tumor analytics: Grad-CAM heatmaps are used to estimate tumor area, diameter, occupancy, localization, staging, volume, and 3D distribution without requiring a separately trained segmentation model.
6. Clinical web application: A Flask-based application provides authentication, patient history, dashboards, automated PDF reports, and APIs for potential hospital-system integration.

Dataset and model

The study uses 7,023 MRI images across four classes:

• Glioma: 1,621
• Meningioma: 1,645
• No tumor: 2,000
• Pituitary: 1,757

The data were divided into 70% training, 15% validation, and 15% testing. Images were resized to 224×224 pixels and augmented. EfficientNet-B0 was fine-tuned using transfer learning.

Key results

Classification

• Overall accuracy: 97.0%
• Glioma: 96.9%
• Meningioma: 96.9%
• No tumor: 98.3%
• Pituitary: 96.0%

The model showed particularly strong performance in distinguishing tumor from no-tumor cases.

Explainability

• Grad-CAM showed 94.3% alignment with tumor regions identified by radiologists.
• LIME and SHAP explanations strongly agreed with Grad-CAM (Spearman ρ = 0.89, p < 0.001).
• This suggests that the model's decisions are focused on clinically meaningful regions.

Tumor analytics
Automated measurements showed strong agreement with expert measurements:

• Tumor area correlation: r = 0.912
• Maximum diameter: r = 0.895
• Tumor occupancy: r = 0.924

Multimodal validation
Among 200 cases containing both MRI images and pathology reports:

• 87.5% had matching or partially matching predictions.
• 12.5% were mismatches.
• The system identified 17 genuine diagnostic inconsistencies, potentially helping physicians detect errors or problematic cases.

Efficiency

• Full pipeline with all XAI methods: 7.1 ± 0.8 seconds/case
• Grad-CAM-only configuration: 3.6 ± 0.4 seconds/case
• Parallel XAI reduced processing time by 42%.
• GPU throughput: approximately 8–10 complete cases/minute, or 16–18 cases/minute with Grad-CAM only.

Clinical evaluation

Five experienced radiologists evaluated the system. Average scores out of 5 were:

• Trust in predictions: 4.2
• Explanation quality: 4.4
• System usability: 4.6
• Clinical utility: 4.3

These results indicate strong acceptance and perceived usefulness among the participating physicians.

Overall contribution

NeuroCAM-X attempts to move brain-tumor AI beyond image-only classification by combining MRI analysis, clinical-report interpretation, multimodal consistency checking, explainable AI, tumor quantification, and clinical workflow integration.

According to the reported experiments, its 97% accuracy exceeds the cited comparison systems, while its main novelty is not accuracy alone but the combination of multimodal diagnosis + triple explainability (Grad-CAM, SHAP, LIME) + automated tumor analytics + clinical deployment in a single framework.

Conclusion

NeuroCAM-X presents a comprehensive explainable hybrid AI framework for brain tumor classification by integrating MRI image analysis, OCR-based clinical report interpretation, Explainable AI, and quantitative tumor analytics into a unified clinical decision support system. The proposed framework achieved 97.0% classification accuracy while providing interpretable predictions through Grad-CAM, SHAP, and LIME, demonstrating that high diagnostic performance and model transparency can be achieved simultaneously. The hybrid decision engine combines MRI predictions with clinical report analysis to improve diagnostic confidence and identify inconsistencies requiring physician review. In addition, the proposed heatmap-based tumor analytics estimate clinically relevant metrics, including tumor area, diameter, occupancy, stage, and volume, without requiring segmentation annotations, making the framework practical for real-world clinical deployment. Clinical evaluation demonstrated high physician acceptance in terms of trust, explanation quality, usability, and clinical utility, indicating the potential of NeuroCAM-X as an effective decision support tool for radiologists. The production-ready web application further supports patient management, visualization of Explainable AI outputs, automated report generation, and secure clinical workflow integration. Overall, NeuroCAM-X demonstrates that explainable, multimodal AI can enhance brain tumor diagnosis by improving prediction accuracy, transparency, and clinical usability. Future work will focus on extending the framework to 3D volumetric MRI analysis, multi-sequence imaging, advanced NLP-based clinical report understanding, federated multi-institutional learning, and large-scale clinical validation to further improve diagnostic performance and support broader clinical adoption.

References

[1] R. Manzari et al., \"BEFUnet: Hybrid CNN-Transformer Architecture for Medical Image Segmentation,\" arXiv preprint arXiv:2402.xxxxx, 2024. [2] Q. U. A. Chaudhary, S. A. Qureshi, T. Sadiq, A. Usman, A. Khawar, S. T. H. Shah, and A. U. Rehman, \"SAlexNet: Superimposed AlexNet Using Residual Attention Mechanism for Accurate and Efficient Automatic Primary Brain Tumor Detection and Classification,\" Results in Engineering, vol. 25, pp. 104025, 2025. [3] K. He, X. Zhang, S. Ren, and J. Sun, \"Deep Residual Learning for Image Recognition,\" in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778. [4] A. Krizhevsky, I. Sutskever, and G. E. Hinton, \"ImageNet Classification with Deep Convolutional Neural Networks,\" in Advances in Neural Information Processing Systems (NIPS), vol. 25, 2012. [5] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, \"Densely Connected Convolutional Networks,\" in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4700-4708. [6] M. Tan and Q. V. Le, \"EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,\" in Proc. International Conference on Machine Learning (ICML), 2019, pp. 6105-6114. [7] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, \"Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,\" in Proc. IEEE International Conference on Computer Vision (ICCV), 2017, pp. 618-626. [8] S. M. Lundberg and S. I. Lee, \"A Unified Approach to Interpreting Model Predictions,\" in Advances in Neural Information Processing Systems (NIPS), 2017, pp. 4765-4774. [9] M. T. Ribeiro, S. Singh, and C. Guestrin, \"Why Should I Trust You? Explaining the Predictions of Any Classifier,\" in Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135-1144. [10] S. Pereira, A. Pinto, V. Alves, and C. A. Silva, \"Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images,\" IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1240-1251, May 2016. [11] M. Havaei et al., \"Brain Tumor Segmentation with Deep Neural Networks,\" Medical Image Analysis, vol. 35, pp. 18-31, Jan. 2017. [12] B. H. Menze et al., \"The Multimodal Brain Tumor Image Segmentation Benchmark (BraTS),\" IEEE Transactions on Medical Imaging, vol. 34, no. 10, pp. 1993-2024, Oct. 2015. [13] G. Litjens et al., \"A Survey on Deep Learning in Medical Image Analysis,\" Medical Image Analysis, vol. 42, pp. 60-88, Dec. 2017. [14] D. Shen, G. Wu, and H. I. Suk, \"Deep Learning in Medical Image Analysis,\" Annual Review of Biomedical Engineering, vol. 19, pp. 221-248, June 2017. [15] J. Ker, L. Wang, J. Rao, and T. Lim, \"Deep Learning Applications in Medical Image Analysis,\" IEEE Access, vol. 6, pp. 9375-9389, 2018. [16] CBTRUS, \"Statistical Report: Primary Brain and Other Central Nervous System Tumors Diagnosed in the United States in 2016-2020,\" Central Brain Tumor Registry of the United States, Hinsdale, IL, Rep., 2024. [17] J. Zhou et al., \"Review of Artificial Intelligence in Brain Tumor Diagnosis and Treatment,\" Frontiers in Oncology, vol. 13, pp. 1239725, 2023. [18] M. Tandel, A. Balestrieri, T. Jujaray, V. Khanna, and L. Saba, \"Multiclass Brain Tumor Classification Using Deep Learning Techniques: A Comprehensive Review,\" Computers in Biology and Medicine, vol. 152, pp. 106362, Jan. 2023. [19] S. Ahmed, M. H. Yap, and J. Tan, \"Explainable Artificial Intelligence for Medical Imaging: A Comprehensive Survey,\" Artificial Intelligence Review, vol. 57, no. 1, pp. 1-45, Jan. 2024. [20] R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, \"Convolutional Neural Networks: An Overview and Application in Radiology,\" Insights into Imaging, vol. 9, pp. 611-629, Aug. 2018. [21] N. Hosny and M. Mohammed, \"Recent Advances in Deep Learning and Explainable AI for Brain Tumor Classification: A Comprehensive Review,\" Neural Computing and Applications, vol. 37, no. 2, pp. 891-920, Jan. 2025. [22] V. Vamsidhar et al., \"Vision Transformer-Based Explainable Framework for Brain Tumor Classification Using MRI Images,\" Expert Systems with Applications, vol. 238, pp. 121847, March 2025.

Copyright

Copyright © 2026 Priyanka Kalyanrao Tawale, Amol Wakhare, Vaibhav Tawale. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.