Smart Neuro-Oncology Assistant: A Chatbot for Brain Tumour Detection and Primary Cancer Prediction

Abstract

The Smart Neuro-Oncology Assistant introduces a novel AI-powered platform that combines deep learning-based brain tumor detection with primary cancer prediction capabilities, particularly for tumors associated with Glioma and Pituitary regions. This interactive system supports patients and healthcare professionals throughout the neuro-oncological assessment process. Utilizing the Xception convolutional neural network architecture, the system achieves 96% overall accuracy in classifying brain MRI scans into four categories: Glioma, Meningioma, Pituitary Tumor, and No Tumor. The platform extends beyond basic classification by evaluating cancer likelihood and delivering personalized guidance through an interactive chatbot powered by Google\'s Gemini 1.5 Flash generative AI model. Key features include secure user authentication, robust image validation, specialist recommendations based on detected tumor types, and comprehensive historical data tracking for longitudinal assessment. Developed using Streamlit for the frontend interface, Flask for backend services, and MongoDB for efficient data management, the system demonstrates exceptional performance with class-specific F1-scores ranging from 0.93 to 0.98. This research contributes to AI-assisted medical diagnostics by promoting a human-centered approach that supports clinical expertise, aiming to streamline diagnostics and enhance patient understanding in neuro-oncology.

Introduction

Overview:

The project introduces an AI-powered medical assistant that combines MRI-based brain tumor classification with a chatbot interface for primary cancer prediction and interactive healthcare support. The system identifies brain tumor types, predicts cancer status, and assists patients with guidance through a conversational interface.

Key Features:

• Tumor Classification: Deep learning model classifies MRI scans into:

  • Glioma (primary cancerous)

  • Meningioma (non-cancerous)

  • Pituitary Tumor (potentially cancerous)

  • No Tumor

• Cancer Prediction Logic:

  • Flags Glioma as cancerous.

  • Refers Pituitary Tumors for further evaluation.

  • Identifies Meningioma and No Tumor as non-cancerous in this system’s context.

• Conversational Interface:

  • Built-in chatbot powered by Google's Gemini 1.5 Flash.

  • Offers treatment suggestions, explanations, doctor recommendations, and historical MRI comparisons.

Architecture & Technologies:

• Frontend: Developed using Streamlit

  • MRI image upload and validation (ensures grayscale input)

  • Chat interface with natural language capabilities

  • Result visualization and historical data access

  • Responsive and patient-friendly UI

• Backend: Built with Flask

  • Handles API services, inference, chat management, and security

  • Uses MongoDB to store user profiles, chat history, and MRI results

• Security:

  • Passwords hashed with Werkzeug

  • Session-based authentication

  • Input sanitization and API-level access control

  • Validates uploaded MRI scans (e.g., grayscale check)

Deep Learning Model:

• Model Used: Xception architecture with transfer learning (ImageNet pre-trained)

  • Fine-tuned final 30 layers

  • Classification head with dropout, batch normalization, and L1/L2 regularization

• Input: 299×299 grayscale MRI scans

• Augmentation Techniques:

  • Standard: Rotation, flipping, zoom, shift

  • Medical-specific: Elastic deformations, brightness/contrast tuning

• Training:

  • Loss: Categorical cross-entropy

  • Optimizer: Adam (learning rate: 2e-5)

  • Techniques: Early stopping, learning rate decay, model checkpointing

  • Class balancing using inverse frequency weighting

• Performance:

  • Accuracy: 96%

  • F1 Score: 0.96 (macro average)

  • Class-wise Precision & Recall:

    • Pituitary: 97% / 99%

    • No Tumor: 97% / 99%

    • Glioma: 94% / 97%

    • Meningioma: 96% / 90%

Literature Review Insights:

• CNNs: Effective for brain tumor classification (e.g., Mahmud et al. used VGG16 achieving ~98% accuracy)

• Ensemble & Transformers: EfficientNetV2 + ViT ensemble improved accuracy (up to 96%)

• Medical Chatbots:

  • GPT-3.5 chatbot: 97.43% response accuracy

  • LLaMA3-based assistant: 88% diagnostic accuracy, image & text support

  • Multilingual chatbot in Hungarian achieved over 90% accuracy

NLP Integration:

• Model: Gemini 1.5 Flash

  • Capable of contextual medical dialogue

  • Translates clinical terms for patient understanding

• Prompt Engineering:

  • Maintains conversational flow

  • Embeds prior interactions and ensures medically accurate explanations

Conclusion

This study presents the successful design and deployment of the Smart Neuro-Oncology Assistant, a web-based diagnostic tool driven by a deep learning model built on the Xception architecture. The model achieved an overall accuracy of 96.5% on the test dataset, reflecting its robustness in brain tumor classification. Class-specific F1-scores further validate its effectiveness, with values of 0.95 for Glioma, 0.93 for Meningioma, and 0.98 for both No Tumor and Pituitary categories. These results underscore the model’s strong capability in accurately distinguishing between complex tumor types in MRI scans, reinforcing its potential as a clinical decision-support tool in neuro-oncology. The integration of this model into a comprehensive web application with user authentication, chat support, and specialist recommendations represents a significant step toward practical AI application in neuro-oncology. By maintaining a human-centered design approach that emphasizes augmenting rather than replacing clinical expertise, the system addresses real-world needs while acknowledging the limitations inherent in automated medical analysis. Future work should focus on several key areas to advance the capabilities and clinical readiness of the Smart Neuro-Oncology Assistant. First, validating the model’s performance across diverse, multi-institutional datasets will be critical to ensure robustness across different imaging protocols and patient populations. Second, integrating explainable AI techniques—such as visual saliency maps—can provide clinicians with greater transparency into the decision-making process of the model, thereby increasing trust in its outputs. Third, conducting prospective clinical trials will be essential to evaluate the system\'s real-world impact on diagnostic workflows, particularly its ability to improve the timeliness and accuracy of neuro-oncological assessments when used as a decision support tool. Additional enhancements are also planned to improve user experience and long-term engagement. The chat history section will be integrated directly into the chatbot interface, allowing users to seamlessly continue conversations from previous sessions, thus offering a more personalized and coherent interaction flow. Moreover, the MRI results section will support side-by-side comparisons of prior and current diagnostic results, enabling users and clinicians to observe tumor progression or regression over time. These improvements aim to transform the platform into a comprehensive, patient-friendly, and clinically relevant assistant capable of supporting both non-expert users and medical professionals. The promising results achieved in this research highlight the potential for deep learning-based assistive technologies to enhance neuro-oncological care workflows. By providing rapid preliminary assessments while maintaining appropriate pathways to specialist care, such systems could contribute to earlier detection and intervention for patients with brain tumors.

References

[1] T. Mahmud et al., “Enhancing Diagnosis: An Ensemble Deep Learning Model for Brain Tumor Detection and Classification,” ResearchGate, Mar. 2024. [Online]. Available: https://www.researchgate.net/publication/379431047 [2] A. Tariq, M. M. Iqbal, M. J. Iqbal, and I. Ahmad, “Transforming Brain Tumor Detection Empowering Multi-Class Classification with Vision Transformers and EfficientNetV2,” Unpublished manuscript, 2024. [3] Z. R. Abdulqader and A. M. Abdulazeez, “Deep and Machine Learning Algorithms for Diagnosing Brain Cancer and Tumors,” ResearchGate, Jun. 2024. [Online]. Available: https://www.researchgate.net/publication/382553718 [4] D. S. Wankhede et al., “Brain Tumor Detection and Classification Using Adjusted InceptionV3, AlexNet, VGG16, VGG19 with ResNet50-152 CNN Model,” Unpublished manuscript, Jun. 2024. [5] S. Ray et al., “An Advanced and Ideal Method for Tumor Detection and Classification from MRI Image Using Gaussian mixture and RCNN,” in Proc. Global AI Summit, 2024, doi: 10.1109/GLOBALAISUMMIT62156.2024.10947836. [6] R. Benni et al., “Evaluating The Impact of Activation Functions on CNN Performance in Brain Tumor Detection,” in Proc. 2024 Asian Conf. on Intelligent Technologies (ACOIT), 2024, doi: 10.1109/ACOIT62457.2024.10941380 [7] A. S. Akilesh et al., “A Novel AI-based Chatbot Application for Personalized Medical Diagnosis and Review Using Large Language Models,” in Proc. RMKMATE, 2023, doi: 10.1109/RMKMATE59243.2023.10368616. [8] A. Patel, R. Shivani, and M. P. B. Manjunatha, “Developing a Virtual Diagnosis and Health Assistant Chatbot Leveraging LLaMA3,” in Proc. CSITSS, 2024, doi: 10.1109/CSITSS64042.2024.10817034. [9] B. Simon et al., “Advancing Medical Assistance: Developing an Effective Hungarian-Language Medical Chatbot with Artificial Intelligence,” Information, vol. 15, no. 6, May 2024. doi: 10.3390/info15060297. [10] A. J. V. M. Mohan et al., “Medical Chatbot,” in Proc. ICNEWS, 2024, doi: 10.1109/ICNEWS60873.2024.10730884. [11] A. A. Asiri et al., “Block-Wise Neural Network for Brain Tumor Identification in Magnetic Resonance Images,” Computational Materials Continuum, vol. 73, pp. 5735–5753, 2022. [12] A. A. Asiri et al., “Multi-Level Deep Generative Adversarial Networks for Brain Tumor Classification on Magnetic Resonance Images,” Intelligent Automation & Soft Computing, vol. 36, pp. 127–143, 2023. [13] A. A. Asiri et al., “A Novel Inherited Modeling Structure of Automatic Brain Tumor Segmentation from MRI,” Computational Materials Continuum, vol. 73, pp. 3983–4002, 2022. [14] C. Balaji and S. Veni, “Automatic Skull Stripping from MRI of Human Brain using Deep Learning Framework for the Diagnosis of Brain Related Diseases,” International Journal of Intelligent Systems and Applications in Engineering, vol. 11, no. 4, pp. 439–445, 2023. [15] M. Sooda, S. Jain, and J. Dogra, “Classification and Pathologic Diagnosis of Gliomas in MR Brain Images,” Procedia Computer Science, vol. 218, pp. 706–717, 2023. [16] S. Seyedhashemi and M. Esmaeili, “Detecting Tumors in MRI Scans using a Convolutional Neural Network,” Authorea, Feb. 22, 2023. [17] C. Srinivas et al., “Deep Transfer Learning Approaches in Performance Analysis of Brain Tumor Classification Using MRI Images,” in Wearable Devices for Smart Healthcare, 2022. [18] D. S. Wankhede and C. J. Shelke, “An Investigative Approach on the Prediction of Isocitrate Dehydrogenase (IDH1) Mutations and Co-deletion of 1p19q in Glioma Brain Tumors,” in Intelligent Systems Design and Applications (ISDA 2022), Lecture Notes in Networks and Systems, vol. 715, 2023 [19] J. Cheng, “Brain Tumor Dataset,” Figshare, 2017. [Online]. Available: https://doi.org/10.6084/m9.figshare.1512427.v5 [20] Sartaj, “Brain Tumor Classification Dataset,” Kaggle, 2023. [Online]. Available: https://www.kaggle.com/datasets/sartajbhuvaji/brain-tumor-classification-mri [21] M. Nickparvar, “Brain Tumor Classification Dataset,” Kaggle, 2023. [Online]. Available: https://www.kaggle.com/datasets/masoudnickparvar/brain-tumor-mri-dataset [22] S. C. Chen et al., “RNA editing-based classification of diffuse Gliomas: predicting isocitrate dehydrogenase mutation and chromosome 1p/19q codeletion,” BMC Bioinformatics, vol. 20, Suppl 19, p. 659, 2019. [Online]. Available: https://doi.org/10.1186/s12859-019-3236-0 [23] S. R. van der Voort et al., “Predicting the 1p/19q Codeletion Status of Presumed Low-Grade Glioma with a Validated ML Algorithm,” Clinical Cancer Research, vol. 25, no. 24, pp. 7455–7462, 2019. [Online]. Available: https://doi.org/10.1158/1078-0432.CCR-19-1127

Copyright

Copyright © 2025 Kartik Deogire , Dhanashree Patil, Sahil Dhake, Shreevallabh Chidrawar, Omkar Jagtap. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.