Comparative Deep Learning Framework for Brain Tumor Detection: Integrating VGG16 Classification and U-Net Segmentation with Performance Evaluation

Abstract

Brain tumors are serious and potentially life-threatening, making early and accurate diagnosis essential for improving survival chances. MRI (Magnetic Resonance Imaging) is a key tool in detecting these tumors without invasive procedures, but analyzing MRI scans manually takes time and can lead to mistakes. In this study, we explore how deep learning—specifically Convolutional Neural Networks (CNNs)—can help automate and improve brain tumor detection using MRI images. We compare the performance of four CNN-based models: EfficientNetB3, VGG16 with fine-tuned transfer learning, ResNet-50, and a custom-built CNN. Our goal is to find out which model performs best in terms of accuracy, precision, recall, and F1-score. After extensive testing, the fine-tuned VGG16 model came out on top as the most accurate and dependable. To build a complete diagnostic system, we combined the VGG16 model with U-Net architecture for tumor segmentation. This approach not only classifies the type of brain tumor—glioma, meningioma, pituitary tumor, or no tumor—but also pinpoints its location within the MRI scan. The segmentation achieved strong results, with an IoU score of 0.87 and a Dice score of 0.91. Overall, our research delivers a comprehensive tool that handles both tumor classification and localization in one streamlined process. The results underscore the power of fine-tuned VGG16 for medical imaging and show how combining it with segmentation can enhance tumor analysis. This solution has the potential to assist radiologists by providing faster, more accurate, and consistent diagnostic support, especially in settings where resources are limited.

Introduction

Brain tumors are critical neurological conditions where early detection and precise classification are essential for effective treatment. MRI is the preferred imaging technique, but manual analysis is time-consuming and error-prone. Recent advances in Artificial Intelligence (AI), especially Deep Learning (DL) with Convolutional Neural Networks (CNNs), have significantly improved automated brain tumor detection and classification from MRI scans.

This study compares four CNN architectures—VGG16 (fine-tuned), EfficientNetB3, ResNet-50, and a custom CNN—using a publicly available brain MRI dataset. VGG16 showed superior performance in classification accuracy, precision, recall, and F1-score. Building on this, the study integrates VGG16 with a U-Net segmentation model to not only classify tumor types but also localize tumor regions in MRI scans, providing a comprehensive diagnostic tool.

The literature review highlights the strengths of CNNs in medical imaging, particularly VGG16’s efficiency with transfer learning, ResNet-50’s deep residual learning, EfficientNet’s scaling approach, and U-Net’s robust segmentation capability. Combining classification and segmentation in one pipeline is a novel approach aimed at enhancing clinical utility.

The methodology involved preprocessing MRI images, training the four classifiers, and implementing U-Net for segmentation. VGG16 achieved the best classification results (~97% accuracy), while the U-Net model attained high segmentation accuracy with Dice scores around 0.91. Integrating these models allows tumor type prediction alongside precise tumor boundary visualization.

Future work suggests incorporating multi-modal MRI, 3D segmentation, real-time inference on edge devices, explainable AI techniques, semi-supervised learning, clinical workflow integration, and personalized treatment prediction to improve robustness and clinical applicability.

References

[1] Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” *Advances in Neural Information Processing Systems*, vol. 25, pp. 1097–1105, 2012. [2] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” *International Conference on Learning Representations (ICLR)*, 2015. [3] S. Khan et al., “Brain Tumor Classification Using Transfer Learning and Fine-Tuned VGG16,” *Journal of Medical Imaging*, 2021. [4] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2016, pp. 770–778. [5] M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in *Proceedings of the 36th International Conference on Machine Learning (ICML)*, 2019, pp. 6105–6114. [6] A. Esteva et al., “Dermatologist-level classification of skin cancer with deep neural networks,” *Nature*, vol. 542, no. 7639, pp. 115–118, 2017. [7] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in *International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)*, 2015, pp. 234–241. [8] Z. Zhang, Q. Liu, and Y. Wang, “Road extraction by deep residual U-Net,” *IEEE Geoscience and Remote Sensing Letters*, vol. 15, no. 5, pp. 749–753, 2018. [9] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: A nested U-Net architecture for medical image segmentation,” in *Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support*, 2018, pp. 3–11.

Copyright

Copyright © 2025 Sanskriti Jhanwar, Saumya Suman. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.