VGG16 and ResNet50 for Brain Tumor Classification: Comparative Performance and Grad-CAM++ Analysis

Abstract

Reliable categorization of brain tumors from magnetic resonance imaging (MRI) is a prerequisite for timely clinical intervention, yet the great majority of transfer-learning studies in this space report a single backbone in isolation, leaving open the question of how much of the reported performance is attributable to the chosen architecture rather than to dataset or training choices. This paper presents a controlled comparative evaluation of two ImageNet-pretrained convolutional backbones, VGG16 and ResNet50, for four-class brain tumor classification (glioma, meningioma, pituitary tumor, no tumor) on a merged corpus of 7,200 MRI slices drawn from the Figshare, SARTAJ, and Br35H repositories. Both backbones are fitted with an identical classification head and trained under an identical two-phase (frozen, then partially fine-tuned) protocol, so that any accuracy differential can be attributed to the backbone rather than to confounding methodological variation. On a held-out test set of 1,600 images, VGG16 attains 94.31% accuracy with a weighted F1-score of 94.20%, while ResNet50 attains 94.00% accuracy with a weighted F1-score of 93.89%; VGG16 is accordingly identified as the better-performing architecture under the present experimental conditions. To move beyond an opaque class label, Grad-CAM++ is applied to the selected model to generate a visual attribution map for each classification decision, highlighting the image regions that most strongly influenced the predicted tumor class. The resulting comparative and interpretability findings are intended as a controlled, architecture-isolated reference point for subsequent work in this line of research rather than as a claim of universally superior performance. Brain Tumor Classification, VGG16, ResNet50, Transfer Learning, Grad-CAM++, Explainable AI, MRI.

Introduction

The text presents a controlled comparison of VGG16 and ResNet50 for four-class brain tumor classification from MRI images, combined with Grad-CAM++ explainability. The main goal is to determine which pretrained CNN backbone performs better when all other experimental conditions are kept the same.

Background

Brain tumors such as glioma, meningioma, and pituitary tumors can have overlapping appearances in MRI scans, making accurate diagnosis challenging. Although MRI provides excellent soft-tissue contrast, manual interpretation is time-consuming and depends heavily on specialist expertise.

Deep-learning models, particularly pretrained CNNs such as VGG16 and ResNet50, are widely used for medical-image classification. However, comparisons between these architectures across different studies are difficult because researchers often use different datasets, preprocessing methods, training schedules, and classification heads.

This study addresses this problem by performing a direct, architecture-isolated comparison of VGG16 and ResNet50 under identical experimental conditions.

Main Objectives and Contributions

The study aims to:

• Compare VGG16 and ResNet50 for four-class brain tumor MRI classification.
• Keep the dataset, preprocessing, augmentation, classification head, and training strategy the same for both models.
• Identify the better-performing architecture through accuracy and detailed error analysis.
• Apply Grad-CAM++ to explain the predictions of the selected model visually.

The study identifies VGG16 as the better-performing model under the experimental conditions used.

Literature Review

Previous studies have reported strong results using VGG16, ResNet-based models, Vision Transformers, EfficientNet, CNNs, and other machine-learning approaches for brain tumor classification.

However, the literature has several limitations:

• Many studies evaluate only a single architecture.
• Different studies use different datasets and experimental settings, making direct comparison difficult.
• Fine-tuning strategies vary considerably.
• Explainability is often treated as an optional visualization rather than an integrated part of model evaluation.
• There is limited systematic comparison of VGG16 versus ResNet50 under matched conditions.

The study therefore attempts to provide a more reliable comparison by controlling these variables.

Dataset

The research combines three publicly available MRI datasets:

• Figshare
• SARTAJ
• Br35H

The resulting dataset contains 7,200 MRI images divided equally among four classes:

Twenty percent of the training data, or 1,120 images, was further used for validation, leaving 4,480 images for gradient updates.

Because both models use exactly the same dataset split, the comparison is designed to isolate the effect of the CNN architecture.

Image Preprocessing

All MRI images are resized to 224 × 224 pixels, matching the standard input size for both pretrained architectures.

Training images undergo augmentation including:

• Rotation up to ±15°
• Width and height shifts
• Shearing
• Zooming
• Horizontal flipping
• Brightness variation

Architecture-specific ImageNet normalization is then applied to each model.

VGG16 and ResNet50 Training

Both models use ImageNet-pretrained weights with their original classification layers removed. An identical custom classification head is attached to both.

The classification head consists of:

Global Average Pooling → Batch Normalization → Dense(512) → Dropout(0.4) → Dense(256) → Dropout(0.3) → Softmax(4)

This ensures that differences in performance are primarily attributable to the convolutional backbone rather than differences in classifier-head design.

Two-Phase Training

Both models follow the same two-stage transfer-learning strategy:

Phase 1: Feature extraction

• Backbone completely frozen.
• Only the new classification head is trained.
• Maximum 20 epochs.
• Adam optimizer.
• Learning rate: 10??.

Phase 2: Fine-tuning

• Only the deepest convolutional block is unfrozen.
• VGG16: Block 5
• ResNet50: conv5_block
• Maximum 20 epochs.
• Learning rate reduced to 10??.

Both models use the same early stopping, learning-rate reduction, and model-checkpointing procedures.

Comparative Evaluation

After training, both models are evaluated on the same 1,600-image test set using:

• Accuracy
• Precision
• Recall
• F1-score
• Confusion matrix

The model with the higher test performance is selected for deployment. According to the study, VGG16 performs better than ResNet50 under the controlled experimental conditions.

Importantly, the researchers deliberately do not create an ensemble of VGG16 and ResNet50 because doing so would increase computational cost and make Grad-CAM++ interpretation more complicated.

Grad-CAM++ Explainability

The selected VGG16 model is further analyzed using Grad-CAM++.

Grad-CAM++ creates a heatmap showing the regions of the MRI image that contributed most strongly to the model's classification. This is particularly useful because tumors can be small, irregular, or located away from the center of an MRI slice.

The output consists of three components:

1. Predicted tumor class
2. Confidence/probability score
3. Grad-CAM++ heatmap showing influential image regions

The heatmap is an explanation of the model's decision, not a tumor segmentation mask. It indicates which regions influenced the prediction but does not precisely outline the tumor boundary.

Conclusion

This paper reported a controlled, architecture-isolated comparison of VGG16 and ResNet50 for four-class brain tumor classification, in which both backbones shared an identical classification head, preprocessing pipeline, and two-phase training schedule. Under these matched conditions, VGG16 attained a test accuracy of 94.31%, marginally exceeding the 94.00% attained by ResNet50, and was accordingly identified as the better-performing architecture in the present experimental setting. Both backbones were shown to concentrate the substantial majority of their classification error on a single class pair, glioma and meningioma, while separating the remaining two categories, no-tumor and pituitary tumor, almost perfectly; this consistent error pattern across two otherwise different architectures points toward an intrinsic visual difficulty in the underlying data for this class pair rather than a shortcoming specific to either backbone. Grad-CAM++ was applied to the selected VGG16 model to supply a visual attribution map for its classification decisions, offering an interpretable complement to the predicted class and confidence score, and the resulting attribution was shown to concentrate on the visually identifiable tumor region rather than on unrelated background structure in the illustrated example. Beyond the specific accuracy figures reported here, the more durable contribution of this study is methodological: by holding the dataset, preprocessing pipeline, classification head, and training schedule identical between VGG16 and ResNet50, the comparison isolates the effect of backbone architecture from the many confounding factors that otherwise make cross-study accuracy comparisons difficult to interpret in this literature. Future work will extend this comparative study in three directions. First, pixel-level tumor localization and quantitative tumor characterization – including segmentation-based hemisphere, shape, and area estimation – will be developed and evaluated as a separately reported stage of this research, building on the classification foundation established here. Second, the training-run-variance question raised in Section V-B will be addressed by repeating each backbone’s training schedule across multiple random seeds and reporting the resulting variance around the mean accuracy, to establish more precisely how much of the observed 0.31-percentage-point gap is attributable to architecture rather than to run-to-run variability. Third, broader external validation across MRI acquired from clinical sources and scanner protocols not represented in the Figshare, SARTAJ, and Br35H repositories will be pursued before any claim of clinical-grade generalization is made for either backbone. Taken together, the results reported in this paper support a modest but well-founded conclusion: under a fixed dataset, preprocessing pipeline, classification head, and training schedule, VGG16 offers a small but consistent accuracy advantage over ResNet50 for four-class brain tumor classification, an advantage that is accompanied by a smaller parameter count rather than traded off against it, and that advantage can be made interpretable through Grad-CAM++ attribution rather than left as an unexplained numerical difference. The controlled experimental design adopted here is intended less as a final verdict on VGG16 versus ResNet50 in general, and more as a template for how such comparisons might be reported more consistently across this literature going forward, so that future backbone comparisons in brain tumor classification are easier to interpret against one another than the present cross-study landscape has allowed.

References

[1] K. S. Prabhas, A. Basem, L. Lakshmi, A. Talha, S. H. Mohammed, M. I. Khan, and N. Ben Khedher, “A deep learning framework for brain tumor detection using CNNs and transfer learning on MRI scans,” Systems and Soft Computing, vol. 7, art. no. 200389, 2025. [2] S. K. Shib, D. R. Dipto, M. T. Rahman, and A. Shufian, “Deep learning-based brain tumor detection and classification using VGG16,” in Proc. 4th Int. Conf. on Robotics, Electrical and Signal Processing Techniques (ICREST), Dhaka, Bangladesh, 2025, pp. 482–486. [3] R. K. Sahoo, S. Mishra, P. Panda, and A. Behera, “Brain tumor detection using deep learning and VGG-16 model,” in Proc. Int. Conf. on Emerging Techniques in Computational Intelligence (ICETCI), Hyderabad, India, 2024, pp. 440–445. [4] H. T. M. Thiyagu, K. Sai Madhurya, M. R. G. Narasimha, and Y. Naga Lokeswar Reddy, “Advanced neural framework for MRI-based brain tumor detection and multiclass classification,” in Proc. Int. Conf. on Advances in Novel Applications of Technology (ICONAT), 2025, pp. 1–6. [5] R. Patil G E, N. K. Chaudhary, B. P. Prajapati, B. K. Shah, P. Jha, and M. K. V, “Optimizing transfer learning for brain tumor detection: fine-tuning strategies and model interpretability insights,” in Proc. IEEE GINOTECH, 2025, pp. 1–5. [6] P. T. Krishnan, P. Krishnadoss, M. Khandelwal, D. Gupta, A. Nihaal, and T. S. Kumar, “Enhancing brain tumor detection in MRI with a rotation invariant Vision Transformer,” Frontiers in Neuroinformatics, vol. 18, 2024. [7] R. Jansi, S. Kowsalya, S. Seetha, and A. Yogadharshini, “A deep learning based brain tumour detection using multimodal MRI images,” in Proc. Int. Conf. on Applied Computing and Recent Advances in Science (ICACRS), 2023, pp. 582–587. [8] M. M. Tayefeh, S. A. G. Ghahramani, and A. M. A. Hemmatyar, “Advancing brain tumor detection via ViRCNN: a fusion of Vision Transformers and Faster R-CNN,” in Proc. IEEE Int. Conf. on Computer and Knowledge Engineering (ICCKE), 2025. [9] A. Golkarieh, S. R. Boroujeni, K. Kiashemshaki, M. Deldadehasl, H. Aghayarzadeh, and A. Ramezani, “Breakthroughs in brain tumor detection: leveraging deep learning and transfer learning for MRI-based classification,” Computer Decision Making: Int. Journal, vol. 2, pp. 708–722, 2025. [10] P. Pilaoon, K. Hamamoto, and N. Maneerat, “Glioma brain tumor classification using convolution neural network and majority voting,” IEEE Access, 2025. [11] C. P. Jetlin and S. P. Annabel, “PyQDCNN: Pyramid QDCNNet for multi-level brain tumor classification using MRI image,” Biomedical Signal Processing and Control, vol. 100, art. no. 107042, 2024. [12] L. Zeng and H. H. Zhang, “Robust brain MRI image classification with SIBOW-SVM,” Computerized Medical Imaging and Graphics, vol. 118, 2024. [13] N. R. Nimmagadda and P. K. Devi, “A deep learning approach for brain tumour classification and detection in MRI images using YOLOv7,” Frontiers in Oncology, 2025. [14] B. Shankar et al., “Deep-CNN based brain tumor classification from MRI images,” in Proc. ICCSCE 2025, Atlantis Press, pp. 1152–1165. [15] J. A. Ali et al., “Multi-class brain tumor MRI segmentation and classification using deep learning and machine learning approaches,” Cancer Imaging, vol. 25, art. no. 131, 2025. [16] R. Disci, F. Gurcan, and A. Soylu, “Advanced brain tumor classification in MR images using pre-trained models and transfer learning,” Cancers, vol. 17, no. 1, 2025. [17] A. Chattopadhay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, “Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks,” in Proc. IEEE Winter Conf. on Applications of Computer Vision (WACV), 2018, pp. 839–847. [18] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in Proc. Int. Conf. on Learning Representations (ICLR), 2015. [19] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. [20] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: visual explanations from deep networks via gradient-based localization,” in Proc. IEEE Int. Conf. on Computer Vision (ICCV), 2017, pp. 618–626.

Copyright

Copyright © 2026 Ganeshula Sai Raghava. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.