Oral Cancer Detection Using Deep Learning

Abstract

Oral cancer is a significant global health concern, with early detection being crucial for improving patient outcomes. Recent advancements in deep learning have shown promise in enhancing the accuracy and efficiency of oral cancer diagnosis. This paper reviews the current state-of-the-art deep learning techniques applied to oral cancer detection, focusing on various architectures such as Convolutional Neural Networks (CNNs), their layers, algorithms like (YOLO) and hybrid methods to improve this model. We analyze the performance metrics of these models, including accuracy, sensitivity, specificity, and F1-score, across different datasets. Furthermore, we discuss the challenges and limitations faced in deploying these models in clinical settings, such as data scarcity, model interpretability, and integration with existing diagnostic workflows. Our findings suggest that the deep learning models have achieved high diagnostic accuracy, further research is needed to address the practical challenges to ensure their widespread adoption in clinical practice. This paper aims to provide a comprehensive overview of the advancements in deep learning for oral cancer detection and to highlight future research directions in this field.

Introduction

Overview

Oral cancer involves uncontrolled cell growth in the mouth, often starting as lesions. A significant type, Oral Squamous Cell Carcinoma (OSCC), accounts for 90% of cases. Detecting and classifying lesions early is crucial for effective treatment. Deep learning methods are increasingly applied to aid in diagnosis and staging.

Detection and Staging

Oral cancer typically affects areas like the tongue, lips, gums, and inside the cheeks. Key symptoms include non-healing sores, lumps, difficulty swallowing or speaking, and persistent throat pain.

Risk factors:

• Tobacco and alcohol use

• Betel nut consumption (common in India)

• HPV infection

• Age and gender (more common in men)

Cancer staging ranges from Stage 0 (no cell damage) to Stage 4c (spread to distant organs like lungs or liver).

Deep Learning in Oral Cancer

Deep learning, especially supervised learning, is effective in cancer detection by learning from labeled image data.

Key techniques:

• Convolutional Neural Networks (CNNs): Used for image classification, extracting features from medical images.

• YOLO (You Only Look Once): Real-time object detection model that identifies lesions in one pass.

Related Research

Past studies used various machine learning techniques (SVM, MLP, ANN, Decision Trees, etc.) to detect cancer presence or classify its types. However, few studies focused on staging of oral cancer — a critical gap addressed in this research.

Proposed System

A comprehensive deep learning-based oral cancer detection system is introduced using YOLOv8 and CNNs. The system includes:

1. Data Collection: From public datasets and real-time camera images.

2. Preprocessing: Image resizing, normalization, and augmentation for better training.

3. Annotation Interface: Manual labeling of images (e.g., lesions).

4. Model Training: YOLOv8 trained on annotated images.

5. Inference Module: Real-time detection of lesions in new images.

Conclusion

In conclusion, our deep learning-based oral cancer detection system is designed to deliver a reliable, accurate, and accessible diagnostic experience for both medical professionals and patients. In our project, we significantly enhanced the model by incorporating an extensive dataset comprising oral images collected from various regions to ensure data diversity. We utilized the state-of-the-art YOLOv8 algorithm, which enabled high precision in identifying cancerous lesions from oral cavity images. As a result of these improvements, our system achieved nearly 98% accuracy in detection. Its adaptability across diverse environments, potential for integration with telemedicine, and scalability made it suitable for widespread deployment—from clinics to community health programs. With these advancements in data coverage, real-time performance, and detection capabilities, the system set a new benchmark in medical diagnostics, combining clinical value with cutting-edge technological innovation. Future enhancements to this system could include integration with medical imaging technologies such as X-rays and CT scans to provide a more comprehensive diagnostic view. Real-time detection through video feeds could enable immediate screenings, particularly in underserved regions. Expanding the dataset to include diverse age groups and cancer stages will further improve model accuracy, while combining YOLOv8 with other deep learning methods like image segmentation could yield even more precise results. A userfriendly clinician interface and personalized risk assessments based on lifestyle and medical history will make the system more intelligent, practical, and inclusive.

References

[1] Fatihah Mohd, Noor Maizura Mohamad Noor, Zainab Abu Bakar, Zainul Ahmad Rajion, Analysis of Oral Cancer Prediction using Features Selection with Machine Learning, ICIT 2015 The 7th International Conference on Information Technology. [2] Ahmad LG*, Eshlaghy AT, Poorebrahimi A, Ebrahimi M and Razavi AR, Using Three Machine Learning Techniques for Predicting Breast Cancer Recurrence, Health & Medical Informatics (2013), 2157-7420. [3] Harikumar Rajaguru and Sunil Kumar Prabhakar, Performance Comparison of Oral Cancer Classification with Gaussian MixtureMeasures and Multi Layer Perceptron, The 16th International Conference on Biomedical Engineering p(2017) 123-129 [4] Amy F. Ziober,Kirtesh R. Patel,Faizan Alawi, Phyllis Gimotty, 4 Randall S. Weber, MichaelM. Feldman, Ara A. Chalian, Gregory S. Weinstein,Jennifer Hunt, and Barry L. Ziober, Identification of a Gene Signature for Rapid Screening of Oral Squamous Cell Carcinoma, American Association for Cancer (2018). [5] Marc Aubreville, Christian Knipfer, Nicolai Oetter, Christian Jaremenko, Erik Rodner, Joachim Denzler, Christopher Bohr, Helmut Neumann, Florian Stelzle, & Andreas Maier, Automatic Classification of Cancerous Tissue in Laserendomicroscopy Images of the Oral Cavity using Deep Learning, SCIENTIFIC Reports,(2017), 7: 11979. [6] Shreyansh A. Prajapati, R. Nagaraj and Suman Mitra, Classification of Dental Diseases Using CNN and Transfer Learning, 5th International Symposium on Computational and Business Intelligence, (2017), 978-15386-1772-4/17. [7] Martin Halicek, Guolan Lu, James V. Little, Xu Wang, Mihir Patel, Christopher C. Griffith, Mark W. El-Deiry, Amy Y. Chen, Baowei Fei, Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging, J. Biomed. Opt. 22(6), 060503 (2017). [8] Ramzi Ben Ali, Ridha Ejbali and Mourad Zaied, Detection and Classification of Dental Caries in X-ray Images Using Deep Neural Networks, The Eleventh International Conference on Software Engineering Advances,(2016), 978-161208-498-5. [9] Konstantina Kourou , Themis P. Exarchos , Konstantinos P. Exarchos ,Michalis V. Karamouzis , Dimitrios I. Fotiadis, Machine learning applications in cancer prognosis and prediction, Computational and structural biotechnology journal, (2015),18-17. [10] Wafaa K. Shams and Zaw Z. Htike, Oral Cancer Prediction Using Gene Expression Profiling and, International Journal of Applied Engineering Research, (2017) ,0973-4562. [11] Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of oral cancer with deep neural networks. Nature, 542(7639), 115–118. [12] Sabarinathan, D., Krishnan, N. M., & Rajendiran, P. (2021). Deep learning models in oral cancer diagnosis and therapy planning: a review. Frontiers in Oral Health, 2, 794248. [13] Uliana, R. S., & Krohling, R. A. (2025). Classifying oral and skin lesions using diffusion models. arXiv. [14] Lian, C., He, Y., Wang, Y., et al. (2024). Oral cancer screening using multimodal deep learning on cytology slides. arXiv. https://arxiv.org/abs/2407.01869. [15] Sharma, M. (2024). Oral squamous cell carcinoma detection using EfficientNetB3 deep learning model. [16] Warin, K., & Suebnukarn, S. (2024). Deep learning in oral cancer: a systematic review. BMC Oral Health, 24, 212. [17] Matsuo, K.; Purushotham, S.; Jiang, B.; Mandelbaum, R.S.; Takiuchi, T.; Liu, Y.; Roman, L.D. Survival outcome prediction in oral cancer: Cox models vs deep-learning model. Am. J. Obstet. Gynecol. 2018. [18] Welikala, R.A.; Remagnino, P.; Lim, J.H.; Chan, C.S.; Rajendran, S.; Kallarakkal, T.G.; Zain, R.B.; Jayasinghe, R.D.; Rimal, J.; Kerr, A.R.; et al. Automated Detection and Classification of Oral Lesions Using Deep Learning for Early Detection of Oral Cancer. IEEE Access 2020, 8, 132677– 132693. [19] Kiruthika, S.; RahmathNisha, S. Automated Oral Cancer Detection and Classification using very Deep Convolutional Neural Network Algorithm. Test Manag. Eng. 2020, 83, 20019–20027. [20] Fati SM, Senan EM, Javed Y. Early diagnosis of Oral squamous cell carcinoma based on histopathological images using deep and hybrid learning approaches. Diagnostics (Basel). 2022;12:1899. [21] Kim DW, Lee S, Kwon S, Nam W, Cha IH, Kim HJ. Deep learning-based survival prediction of oral cancer patients. Sci Rep. 2019;9:6994. [22] Chandu A, Adams G, Smith AC. Factors affecting survival in patients with oral cancer: an Australian perspective. Int J Oral Maxillofac Surg. 2005;34:514–20. [23] Ren, Z.; Hu, C.; He, H.; Li, Y.; Lyu, J. Global and regional burdens of oral cancer from 1990 to 2017: Results from the global burden of disease study. Cancer Commun. 2020, 40, 81–92. [24] Daisuke K, Shumpei I. Machine Learning Methods for Histo pathological Image Analysis.Compt and Struct Biotec J, 2020; 16: 34-42. [25] Shruti DP, Wayakule JM, Apurva DK. Skin Segmentation Using YCbCr and RGB Color Models. Int J of Adv Res Comp Sci and Soft Eng, 2014; 4(7).

Copyright

Copyright © 2025 Rajababu T, Dr. Sureshkumar RG, Abishek S, Kathirvelan M, Mohamed Shafiudeen A, Sandakumar A. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.