AI-Powered Lung Cancer Classification Using Hybrid Histopathological Image Analysis

Abstract

Lung cancer is a major cause of cancer-related mortality, and accurate classification of histopathological tissue patterns can support the analysis of lung cancer subtypes. Manual examination of histopathological images is time-consuming and requires careful assessment of cellular and tissue-level morphological patterns. To assist automated histopathological image analysis, this study presents a hybrid deep learning and machine learning framework for three-class classification of lung histopathological images. The proposed system employs an ImageNet-pretrained DenseNet201 convolutional neural network for deep feature extraction through transfer learning, followed by frozen-base training and fine-tuning of selected network layers. A learned 256-dimensional feature representation is subsequently extracted from the trained network and provided to a CatBoost classifier for final classification. The system categorizes images into lung adenocarcinoma, lung squamous cell carcinoma, and Benign Lung Tissue. On a validation set of 3,000 images, the proposed hybrid framework achieved an overall classification accuracy of 99.90%, with 2,997 images correctly classified. The framework combines deep convolutional feature learning with gradient-boosted machine learning classification to provide an automated and modular approach for lung histopathological image classification

Introduction

The text presents PulmoScan, an automated system for classifying lung histopathological images into three categories: lung adenocarcinoma, lung squamous cell carcinoma, and benign lung tissue. The system is designed to reduce the complexity of manual histopathological image analysis by combining deep learning with traditional machine learning.

The proposed approach uses the LC25000 dataset, specifically its lung subset containing 15,001 images across the three classes. Images are resized to 224 × 224 pixels, normalized, and augmented during training using techniques such as rotation, shifting, flipping, zooming, and shearing.

For feature learning, the study uses ImageNet-pretrained DenseNet201. The model is initially trained with its convolutional base frozen and is then fine-tuned by unfreezing the last 50 layers. The trained network generates a 256-dimensional feature vector for each image. Instead of using DenseNet201's final classification layer, these extracted features are passed to a CatBoost classifier, which performs the final three-class classification.

The dataset is divided into 12,001 training images and 3,000 validation images, with the validation set containing 1,000 images from each class. The DenseNet201 model achieved 100% validation accuracy during the initial training phase and 99.90% during fine-tuning. The final DenseNet201–CatBoost hybrid framework achieved 99.90% validation accuracy.

The main objectives of the system are to automate lung tissue classification, improve feature learning through transfer learning, use DenseNet201 for deep feature extraction, classify the resulting features using CatBoost, and evaluate performance using accuracy, precision, recall, F1-score, and confusion-matrix analysis. The system also provides a prediction interface that displays the predicted tissue class and its probability.

In short

Input histopathology image → preprocessing → DenseNet201 feature extraction → 256-dimensional features → CatBoost classification → three-class prediction

Copyright

Copyright © 2026 D. Suguna Seleena Kennedy, Dr. R. Venkata Lakshmi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.