A Comparative Review of Machine Learning Approaches for Cell-Type Classification in Pancreatic scRNA-seq Data: Perspectives for Diabetes Research

Abstract

Single-cell RNA sequencing (scRNA-seq) has fundamentally transformed the study of cellular heterogeneity in complex tissues, including the human pancreas. The accurate identification and classification of pancreatic cell types — particularly beta, alpha, delta, and ductal cells — is critical for advancing our understanding of Type 1 and Type 2 Diabetes mellitus. This paper presents a comparative review of machine learning (ML) and deep learning (DL) methods applied to scRNA-seq data for pancreatic cell-type classification, with a specific focus on diabetes research. We survey classical approaches including Random Forests, Support Vector Machines, and kNearest Neighbours, as well as deep learning methods such as autoencoders, graph neural networks, and transformerbased foundation models including scBERT and Geneformer. A comparative performance analysis across benchmark datasets reveals that transformer-based architectures consistently outperform classical methods, achieving F1 scores of 0.92–0.99 on pancreatic datasets, while classical ML approaches plateau at 0.71–0.84. We discuss challenges unique to diabetes-focused scRNA-seq analysis including class imbalance, technical batch effects, and the need for interpretable biomarker outputs.

Introduction

Diabetes mellitus affects over 537 million adults worldwide, with Type 1 Diabetes (T1D) caused by autoimmune destruction of pancreatic beta cells and Type 2 Diabetes (T2D) resulting from insulin resistance and beta-cell dysfunction. Understanding disease progression requires studying gene expression at the single-cell level, making single-cell RNA sequencing (scRNA-seq) a powerful tool for identifying cellular heterogeneity and rare cell populations. Large datasets such as the Human Pancreas Analysis Program (HPAP) provide valuable resources for diabetes research.

A major challenge in scRNA-seq analysis is accurately classifying cell types from high-dimensional and sparse gene expression data. This review examines machine learning approaches for pancreatic cell-type classification, categorizing them into classical machine learning, deep learning, and transformer-based foundation models, and compares their performance, interpretability, and suitability for diabetes research.

Pancreatic scRNA-seq datasets present several computational challenges, including data sparsity, severe class imbalance due to the low number of beta cells in T1D, donor-to-donor batch effects, and gradual transitions between related cell types. These factors complicate accurate cell-type classification and require advanced computational methods.

Classical machine learning techniques such as Random Forests, Support Vector Machines (SVMs), and k-Nearest Neighbors (kNN) provide moderate classification accuracy but struggle with rare cell populations and complex gene interactions. Deep learning methods, including scVI and Graph Neural Networks (GNNs), improve performance by learning latent representations and modeling relationships between cells, achieving higher classification accuracy while better handling batch effects and transitional cell states.

Recent transformer-based foundation models, including scBERT and Geneformer, represent the current state of the art. Pretrained on millions of single-cell transcriptomes, these models capture complex gene relationships and achieve the highest classification performance, with Geneformer reporting F1 scores between 0.97 and 0.99 on pancreatic datasets. In addition to improved accuracy, transformer models provide biologically interpretable outputs through attention mechanisms that help identify disease-associated genes and potential biomarkers.

Comparative analysis shows that transformer models outperform traditional methods in accuracy and robustness but require substantial computational resources, particularly high-end GPUs. Classical methods remain computationally efficient and interpretable but are less effective for rare cell detection and complex biological patterns.

Conclusion

This review has surveyed ML methods for cell-type classification in pancreatic scRNA-seq data with a focus on diabetes research. The field has progressed from classical ML approaches with F1 scores in the 0.72–0.84 range to transformer-based foundation models achieving 0.97–0.99, driven by large pretraining corpora and novel tokenisation strategies. Key remaining challenges include class imbalance handling for rare beta cell populations, cross-cohort generalisation, and standardised interpretability evaluation. As single-cell genomics datasets from T1D and T2D cohorts continue to scale, the importance of accurate and interpretable classification methods will only increase. Transformer-based models, combining predictive accuracy with biologically meaningful attention patterns, represent the most promising direction for the next generation of diabetes-focused single-cell analysis tools.

References

[1] International Diabetes Federation, IDF Diabetes Atlas, 10th Edition, Brussels: IDF, 2021. [2] M. A. Atkinson, G. S. Eisenbarth, and A. W. Michels, \"Type 1 diabetes,\" The Lancet, vol. 383, pp. 69–82, 2014. [3] V. Y. Kiselev, T. S. Andrews, and M. Hemberg, \"Challenges in unsupervised clustering of single-cell RNA-seq data,\" Nature Reviews Genetics, vol. 20, pp. 273–282, 2019. [4] S. Freytag et al., \"Comparison of clustering tools in R for medium-sized 10X Genomics single-cell RNAsequencing data,\" F1000Research, vol. 7, 2018. [5] R. Lopez et al., \"Deep generative modelling for single-cell transcriptomics,\" Nature Methods, vol. 15, pp. 1053–1058, 2018. [6] J. Wang et al., \"scGNN is a novel graph neural network framework for single-cell RNA-seq analyses,\" Nature Communications, vol. 12, no. 1882, 2021. [7] F. Yang et al., \"scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data,\" Nature Machine Intelligence, vol. 4, pp. 852–866, 2022. [8] C. V. Theodoris et al., \"Transfer learning enables predictions in network biology,\" Nature, vol. 618, pp. 616–624, 2023. [9] M. D. Luecken et al., \"Benchmarking atlas-level data integration in single-cell genomics,\" Nature Methods, vol. 19, pp. 41–50, 2022. [10] U. Raudvere et al., \"g:Profiler: a web server for functional enrichment analysis,\" Nucleic Acids Research, vol. 47, pp. W191–W198, 2019.

Copyright

Copyright © 2026 Yajvendra Saini, Ananya Pandey, Aanchal Sharma, Mrs. Pratibha Soni. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.