Email Spam Detection: Comparative Review of Deep, Transformer, and LLM Techniques

Abstract

Email remains a mission-critical medium for personal and enterprise communication—and a persistent vector for abuse. Modern spam ranges from bulk advertisements to sophisticated social engineering and phishing, creating burdens on users and infrastructure while elevating risk. Recent advancements have shifted research decisively from sparse, hand-engineered features to deep neural architectures and transformer-based encoders; the latest wave introduces large language models (LLMs) with zero/few-shot classification and stronger semantic understanding. This review synthesizes peer-reviewed developments across that spectrum, comparing representative approaches and highlighting consistent themes: transformers generally outperform classical baselines on content-centric tasks; deep and graph-augmented models capture longer-range and relational cues; and LLMs enable intent-aware classification but pose cost, latency, and governance challenges. We identify open problems around dataset shift, multilingual and code-mixed content, explainability, measurement realism, and production constraints. We conclude by recommending hierarchical spam classification—a two-stage pipeline that first filters spam and then semantically categorizes legitimate mail—as a practical direction to improve downstream triage and analyst productivity while retaining precision at the perimeter.

Introduction

Email remains vital for communication, but spam—comprising 50–85% of global email traffic—poses significant security risks such as phishing and malware. Early spam filters relied on static, rule-based methods (e.g., keyword blacklists), which were simple but easily circumvented and hard to maintain.

Starting in the early 2000s, classical machine learning (ML) techniques like Naïve Bayes and Support Vector Machines became popular, using labeled data to classify spam more effectively. These models encoded emails as feature vectors (words, metadata) and achieved high accuracy (often above 90%). However, spammers adapted with new tactics like gibberish text and image spam, leading to a continual arms race.

In the past decade, deep learning models such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have improved spam detection by automatically learning complex patterns and context within email text and images, often exceeding 95% accuracy. Hybrid architectures combining CNNs, RNN variants (LSTM, GRU), and attention mechanisms further enhanced performance and generalization.

More recently, transformer-based models like BERT have advanced spam filtering by understanding word meanings in context, improving detection of subtle phishing attempts. Integrations with graph-based methods capture relationships between emails, further boosting accuracy. Large language models (LLMs) like GPT show promise in zero- or few-shot spam classification but are not yet full replacements for specialized filters.

Overall, the field has evolved from brittle rule-based systems to adaptive, context-aware models leveraging classical ML, deep learning, transformers, and now LLMs. These advances enable more nuanced, hierarchical spam filtering pipelines that better protect users from increasingly sophisticated threats.

Conclusion

This review examined the progression of email spam detection from rule-based filters and classical machine learning to deep neural networks, transformers, and large language models (LLMs). Within this landscape, our hierarchical, two-stage architecture demonstrates that fine-tuning a compact transformer (DistilBERT) can deliver state-of-the-art spam filtering, achieving ~99.2% accuracy—surpassing a strong classical baseline such as a linear SVM (~98.3%). The second stage employs a zero-shot LLM committee (e.g., GPT-4, Gemini, DeepSeek-Chat) to assign semantic sub-categories to legitimate mail, reaching 91.6% accuracy with a macro-F1?0.913 without any task-specific training. Majority-vote ensembling across LLMs consistently outperforms any single model, mitigating model-specific biases and stabilizing predictions. Together, these results show that high-accuracy spam detection and fine-grained ham organization can be achieved in one cohesive pipeline—moving beyond the binary framing that dominates prior work. Practical viability follows from the way the pipeline allocates computation. Stage 1 leverages DistilBERT’s contextual understanding and efficiency to remove the large majority of spam at low latency; Stage 2 applies deeper semantic reasoning only to the much smaller fraction of ham, enabling automated routing of alerts, support inquiries, updates, and similar categories. In mixed traffic, the overall system accuracy is ~95%, with near-perfect spam recall and LLM-committee latencies around 4–6 seconds per message—appropriate for asynchronous handling of ham while keeping the critical path fast. This division of labor makes the pipeline scalable and cost-aware, aligning well with enterprise needs such as triage, prioritization, and reduction of manual email handling. Notwithstanding these gains, several constraints remain. First, explainability is limited: both transformer and LLM decisions are difficult to interpret, which complicates user trust, auditing, and compliance in regulated settings. Second, real-time efficiency at provider scale is non-trivial. Even with a lightweight transformer front-end, very large volumes and bursty traffic can expose throughput bottlenecks; careful optimization of inference, batching, and serving infrastructure is essential. Third, data limitations persist. Class imbalance across ham sub-categories can bias learning, and most publicly available, labeled corpora are English-centric, leaving performance in multilingual or domain-shifted environments under-validated. Fourth, adversarial evolution is continuous: new obfuscations, phishing templates, and content styles will erode performance unless models are refreshed. Finally, enterprise integration raises engineering and policy questions—data privacy (especially when cloud LLM APIs are involved), reliability, fallbacks, and interoperability with legacy systems. Addressing these facets will determine whether promising research prototypes translate into robust, maintainable production services.

References

[1] F. Jáñez-Martino, R. Alaiz-Rodríguez, V. González-Castro, E. Fidalgo, and E. Alegre, “A review of spam email detection: analysis of spammer strategies and the dataset shift problem,” Artificial Intelligence Review, vol. 56, no. 2, pp. 1145–1173, 2023. doi: 10.1007/s10462-022-10195-4. [2] E. G. Dada, J. S. Bassi, H. Chiroma, S. M. Abdulhamid, A. O. Adetunmbi, and O. E. Ajibuwa, “Machine learning for email spam filtering: review, approaches and open research problems,” Heliyon, vol. 5, no. 6, e01802, 2019. doi: 10.1016/j.heliyon.2019.e01802. [3] A. Barushka and P. Hájek, “Spam filtering using integrated distribution-based balancing approach and regularized deep neural networks,” Applied Intelligence, vol. 48, no. 10, pp. 3538–3556, 2018. doi: 10.1007/s10489-018-1161-y. [4] S. Zavrak and ?. Y?lmaz, “Email spam detection using hierarchical attention hybrid deep learning method,” Expert Systems with Applications, vol. 233, 120977, 2023. doi: 10.1016/j.eswa.2023.120977. [5] J. Doshi, K. Parmar, R. Sanghavi, and N. Shekokar, “A comprehensive dual-layer architecture for phishing and spam email detection,” Computers & Security, vol. 133, 103378, 2023. doi: 10.1016/j.cose.2023.103378. [6] V. S. Tida and C.-Y. Hsu, “Universal spam detection using transfer learning of BERT model,” in Proc. 55th Hawaii Int’l Conf. System Sciences (HICSS), 2022. doi: 10.24251/HICSS.2022.921. [7] F. Zouak, O. El Beqqali, and J. Riffi, “BERT-GraphSAGE: hybrid approach to spam detection,” Journal of Big Data, vol. 12, 128, 2025. doi: 10.1186/s40537-025-01176-9. [8] Y. Wu, S. Si, Y. Zhang, J. Gu, and J. Wosik, “Evaluating the performance of ChatGPT for spam email detection,” arXiv:2402.15537, 2024. doi: 10.48550/arXiv.2402.15537. [9] N. Ahmed, R. Amin, H. Aldabbas, D. Koundal, B. Alouffi, and T. Shah, “Machine learning techniques for spam detection in email and IoT platforms: analysis and research challenges,” Security and Communication Networks, vol. 2022, Art. ID 1862888, 2022. doi: 10.1155/2022/1862888. [10] E. H. Tusher, M. A. Ismail, M. A. Rahman, A. H. Alenezi, and M. Uddin, “Email spam: a comprehensive review of optimized detection methods, challenges, and open research problems,” IEEE Access, vol. 12, pp. 143627–143657, 2024. doi: 10.1109/ACCESS.2024.3467996. [11] R. Meléndez, M. Ptaszy?ski, and F. Masui, “Comparative investigation of traditional machine-learning models and transformer models for phishing email detection,” Electronics, vol. 13, no. 24, 4877, 2024. doi: 10.3390/electronics13244877. [12] G. Nasreen, M. M. Khan, M. Younus, and B. Zafar, “Email spam detection by deep learning models using novel feature selection technique and BERT,” Egyptian Informatics Journal, vol. 26, 100473, 2024. doi: 10.1016/j.eij.2024.100473. [13] K. F. Rafat, M. F. Shahbaz, M. S. Memon, W. A. Khan, and F. Akhund, “Evading obscure communication from spam emails,” PLoS One, vol. 17, no. 2, e0263451, 2022. doi: 10.1371/journal.pone.0263451. [14] S. Saleem, Z. U. Islam, S. S. U. Hasan, H. Akbar, M. F. Khan, and S. A. Ibrar, “Spam email detection using long short-term memory and gated recurrent unit,” Applied Sciences, vol. 15, no. 13, 7407, 2025. doi: 10.3390/app15137407. [15] M. A. Uddin, M. Mahiuddin, and A. A. Chowdhury, “Explainable email spam detection: a transformer-based language modeling approach,” in Proc. 27th Int. Conf. Computer and Information Technology (ICCIT), 2024, pp. 1–6. doi: 10.1109/ICCIT64611.2024.11022595. [16] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. doi: 10.48550/arXiv.1810.04805. [17] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” arXiv:1910.01108, 2019. doi: 10.48550/arXiv.1910.01108. [18] K. I. Roumeliotis, N. D. Tselikas, and D. K. Nasiopoulos, “Next-generation spam filtering: Comparative fine-tuning of LLMs, NLPs, and CNN models for email spam classification,” Electronics, vol. 13, no. 11, 2034, 2024. doi: 10.3390/electronics13112034. [19] M. Schmitt, P. M. Sowa, K. Huguenin, B. Smeets, and P. Simoens, “Digital deception: generative artificial intelligence in social engineering,” Artificial Intelligence Review, 2024. doi: 10.1007/s10462-024-10973-2. [20] M. Stow and B. S. Ezonfa, “Two-stage email classification model for enhanced spam filtering through feature transformation and iterative learning,” International Journal of Computer Sciences and Engineering, vol. 13, no. 2, pp. 16–27, 2025. doi: 10.26438/ijcse/v13i2.1627. [21] H. A. Al-Kabbi, M.-R. Feizi-Derakhshi, and S. Pashazadeh, “A hierarchical two-level feature fusion approach for SMS spam filtering,” Intelligent Automation & Soft Computing, vol. 39, no. 4, pp. 665–682, 2024. doi: 10.32604/iasc.2024.050452. [22] A. Karim, S. Azam, B. Shanmugam, K. Kannoorpatti, and M. Alazab, “A comprehensive survey for intelligent spam email detection,” IEEE Access, vol. 7, pp. 168261–168295, 2019. doi: 10.1109/ACCESS.2019.2954791. [23] A. Poobalan, K. Ganapriya, K. Kalaivani, and K. T. Parthiban, “A novel and secured email classification using deep neural network with bidirectional long short-term memory,” Computer Speech & Language, vol. 89, p. 101667, 2024. doi: 10.1016/j.csl.2024.101667. [24] M. Adnan, M. O. Imam, M. F. Javed, and I. Murtza, “Improving spam email classification accuracy using ensemble techniques: a stacking approach,” International Journal of Information Security, vol. 23, no. 1, pp. 505–518, 2024. doi: 10.1007/s10207-023-00756-1.

Copyright

Copyright © 2025 Ajay Chani, Jai Bhagwan. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.