Artificial Intelligence for Real-Time Cyber Threat Classification and Emerging Threat Detection: A Structured Review of Methods, Datasets, Challenges, and Research Directions

Abstract

The swift adoption of cloud computing, the Internet of Things (IoT), fifth-generation (5G) communication, and edge computing has extensively broadened the cyber-attack surface, facilitating the emergence of threats with increased scale, speed, and complexity. Traditional detection techniques, which rely on signatures, rules, and statistical analysis, continue to be effective for identifying known attacks; however, their capacity to detect zero-day exploits, polymorphic malware, and advanced persistent threats is limited. Consequently, there has been a rise in the implementation of artificial intelligence (AI) for adaptive and real-time cyber threat analysis. This review evaluates the evolution of AI-driven methodologies for real-time cyber threat classification and the identification of new threats, drawing from over fifty peer-reviewed studies identified through a structured search of leading scholarly databases. Instead of treating the studies in isolation, the literature is synthesized based on detection objectives, computational techniques, datasets, evaluation metrics, deployment environments, and acknowledged limitations. The review juxtaposes conventional detection methods with machine learning, deep learning, explainable AI (XAI), reinforcement learning, federated learning, graph neural networks (GNNs), large language models (LLMs), and generative AI. It places particular emphasis on frequently utilized cybersecurity datasets and evaluation practices, as well as ongoing challenges related to class imbalance, adversarial manipulation, computational overhead, model interpretability, privacy concerns, and inadequate validation in real-world scenarios. The reviewed literature indicates that AI-based methodologies often demonstrate superior detection capabilities for intricate and previously unseen attack patterns compared to traditional methods; however, direct performance comparisons are complicated due to discrepancies in datasets, experimental designs, and evaluation protocols. Moreover, a limited number of proposed models have been evaluated under authentic operational circumstances. In light of these observations, the review identifies significant research gaps and outlines prospective paths for developing explainable, privacy-conscious, computationally efficient, and readily deployable AI-enabled cyber-defence systems.

Introduction

This review examines the role of artificial intelligence (AI) in real-time cyber threat classification and emerging threat detection. The rapid adoption of cloud computing, IoT, 5G, edge computing, and digital financial systems has increased connectivity and efficiency but has also expanded the cybersecurity attack surface. Modern threats—including malware, ransomware, phishing, botnets, DDoS attacks, insider threats, APTs, zero-day exploits, and AI-assisted attacks—are becoming more sophisticated and difficult for traditional security systems to detect.

Traditional vs. AI-Based Detection

Conventional cybersecurity relies mainly on:

• Signature-based detection – highly effective against known threats but unable to identify new or modified attacks without existing signatures.
• Rule-based detection – transparent and effective in stable environments but requires continuous expert maintenance.
• Anomaly-based detection – capable of identifying unknown threats but can generate many false positives.
• Heuristic-based detection – can detect modified or suspicious malware but may still miss advanced attacks.
• IDS/IPS systems – provide continuous monitoring but struggle with encrypted traffic, large-scale security data, and sophisticated attacks.

These limitations have encouraged a transition toward adaptive AI-based cybersecurity systems.

AI and Machine Learning Approaches

AI can process large volumes of security data, identify patterns, detect anomalies, classify attacks, and support rapid decision-making.

The review discusses several approaches:

• Supervised learning: Decision Trees, Random Forest, SVM, Logistic Regression, and Naïve Bayes are effective for known attack categories but depend heavily on high-quality labelled datasets.
• Unsupervised learning: K-Means, DBSCAN, and hierarchical clustering can discover previously unknown patterns but often produce higher false-positive rates.
• Deep learning: CNNs, RNNs, LSTMs, and Transformers automatically extract complex features and are useful for intrusion detection, malware classification, phishing detection, and traffic analysis. Their major disadvantages are high computational and data requirements.
• Hybrid models: Combine multiple learning approaches to improve accuracy, adaptability, and robustness, although they increase system complexity.

Emerging AI Technologies

The review particularly highlights six emerging technologies:

1. Explainable AI (XAI):
   Makes AI decisions understandable to security analysts. Techniques such as SHAP help identify why a system classified an activity as malicious. However, standardized methods for measuring explanation quality are still lacking.
2. Reinforcement Learning (RL):
   Enables systems to learn defensive actions through rewards and penalties. It can support adaptive intrusion detection, automated response, cyber deception, and vulnerability assessment. Challenges include training stability, realistic environments, reward design, and adversarial manipulation.
3. Federated Learning (FL):
   Allows organizations or devices to collaboratively train models without sharing raw security data. This improves privacy but introduces challenges involving non-IID data, communication overhead, model synchronization, and model-poisoning attacks.
4. Graph Neural Networks (GNNs):
   Represent hosts, users, devices, files, and communications as interconnected graphs. This allows detection of suspicious relationships and multi-stage attacks such as APTs. Scalability, graph construction, interpretability, and real-time processing remain challenges.
5. Large Language Models (LLMs):
   Can analyze threat reports, logs, vulnerability descriptions, and incident histories; extract indicators of compromise; summarize incidents; and assist security analysts. However, high computational costs, hallucinations, prompt injection, and adversarial inputs remain concerns.
6. Generative AI:
   Can generate synthetic security data, simulate attacks, support vulnerability assessment, and improve training datasets. At the same time, attackers can exploit it to generate convincing phishing messages, social-engineering content, and malicious code.

Research Objectives and Questions

The review aims to determine:

• How AI improves real-time cyber threat classification compared with traditional approaches.
• Which datasets and evaluation metrics are currently used and how their differences affect research comparisons.
• What methodological, technical, and deployment challenges remain.
• Which future approaches can create explainable, privacy-preserving, adaptive, scalable, and deployable cyber-defence systems.

Key Research Gaps

Despite significant progress, several issues remain:

• Dataset imbalance and poor dataset diversity
• Lack of standardized datasets and evaluation metrics
• High computational and deployment costs
• Difficulty achieving reliable real-time detection
• Limited explainability of complex AI models
• Vulnerability to adversarial attacks
• Privacy and data-sharing concerns
• Scalability problems in large networks
• Poor generalization across different environments
• Challenges in deploying AI on resource-constrained IoT/edge devices

Conclusion

Artificial intelligence has emerged as a pivotal component in cybersecurity, facilitating intelligent, adaptive, and real-time threat detection. This review assesses AI-driven threat classification methodologies, encompassing traditional detection techniques, classical machine learning, deep learning, and newer approaches. It includes an analysis of benchmark datasets, evaluation metrics, comparative insights, challenges, gaps, and future directions. The findings indicate that AI methodologies enhance the accuracy, speed, and scalability of detection in comparison to traditional methods, with machine learning, deep learning, explainable AI, federated learning, and edge intelligence contributing to more effective and adaptive defence mechanisms. However, significant limitations remain, including a lack of realistic datasets, challenges in interpretability, susceptibility to adversarial attacks, high computational demands, privacy concerns, and inadequate validation under real-world conditions. The comparative analysis reveals that while strong benchmark results are observed, they have not yet translated into reliable operational performance. Future research should thus focus on the development of explainable, secure, resource-efficient, and adaptive models, which must be assessed using contemporary datasets and in practical environments. Emphasis on privacy-preserving learning, autonomous defence mechanisms, and standardized, reproducible evaluation processes will likely influence the evolution of next-generation cyber-defence systems. Results from individual studies should be interpreted within the context of their specific datasets, settings, and evaluation metrics.

References

[1] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. [2] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Hoboken, NJ, USA: Pearson, 2020. [3] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, \"A detailed analysis of the KDD Cup 99 data set,\" in Proc. IEEE Symp. Comput. Intell. Secur. Defense Appl. (CISDA), 2009, pp. 1-6, doi: 10.1109/CISDA.2009.5356528. [4] N. Moustafa and J. Slay, \"UNSW-NB15: A comprehensive data set for network intrusion detection systems,\" in Proc. Mil. Commun. Inf. Syst. Conf. (MilCIS), 2015, pp. 1-6, doi: 10.1109/MilCIS.2015.7348942. [5] N. Koroniotis, N. Moustafa, E. Sitnikova, and B. Turnbull, \"Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset,\" Future Gener. Comput. Syst., vol. 100, pp. 779-796, 2019, doi: 10.1016/j.future.2019.05.041. [6] M. Ring, D. Wunderlich, D. Scheuring, D. Landes, and A. Hotho, \"A survey of network-based intrusion detection data sets,\" Comput. Secur., vol. 86, pp. 147-167, 2019, doi: 10.1016/j.cose.2019.06.005. [7] A. Khraisat, I. Gondal, P. Vamplew, and J. Kamruzzaman, \"Survey of intrusion detection systems: Techniques, datasets and challenges,\" Cybersecurity, vol. 2, no. 20, 2019, doi: 10.1186/s42400-019-0038-7. [8] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, \"Toward generating a new intrusion detection dataset and intrusion traffic characterization,\" in Proc. 4th Int. Conf. Inf. Syst. Secur. Privacy (ICISSP), 2018, pp. 108-116, doi: 10.5220/0006639801080116. [9] M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, \"Network anomaly detection: Methods, systems and tools,\" IEEE Commun. Surveys Tuts., vol. 16, no. 1, pp. 303-336, 2014, doi: 10.1109/SURV.2013.052213.00046. [10] M. A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, \"Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study,\" J. Inf. Secur. Appl., vol. 50, art. 102419, 2020, doi: 10.1016/j.jisa.2019.102419. [11] M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras, and H. Janicke, \"Edge-IIoTset: A new comprehensive realistic cyber security dataset of IoT and IIoT applications for centralized and federated learning,\" IEEE Access, vol. 10, pp. 40281-40306, 2022, doi: 10.1109/ACCESS.2022.3165809. [12] Y. LeCun, Y. Bengio, and G. Hinton, \"Deep learning,\" Nature, vol. 521, no. 7553, pp. 436-444, 2015, doi: 10.1038/nature14539. [13] T. Chen and C. Guestrin, \"XGBoost: A scalable tree boosting system,\" in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), 2016, pp. 785-794, doi: 10.1145/2939672.2939785. [14] L. Breiman, \"Random forests,\" Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001, doi: 10.1023/A:1010933404324. [15] C. Cortes and V. Vapnik, \"Support-vector networks,\" Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995, doi: 10.1007/BF00994018. [16] F. Pedregosa et al., \"Scikit-learn: Machine learning in Python,\" J. Mach. Learn. Res., vol. 12, pp. 2825-2830, 2011. [17] I. Goodfellow, J. Shlens, and C. Szegedy, \"Explaining and harnessing adversarial examples,\" in Proc. Int. Conf. Learn. Representations (ICLR), 2015. [18] D. Dua and C. Graff, \"UCI machine learning repository,\" Univ. California, Irvine, CA, USA, 2019. [19] S. Ismail, S. Dandan, and A. Qushou, \"Intrusion detection in IoT and IIoT: Comparing lightweight machine learning techniques using TON_IoT, WUSTL-IIOT-2021, and Edge-IIoTset datasets,\" IEEE Access, vol. 13, pp. 73468-73485, 2025. [20] M. Humayun, N. Tariq, M. Alfayad, M. Zakwan, G. Alwakid, and M. Assiri, \"Securing the Internet of Things in artificial intelligence era: A comprehensive survey,\" IEEE Access, vol. 12, pp. 25469-25490, 2024, doi: 10.1109/ACCESS.2024.3365634. [21] V. Chandola, A. Banerjee, and V. Kumar, \"Anomaly detection: A survey,\" ACM Comput. Surv., vol. 41, no. 3, art. 15, 2009, doi: 10.1145/1541880.1541882. [22] D. E. Denning, \"An intrusion-detection model,\" IEEE Trans. Softw. Eng., vol. SE-13, no. 2, pp. 222-232, 1987, doi: 10.1109/TSE.1987.232894. [23] M. V. Mahoney and P. K. Chan, \"An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection,\" in Proc. Recent Advances in Intrusion Detection (RAID), 2003, pp. 220-237, doi: 10.1007/978-3-540-45248-5_13. [24] A. H. Salem, S. M. Azzam, O. E. Emam, and A. A. Abohany, \"Advancing cybersecurity: A comprehensive review of AI-driven detection techniques,\" J. Big Data, vol. 11, art. 105, 2024, doi: 10.1186/s40537-024-00957-y. [25] A. Ali, M. Charfeddine, B. Ammar, B. B. Hamed, F. Albalwy, A. Alqarafi, and A. Hussain, \"Unveiling machine learning strategies and considerations in intrusion detection systems: A comprehensive survey,\" Front. Comput. Sci., vol. 6, art. 1387354, 2024, doi: 10.3389/fcomp.2024.1387354. [26] H.-J. Liao, C.-H. R. Lin, Y.-C. Lin, and K.-Y. Tung, \"Intrusion detection system: A comprehensive review,\" J. Netw. Comput. Appl., vol. 36, no. 1, pp. 16-24, 2013, doi: 10.1016/j.jnca.2012.09.004. [27] C. Molnar, Interpretable Machine Learning, 2nd ed., 2022. [Online]. Available: https://christophm.github.io/interpretable-ml-book/ [28] S. M. Lundberg and S.-I. Lee, \"A unified approach to interpreting model predictions,\" in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017. [29] J. L. Hernandez-Ramos, G. Karopoulos, E. Chatzoglou, V. Kouliaridis, E. Marmol, A. Gonzalez-Vidal, and G. Kambourakis, \"Intrusion detection based on federated learning: A systematic review,\" arXiv:2308.09522, 2023. [30] P. Kairouz et al., \"Advances and open problems in federated learning,\" Found. Trends Mach. Learn., vol. 14, nos. 1-2, pp. 1-210, 2021, doi: 10.1561/2200000083. [31] F. Guan, T. Zhu, W. Zhou, and K.-K. R. Choo, \"Graph neural networks: A survey on the links between privacy and security,\" Artif. Intell. Rev., vol. 57, art. 40, 2024, doi: 10.1007/s10462-023-10656-4. [32] M. Zhong, M. Lin, C. Zhang, and Z. Xu, \"A survey on graph neural networks for intrusion detection systems: Methods, trends and challenges,\" Comput. Secur., vol. 141, art. 103821, 2024, doi: 10.1016/j.cose.2024.103821. [33] T. N. Kipf and M. Welling, \"Semi-supervised classification with graph convolutional networks,\" in Proc. Int. Conf. Learn. Representations (ICLR), 2017. [34] H. Xu, S. Wang, N. Li, K. Wang, Y. Zhao, K. Chen, T. Yu, Y. Liu, and H. Wang, \"Large language models for cyber security: A systematic literature review,\" arXiv:2405.04760, 2024. [35] J. Zhang, H. Bu, H. Wen, Y. Liu, H. Fei, R. Xi, L. Li, Y. Yang, H. Zhu, and D. Meng, \"When LLMs meet cybersecurity: A systematic literature review,\" Cybersecurity, vol. 8, no. 1, art. 55, pp. 1-41, 2025, doi: 10.1186/s42400-025-00361-w. [36] H. He and E. A. Garcia, \"Learning from imbalanced data,\" IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263-1284, 2009, doi: 10.1109/TKDE.2008.239. [37] G. Li, P. Zhu, J. Li, Z. Yang, N. Cao, and Z. Chen, \"Security matters: A survey on adversarial machine learning,\" arXiv:1810.07339, 2018. [38] A. Vassilev, A. Oprea, A. Fordyce, H. Anderson, X. Davies, and M. Hamin, \"Adversarial machine learning: A taxonomy and terminology of attacks and mitigations,\" NIST AI 100-2e2023, Nat. Inst. Standards Technol., 2024, doi: 10.6028/NIST.AI.100-2e2023. [39] A. B. Buczak and E. Guven, \"A survey of data mining and machine learning methods for cyber security intrusion detection,\" IEEE Commun. Surveys Tuts., vol. 18, no. 2, pp. 1153-1176, 2016, doi: 10.1109/COMST.2015.2494502. [40] A. Aldhaheri, F. Alwahedi, M. A. Ferrag, and A. Battah, \"Deep learning for cyber threat detection in IoT networks: A review,\" Internet Things Cyber-Phys. Syst., vol. 4, pp. 110-128, 2024, doi: 10.1016/j.iotcps.2023.09.003. [41] A. Alsaedi, N. Moustafa, Z. Tari, A. Mahmood, and A. Anwar, \"TON_IoT telemetry dataset: A new generation dataset of IoT and IIoT for data-driven intrusion detection systems,\" IEEE Access, vol. 8, pp. 165130-165150, 2020, doi: 10.1109/ACCESS.2020.3022862. [42] M. Zolanvari, M. A. Teixeira, L. Gupta, K. M. Khan, and R. Jain, \"WUSTL-IIOT-2021 dataset for IIoT cybersecurity research,\" IEEE DataPort, 2022, doi: 10.21227/yftq-n229. [43] A. Rjoub, J. Bentahar, O. Abdel Wahab, R. Mizouni, A. Song, R. Cohen, H. Otrok, and A. Mourad, \"A survey on explainable artificial intelligence for cybersecurity,\" arXiv:2303.12942, 2023. [44] C. Mendes and T. N. Rios, \"Explainable artificial intelligence and cybersecurity: A systematic literature review,\" arXiv:2303.01259, 2023. [45] R. Chalapathy and S. Chawla, \"Deep learning for anomaly detection: A survey,\" arXiv:1901.03407, 2019. [46] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, \"A deep learning approach to network intrusion detection,\" IEEE Trans. Emerg. Topics Comput. Intell., vol. 2, no. 1, pp. 41-50, 2018, doi: 10.1109/TETCI.2017.2772792. [47] C. Yin, Y. Zhu, J. Fei, and X. He, \"A deep learning approach for intrusion detection using recurrent neural networks,\" IEEE Access, vol. 5, pp. 21954-21961, 2017, doi: 10.1109/ACCESS.2017.2762418. [48] A. Patcha and J.-M. Park, \"An overview of anomaly detection techniques: Existing solutions and latest technological trends,\" Comput. Netw., vol. 51, no. 12, pp. 3448-3470, 2007, doi: 10.1016/j.comnet.2007.02.001. [49] R. Sommer and V. Paxson, \"Outside the closed world: On using machine learning for network intrusion detection,\" in Proc. IEEE Symp. Secur. Privacy, 2010, pp. 305-316, doi: 10.1109/SP.2010.25. [50] H. Kheddar, \"Transformers and large language models for efficient intrusion detection systems: A comprehensive survey,\" Inf. Fusion, vol. 124, art. 103347, 2025, doi: 10.1016/j.inffus.2025.103347. [51] M. A. Ferrag, M. Ndhlovu, N. Tihanyi, L. C. Cordeiro, M. Debbah, T. Lestable, and N. S. Thandi, \"Revolutionizing cyber threat detection with large language models: A privacy-preserving BERT-based lightweight model for IoT/IIoT devices,\" IEEE Access, vol. 12, pp. 23733-23750, 2024, doi: 10.1109/ACCESS.2024.3363469. [52] E. C. P. Neto, S. Dadkhah, R. Ferreira, A. Zohourian, R. Lu, and A. A. Ghorbani, \"CICIoT2023: A real-time dataset and benchmark for large-scale attacks in IoT environment,\" Sensors, vol. 23, no. 13, art. 5941, 2023, doi: 10.3390/s23135941.

Copyright

Copyright © 2026 Jaswanth Syam Sundar Garugu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.