A Comprehensive Review using Machine Learning and Deep Learning Methods for Intelligent Intrusion Detection in the Domain of Cybersecurity

Abstract

Organizations today depend heavily on cloud computing, IoT devices, and large-scale well-connected systems. This expansion brings serious cybersecurity risks though. Threats like ransomware, zero-day exploits, DDoS attacks, and advanced persistent threats are happening more often and getting more sophisticated. Current signature-based intrusion detection systems (IDS) struggle with these evolving attacks because they depend on predefined rules and known patterns. This limits how well they work against new threats. That\'s why researchers are exploring machine learning (ML) and deep learning (DL) techniques to build smarter intrusion detection solutions. This paper reviews ML-based, DL-based, and hybrid ML-DL methods for intrusion detection. We examine benchmark datasets including UNSW-NB15 and CIC-IDS2017, looking at their features, benefits, and drawbacks. Class imbalance and high-dimensional feature spaces create problems. The review covers several important models: Random Forest, XGBoost, Convolutional Neural Networks, Long Short-Term Memory networks, and Autoencoders. Hybrid architectures look most promising since they combine deep feature extraction with traditional machine learning classifiers, getting better detection accuracy and fewer false positives. We also identify research gaps, discuss computational challenges, and suggest future directions for building scalable, real-time, and interpretable IDS solutions that meet today\'s cybersecurity needs.

Introduction

The text presents a comprehensive review of Intrusion Detection Systems (IDS) in the context of modern, highly networked environments driven by cloud computing, IoT, and distributed architectures. As cyber threats such as ransomware, DDoS attacks, zero-day exploits, botnets, and APTs grow in scale and complexity, traditional security mechanisms struggle to protect data confidentiality, integrity, and availability.

Conventional signature-based IDS are effective against known attacks with low false positives but fail to detect novel or evolving threats and require frequent database updates. Anomaly-based IDS can identify unknown attacks by learning normal behavior, but they suffer from high false alarm rates due to dynamic network and user behavior. IDS deployment strategies include Network-Based IDS and Host-Based IDS, yet traditional architectures cannot cope with today’s network complexity.

To address these limitations, recent research increasingly adopts machine learning (ML) and deep learning (DL) techniques. Early IDS studies relied on datasets like KDD Cup 1999 and NSL-KDD, which are now outdated. Modern datasets such as UNSW-NB15 and CIC-IDS2017 better reflect real-world traffic but still pose challenges like class imbalance and high dimensionality. While ML models (e.g., Decision Trees, SVM, Random Forest, XGBoost) are efficient and interpretable, they struggle with rare and evolving attacks. DL models (CNNs, LSTMs, Autoencoders) improve detection by learning spatial and temporal patterns but are computationally expensive and harder to deploy in real time.

The paper proposes a conceptual hybrid IDS framework that combines ML and DL to leverage their complementary strengths. The framework includes data acquisition, preprocessing, deep feature extraction using Autoencoders, temporal modeling with Bi-LSTM, and final classification using XGBoost. By integrating CNN-based spatial learning, LSTM-based temporal analysis, and robust ensemble classification, the hybrid approach achieves higher accuracy (up to 96%) with optimized latency, outperforming traditional ML-based IDS in complex LAN and router environments.

A comparative analysis shows that hybrid ML–DL IDS models provide superior detection accuracy, reduced false positives, and better handling of class imbalance compared to standalone ML or DL methods. However, challenges remain, including computational overhead, scalability, real-time deployment, and lack of explainability.

The paper identifies key research gaps and future directions, including:

• Development of explainable hybrid IDS for real-time use,

• Privacy-preserving federated learning to avoid centralized data collection,

• Semi-supervised learning for IoT environments with limited labeled data,

• Use of Graph Neural Networks (GNNs) to model network topology and multi-stage attacks,

• Lightweight, energy-efficient models for edge and IoT deployments,

• Improved adversarial robustness and transfer learning capabilities.

Overall, the text concludes that hybrid ML–DL intrusion detection systems represent a promising path forward for modern cybersecurity, but achieving practical, trustworthy, and scalable deployment requires advances in explainability, privacy, efficiency, and adaptability.

Conclusion

This paper reviews machine learning, deep learning, and hybrid ML, DL approaches for intrusion detection in modern cybersecurity settings. It highlights the shortcomings of traditional signature-based IDS in dealing with complex and changing threats. Machine learning models offer efficient and understandable solutions, but they rely heavily on manually crafted features. Deep learning methods, like CNNs, LSTMs, and Autoencoders, enhance detection accuracy by automatically learning features, though this comes with increased computational demands. This review shows that hybrid ML, DL frameworks successfully combined and enhanced both the approaches. They achieve better detection accuracy, reduced false positives, and greater robustness across benchmark datasets. This makes it promising option for scalable intrusion detection systems.

References

[1] S. J. Stolfo, W. Fan, W. Lee, A. Prodromidis, and P. K. Chan, “Cost-based modeling for fraud and intrusion detection: Results from the JAM project,” DARPA Information Survivability Conference and Exposition, 2000. [2] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), 2009. [3] N. Moustafa and J. Slay,“UNSW-NB15: A comprehensive data set for network intrusion detection systems,”Military Communications and Information Systems Conference (MilCIS), 2015. [4] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization,”International Conference on Information Systems Security and Privacy (ICISSP), 2018.(CIC-IDS2017) [5] T. Chen and C. Guestrin,“XGBoost: A scalable tree boosting system,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016. [6] A. L. Buczak and E. Guven,“A survey of data mining and machine learning methods for cyber security intrusion detection,” IEEE Communications Surveys & Tutorials, vol. 18, no. 2, pp. 1153–1176, 2016. [7] Li, P. Yi, Y. Wu, L. Pan, and J. Li, “A new intrusion detection system based on KNN classification algorithm in wireless sensor network,”Journal of Electrical and Computer Engineering, 2014. [8] M. A. Aydin, A. H. Zaim, and K. G. Ceylan, “A hybrid intrusion detection system design for computer network security,” Computers & Electrical Engineering, Elsevier, 2009. [9] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436–444, 2015. [10] C. Yin, Y. Zhu, J. Fei, and X. He,“A deep learning approach for intrusion detection using recurrent neural networks,” IEEE Access, vol. 5, pp. 21954–21961, 2017. [11] J. Kim, J. Kim, H. L. T. Thu, and H. Kim, “Long short-term memory recurrent neural network classifier for intrusion detection,” International Conference on Platform Technology and Service (PlatCon), IEEE, 2016. [12] Y. Mirsky, T. Doitshman, Y. Elovici, and A. Shabtai,“Kitsune: An ensemble of autoencoders for online network intrusion detection,”Network and Distributed System Security Symposium (NDSS), 2018. [13] Z. Wang, Y. Zeng, Y. Liu, and D. Li, “Deep belief network integrating XGBoost for intrusion detection,” IEEE Access, vol. 6, pp. 1908–1918, 2018. [14] H. Hindy et al.,“A taxonomy and reminder of intrusion detection systems,” IEEE Communications Surveys & Tutorials, 2020. [15] A. Javaid, Q. Niyaz, W. Sun, and M. Alam, “A deep learning approach for network intrusion detection system,” Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies, 2016. [16] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, “A deep learning approach to network intrusion detection,”IEEE Transactions on Emerging Topics in Computational Intelligence, 2018. [17] A. Ahmad et al.,“A deep learning based intrusion detection system for IoT networks,”IEEE Internet of Things Journal, 2021. [18] E. Hodo et al.,“Threat analysis of IoT networks using artificial neural network intrusion detection system,”IEEE Transactions on Emerging Topics in Computational Intelligence, 2022. [19] S. Yaras et al.,“Hybrid deep learning and metaheuristic-based intrusion detection system for IoT,”Electronics, MDPI, 2024. [20] A. Pinto et al.,“A comprehensive survey on machine learning-based intrusion detection systems,”IEEE Access, 2023. [1] S. J. Stolfo, W. Fan, W. Lee, A. Prodromidis, and P. K. Chan, “Cost-based modeling for fraud and intrusion detection: Results from the JAM project,” DARPA Information Survivability Conference and Exposition, 2000. [2] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the KDD CUP 99 data set,” IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), 2009. [3] N. Moustafa and J. Slay,“UNSW-NB15: A comprehensive data set for network intrusion detection systems,”Military Communications and Information Systems Conference (MilCIS), 2015. [4] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection dataset and intrusion traffic characterization,”International Conference on Information Systems Security and Privacy (ICISSP), 2018.(CIC-IDS2017) [5] T. Chen and C. Guestrin,“XGBoost: A scalable tree boosting system,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016. [6] A. L. Buczak and E. Guven,“A survey of data mining and machine learning methods for cyber security intrusion detection,” IEEE Communications Surveys & Tutorials, vol. 18, no. 2, pp. 1153–1176, 2016. [7] Li, P. Yi, Y. Wu, L. Pan, and J. Li, “A new intrusion detection system based on KNN classification algorithm in wireless sensor network,”Journal of Electrical and Computer Engineering, 2014. [8] M. A. Aydin, A. H. Zaim, and K. G. Ceylan, “A hybrid intrusion detection system design for computer network security,” Computers & Electrical Engineering, Elsevier, 2009. [9] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436–444, 2015. [10] C. Yin, Y. Zhu, J. Fei, and X. He,“A deep learning approach for intrusion detection using recurrent neural networks,” IEEE Access, vol. 5, pp. 21954–21961, 2017. [11] J. Kim, J. Kim, H. L. T. Thu, and H. Kim, “Long short-term memory recurrent neural network classifier for intrusion detection,” International Conference on Platform Technology and Service (PlatCon), IEEE, 2016. [12] Y. Mirsky, T. Doitshman, Y. Elovici, and A. Shabtai,“Kitsune: An ensemble of autoencoders for online network intrusion detection,”Network and Distributed System Security Symposium (NDSS), 2018. [13] Z. Wang, Y. Zeng, Y. Liu, and D. Li, “Deep belief network integrating XGBoost for intrusion detection,” IEEE Access, vol. 6, pp. 1908–1918, 2018. [14] H. Hindy et al.,“A taxonomy and reminder of intrusion detection systems,” IEEE Communications Surveys & Tutorials, 2020. [15] A. Javaid, Q. Niyaz, W. Sun, and M. Alam, “A deep learning approach for network intrusion detection system,” Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies, 2016. [16] N. Shone, T. N. Ngoc, V. D. Phai, and Q. Shi, “A deep learning approach to network intrusion detection,”IEEE Transactions on Emerging Topics in Computational Intelligence, 2018. [17] A. Ahmad et al.,“A deep learning based intrusion detection system for IoT networks,”IEEE Internet of Things Journal, 2021. [18] E. Hodo et al.,“Threat analysis of IoT networks using artificial neural network intrusion detection system,”IEEE Transactions on Emerging Topics in Computational Intelligence, 2022. [19] S. Yaras et al.,“Hybrid deep learning and metaheuristic-based intrusion detection system for IoT,”Electronics, MDPI, 2024. [20] A. Pinto et al.,“A comprehensive survey on machine learning-based intrusion detection systems,”IEEE Access, 2023.

Copyright

Copyright © 2026 Miss. Alwiya Shaikh , Dr. Anamika Jain. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.