Deep Learning Intrusion Detection: A Critical Reflection on the Inherent Biases of Datasets and Evaluation Misconceptions
Main Article Content
Keywords
deep learning, intrusion detection, evaluation metric, class imbalance
Abstract
Deep learning-based intrusion detection is an important means of network security protection, but the existing validation and evaluation mechanisms still have many systematic problems. This paper discusses three aspects: mainstream validation methods, inherent biases in public datasets, and the failure of evaluation metrics, and explains the reasons for the “false prosperity” in this field. Models cannot truly understand attack characteristics and only remember the statistical features of old datasets. When the sample classes are imbalanced, metrics such as accuracy and F1 can easily be misleading, and the good performance of models on test sets is difficult to reflect their actual detection effect. This paper points out that slow data updating, unreasonable evaluation standards, and insufficient model robustness are the main difficulties at present, and puts forward some ideas for establishing objective evaluation standards. This review hopes to remind researchers to pay attention to the reliability of model validation and promote deep learning-based intrusion detection from laboratory testing to real network protection scenarios.
References
- [1]Zhao, P., Wang, H., Liu, Y., Zhang, S., Zhao, X., & Chi, Z. (2026). Review of deep learning in network intrusion detection. Computer Engineering and Applications, 62(10), 74-88. https://doi.org/10.3778/j.issn.1002-8331.2510-0117
- [2]Chen, J., Liu, W., Qiu, X., Lv, W., & Xiong, Y. (2025). CRGT-SA: An interlaced and spatiotemporal deep learning model for network intrusion detection. Frontiers of Information Technology & Electronic Engineering, 26(7), 1115-1130. https://doi.org/10.1631/FITEE.2400459
- [3]Zhang, H., Liang, Y., Lan, J., & Zhao, Q. (2024). Intrusion detection method based on memory module and filtered generative adversarial network. Computer Engineering, 50(6), 197-207. https://doi.org/10.19678/j.issn.1000-3428.0068157
- [4]Liu, Y., & Sun, Z. (2024). IWSN intrusion detection enhancement model for adversarial defense. Journal of Chinese Computer Systems, 45(8), 1980-1986. https://doi.org/10.20009/j.cnki.21-1106/TP.2023-0124
- [5]Wang, Z., Zhang, L., & Li, D. (2021). Survey of intrusion detection systems for Internet of Things based on machine learning. Computer Engineering and Applications, 57(4), 18-27. https://doi.org/10.3778/j.issn.1002-8331.2010-0457
- [6]Cao, L., Li, Z., Yang, Y., & Zhao, L. (2021). Intrusion detection method based on two-layer attention networks. Computer Engineering and Applications, 57(19), 142-149. https://doi.org/10.3778/j.issn.1002-8331.2006-0220
- [7]Ding, H., Wan, L., Zhou, K., Long, T., & Xin, Z. (2019). Study on intrusion detection based on deep convolution neural network. Computer Science, 46(10), 173-179. https://doi.org/10.11896/JSJKX.180801429
- [8]Yang, H., & Wang, F. (2019). Network intrusion detection model based on improved convolutional neural network. Journal of Computer Applications, 39(9), 2604-2610. https://doi.org/10.11772/ j.issn.1001-9081.2019020327
- [9]Fu, Z., Xu, Y., Wu, Z., Xu, D., & Xie, X. (2020). SVM-KNN network intrusion detection method based on incremental learning. Computer Engineering, 46(4), 115-122. https://doi.org/10.19678/j.issn.1000-3428.0054701
- [10]Ding, L., Wu, Z., & Su, J. (2020). Intrusion detection method based on ensemble deep forests. Computer Engineering, 46(3), 144-150. https://doi.org/10.19678/j.issn.1000-3428.0053018
- [11]Wang, M., Wang, Y., & Han, J. (2018). Application of deep learning in new intrusion detection system. Microelectronics & Computer, 35(7), 83-86. https://doi.org/10.19304/j.cnki.issn1000-7180.2018.07.017
- [12]Li, C., Wu, Y., Qian, Z., Sun, Z., & Wang, W. (2018). DDoS attack detection and defense in SDN based on a hybrid deep learning model. Journal on Communications, 39(7), 176-187. https://doi.org/10.11959/j.issn.1000-436x.2018128
- [13]Jia, F., & Kong, L. (2017). Intrusion detection algorithm based on convolutional neural network. Transactions of Beijing Institute of Technology, 37(12), 1271-1275. https://doi.org/10.15918/j.tbit1001-0645.2017.12.011
- [14]Gao, N., Gao, L., & He, Y. (2016). Deep belief nets model oriented to intrusion detection system. Systems Engineering and Electronics, 38(9), 2201-2207. https://doi.org/10.3969/j.issn.1001-506X.2016.09.33
- [15]Wang, Z., Zhang, L., & Li, D. (2021). Survey of intrusion detection systems for Internet of Things based on machine learning. Computer Engineering and Applications, 57(4), 18-27. https://doi.org/10.3778/j.issn.1002-8331.2010-0457
- [16]Yin, C., Zhu, Y., Fei, J., & He, X. (2017). A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access, 5, 21954-21961. https://doi.org/10.1109/ACCESS.2017.2762418
- [17]Alrawashdeh, K., & Purdy, C. (2016). Toward an online anomaly intrusion detection system based on deep learning. In 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 195-200). IEEE. https://doi.org/10.1109/ICMLA.2016.0040
- [18]Hnamte, V., & Hussain, J. (2023). DCNNBiLSTM: An efficient hybrid deep learning-based intrusion detection system. Telematics and Informatics Reports, 10, Article 100053. https://doi.org/10.1016/j.teler.2023.100053
- [19]Kasongo, S. M. (2023). A deep learning technique for intrusion detection system using a recurrent neural networks based framework. Computer Communications, 199, 113-125. https://doi.org/10.1016/j.comcom.2022.12.010
- [20]Al-Qatf, M., Lasheng, Y., Al-Habib, M., & Al-Sabahi, K. (2018). Deep learning approach combining sparse autoencoder with SVM for network intrusion detection. IEEE Access, 6, 52843-52856. https://doi.org/10.1109/ACCESS.2018.2869577
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.