Research on a Network Intrusion Detection Method Based on LSTM and Transformer
Main Article Content
Keywords
network intrusion detection, deep learning, LSTM, Transformer, fusion model, NSL-KDD
Abstract
Network intrusion detection is an important part of cybersecurity defense. Deep learning models have been widely used for this task, but it remains unclear whether the high performance reported in top-journal studies comes from model-structure innovation or from data-engineering methods. To address this problem, this study proposes a lightweight LSTM-Transformer fusion model. A bidirectional LSTM is used to extract local temporal features, and a single-layer Transformer encoder is used to model global dependencies. Experimental results on the NSL-KDD dataset show that, with fewer parameters and a shorter training time, the model achieves an F1 value of 0.7745, which is 2.97 percentage points higher than that of a pure Transformer. Further decomposition shows that data engineering accounts for about 68% of the performance improvement, while the model reaches only 32.90% accuracy on attack types unseen in the training set.
References
- [1]Manocchio, L. D., Layeghy, S., Lo, W. W., Kulatilleke, G. K., Sarhan, M., & Portmann, M. (2024). Flowtransformer: A transformer framework for flow-based network intrusion detection systems. Expert Systems with Applications, 241, 122564. https://doi.org/10.1016/j.eswa.2023.122564
- [2]Zhang, Z., Si, X., Li, L., Gao, Y., Li, X., Yuan, J., & Xing, G. (2023). An intrusion detection method based on Transformer-LSTM model. In 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE) (pp. 352-355). IEEE. https://doi.org/10.1109/NNICE58320.2023.10105733
- [3]Liu, Y., & Wu, L. (2023). Intrusion detection model based on improved Transformer. Applied Sciences, 13(10), 6251. https://doi.org/10.3390/app13106251
- [4]Zhang, J., Zhang, X., Liu, Z., Fu, F., Jiao, Y., & Xu, F. (2023). A network intrusion detection model based on BiLSTM with multi-head attention mechanism. Electronics, 12(19), 4170. https://doi.org/10.3390/electronics12194170
- [5]Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
- [6]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
- [7]Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (pp. 1-6). IEEE. https://doi.org/10.1109/CISDA.2009.5356528
- [8]Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al-Nemrat, A., & Venkatraman, S. (2019). Deep learning approach for intelligent intrusion detection system. IEEE Access, 7, 41525-41550. https://doi.org/10.1109/ACCESS.2019.2895334
- [9]Xu, C., Shen, J., Du, X., & Zhang, F. (2018). An intrusion detection system using a deep neural network with gated recurrent units. IEEE Access, 6, 48697-48707. https://doi.org/10.1109/ACCESS.2018.2867564
- [10]Yin, C., Zhu, Y., Fei, J., & He, X. (2017). A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access, 5, 21954-21961. https://doi.org/10.1109/ACCESS.2017.2762418
- [11]Wu, K., Chen, Z., & Li, W. (2018). A novel intrusion detection model for a massive network using convolutional neural networks. IEEE Access, 6, 50850-50859. https://doi.org/10.1109/ACCESS.2018.2868993
- [12]Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321-357. https://doi.org/10.1613/jair.953
- [13]Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
- [14]Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958.
- [15]Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep learning (Vol. 1, No. 2, pp. 1-800). Cambridge: MIT press.
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.