A Driver Fatigue Detection Mechanism Based on a Lightweight RGB-IR Fusion Algorithm

Main Article Content

Weixuan Shao
School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China

DOI: https://doi.org/10.70267/mseac.20264155

Keywords

driver fatigue detection, YOLO, RGB-IR, cross-modal fusion, intelligent transportation systems

Abstract

To address the degradation of feature representation in a single RGB modality under low-light conditions, and the lack of texture and semantic information in a single IR modality, this paper proposes a lightweight fatigue detection method based on RGB-IR dual-modal fusion. This method realizes cross-modal information fusion at the feature layer by building a dual-stream feature extraction framework, and introduces a feature alignment mechanism to improve the semantic consistency of RGB and IR; At the same t ime, it combines lightweight network design and unidirectional feature aggregation strategy to reduce the complexity of model calculation, and improve the real-time performance of the system through asynchronous dual-stream inference mechanism. The experimental results show that the dual -flow model based on the dual lightweight strategy shows better detection robustness under extremely dark light conditions: mAP@0.5 is upgraded from 0.314 of the single - stream RGB model to 0.547, mAP@0.5:0.95 increased from 0.200 to 0.359; Compared with the dual -flow RGB IR baseline model, the number of parameters and FLOPs decreased by 37.45% and 24.57% respectively. Under the asynchronous inference mechanism, the effective frame rate of the system reaches 72.09 FPS. It can be seen that the dual lightweight strategy achieves a better balance between detection performance, model complexity and real-time performance, providing a feasible solution for in -vehicle driver fatigue detection in low-light environments.

Abstract 13 | PDF Downloads 10

References

  • [1] Fu, B., Boutros, F., Lin, C. -T., & Damer, N. (2024). A survey on drowsiness detection: Modern applications and methods. IEEE Transactions on Intelligent Vehicles, 9 (11), 7279 –7300. https://doi.org/10.1109/TIV.2024.3395889
  • [2] Ayas, S., Donmez, B., & Tang, X. (2024). Drowsiness mitigation through driver state monitoring systems: A scoping review. Human Factors, 66(9), 2218–2243. https://doi.org/10.1177/00187208231208523
  • [3] Jiao, Y., Zhang, Y., Liu, W., & Jiao, Z. (2026). Detecting driver sleepiness from physiological indicators using a CNN -LSTM self-attention model. IEEE Journal of Biomedical and Health Informatics, 30 (5), 4082–4095. https://doi.org/10.1109/JBHI.2025.3629974
  • [4] V. P. B., & Chinara, S. (2021). Automatic classification methods for detecting drowsiness using wavelet packet transform extracted time -domain features from single -channel EEG signal. Journal of Neuroscience Methods, 347, Article 108927. https://doi.org/10.1016/j.jneumeth.2020.108927
  • [5] Baccour, M. H., Driewer, F., Schäck, T., & Kasneci, E. (2022). Comparative analysis of vehicle -based and driver -based features for driver drowsiness monitoring by support vector machines. IEEE Transactions on Intelligent Transportation Systems, 23 (12), 23164 –23178. https://doi.org/10.1109/TITS.2022.3207965
  • [6] Ganguly, B., Dey, D., & Munshi, S. (2025). An attention deep learning framework -based drowsiness detection model for intelligent transportation systems. IEEE Transactions on Intelligent Transportation Systems, 26(4), 4517–4527. https://doi.org/10.1109/TITS.2025.3544138
  • [7] Khan, M. A., Malik, S. A., & Ullah, H. (2022). Intelligent driver drowsiness detection for traffic safety based on multi CNN deep model and facial subsampling. IEEE Transactions on Intelligent Transportation Systems, 23(10), 19743–19752. https://doi.org/10.1109/TITS.2021.3134222
  • [8] Aprilia, S. J. N., & Fitrianah, D. (2024). Drowsiness detection in low -light conditions using YOLO and CLAHE augmentation for real -time applications. In Proceedings of the 2024 7th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (pp. 742 –746). https://doi.org/10.1109/ISRITI64779.2024.10963648
  • [9] Kim, K.-J., Baek, J. W., Lim, K. -T., & Shin, M. (2021). Low -cost real-time driver drowsiness detection based on convergence of IR images and EEG signals. In Proceedings of the 2021 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (pp. 362 –367). https://doi.org/10.1109/ICAIIC51459.2021.9415193
  • [10] Gowda, M. S., Talasila, V., Umar, S. A. R., & Alva, R. (2024). A multimodal approach to detect driver drowsiness. In Proceedings of the 2024 5th International Conference on Circuits, Control, Communication and Computing (I4C) (pp. 554–560). https://doi.org/10.1109/I4C62240.2024.10748433
  • [11] Wang, S., Sun, G., Dong, L., & Zheng, B. (2026). CARNet: Cross-attention guided feature reconstruction for RGB -T object detection. Expert Systems with Applications, 298 , Article 129865. https://doi.org/10.1016/j.eswa.2025.129865
  • [12] Palo, P., Nayak, S., Gupta, K., & Uttarkabat, S. (2025). A three -stage multimodal approach to driver activity recognition with RGB and infrared video analysis. In Proceedings of the 2025 IEEE International Conference on Imaging Systems and Techniques (IST). https://doi.org/10.1109/IST66504.2025.11268357
  • [13] Herath, D., Abeyrathne, C., & Jayaweera, P. (2025). Vision -based driver drowsiness monitoring: Comparative analysis of YOLOv5–v11 models. arXiv. https://arxiv.org/abs/2509.17498
  • [14] Dou, X., Wang, T., & Shao, S. (2023). A lightweight YOLOv5 model integrating GhostNet and attention mechanism. In Proceedings of the 2023 4th International Conference on Computer Vision, Image and Deep Learning (CVIDL) (pp. 348–352). https://doi.org/10.1109/CVIDL58838.2023.10166155
  • [15] Saxena, S., Angel, Khurana, M., Tiwari, S., & Shakya, H. K. (2026). Low -light driver drowsiness detection for real -time safety assistance using dual attention mechanisms in deep learning model. Scientific Reports, 14(1), Article 44442. https://doi.org/10.1038/s41598-026-44442-3
  • [16] Liang, X., Yao, W., Fang, X., & Zhang, C. (2024). FMIF: Facial multi -feature information fusion for driver fatigue detection. Signal, Image and Video Processing, 19 , 121. https://doi.org/10.1007/s11760- 024-03573-8
  • [17] Sun, Y., Meng, Y., Wang, Q., Tang, M., Shen, T., & Wang, Q. (2024). Visible and infrared image fusion for object detection: A survey. In International Conference on Image, Vision and Intelligent Systems (ICIVIS) (pp. 236–248). Springer. https://doi.org/10.1007/978-981-97-0855-0_24
  • [18] Bouraffa, Y., Benzaoui, A., & Bouridane, A. (2026). Feature-disentangling RGB-NIR fusion network for remote driver physiological measurement. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 657 –666). https://doi.org/10.1109/WACV57317.2026.00400
  • [19] Sun, L., Wang, W., Yuan, T., Mi, L., Dai, H., Liu, Y., & Fu, X. (2024). BiSwift: Bandwidth orchestrator for multi -stream video analytics on edge. In Proceedings of IEEE INFOCOM (pp. 1 –10). https://doi.org/10.48550/arXiv.2312.15740
  • [20] Pereira, D., Ghosh, S., & Dey, S. (2024). DRL -based multi-stream scheduling of inference pipelines on edge devices. In Proceedings of the IEEE International Conference on VLSI Design (VLSID) (pp. 324– 329). https://doi.org/10.1109/VLSID60093.2024.00060

Article Sidebar

PDF
Published
Apr 15, 2026
Conference Proceedings Volume
Vol. 20 (2026): Proceedings of 2026 International Conference on Materials Science, Energy Engineering and Automation Control (IC-MSEAC 2026)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Shao, Weixuan. “A Driver Fatigue Detection Mechanism Based on a Lightweight RGB-IR Fusion Algorithm”. Exploring Science Academic Conference Series, vol. 20, Apr. 2026, pp. 41-55, https://doi.org/10.70267/mseac.20264155.