Multi-Window Detail Enhancement Based Infrared and Visible Image Fusion
Main Article Content
Keywords
infrared and visible image fusion, multi-window attention, deep learning
Abstract
The goal of infrared and visible image fusion is to combine complementary information from infrared and visible images of the same scene to generate a high-quality composite image that integrates the advantages of both modalities. Although many current fusion methods achieve satisfactory results, they still suffer from limitations such as insufficient feature resolution extraction from infrared images and inadequate texture information extraction from visible images. This paper proposes a novel fusion method designed to enhance the resolution, detail preservation, and visual consistency of the fused image. The method integrates multi-window detail enhancement with multi-layer residual connections, employing a detail selector and a global feature extractor to separately capture high-frequency and low-frequency features from the infrared and visible images. Experimental results demonstrate that, compared to existing approaches, the proposed method achieves superior fusion quality and better preservation of image details, providing higher-quality data for subsequent image processing tasks.
References
- Liu, K., Li, M., Chen, C., Rao, C., Zuo, E., Wang, Y., Yan, Z., Wang, B., Chen, C., & Lv, X. (2024). DSFusion: Infrared and visible image fusion method combining detail and scene information. Pattern Recognition, 154, Article 110633. https://doi.org/10.1016/J.PATCOG.2024.110633
- Liu, X., Huo, H., Li, J., Pang, S., & Zheng, B. (2024). A semantic-driven coupled network for infrared and visible image fusion. Information Fusion, 108, Article 102352. https://doi.org/10.1016/J.INFFUS.2024.102352
- Liu, Y., Liu, S., & Wang, Z. (2015). A general framework for image fusion based on multi-scale transform and sparse representation. Information Fusion, 24, 147-164. https://doi.org/10.1016/J.INFFUS.2014.09.004
- Ma, J., Ma, Y., & Li, C. (2019). Infrared and visible image fusion methods and applications: A survey. Information Fusion, 45, 153-178. https://doi.org/10.1016/J.INFFUS.2018.02.004
- Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019). FusionGAN: A generative adversarial network for infrared and visible image fusion. Information Fusion, 48, 11-26. https://doi.org/10.1016/J.INFFUS.2018.09.004
- Park, S. C., Park, M. K., & Kang, M. G. (2003). Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine, 20(3), 21-36. https://doi.org/10.1109/MSP.2003.1203207
- Pizurica, A., Philips, W., Lemahieu, I., & Acheroy, M. (2003). A versatile wavelet domain noise filtration technique for medical imaging. IEEE Transactions on Medical Imaging, 22(3), 323-331. https://doi.org/10.1109/TMI.2003.809588
- Tang, L., Yuan, J., & Ma, J. (2022). Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network. Information Fusion, 82, 28-42. https://doi.org/10.1016/J.INFFUS.2021.12.004
- Tang, W., He, F., Liu, Y., Duan, Y., & Si, T. (2023). DATFuse: Infrared and visible image fusion via dual attention transformer. IEEE Transactions on Circuits and Systems for Video Technology, 33(7), 3159-3172. https://doi.org/10.1109/TCSVT.2023.3234340
- Toet, A. (1989). Image fusion by a ration of low-pass pyramid. Pattern Recognition Letters, 9(4), 245-253. https://doi.org/10.1016/0167-8655(89)90003-2
- Toet, A. (2017). The TNO multiband image data collection. Data in Brief, 15, 249-251. https://doi.org/10.1016/J.DIB.2017.09.038
- Wang, Z., Wang, J., Wu, Y., Xu, J., & Zhang, X. (2022). UNFusion: A unified multi-scale densely connected network for infrared and visible image fusion. IEEE Transactions on Circuits and Systems for Video Technology, 32(6), 3360-3374. https://doi.org/10.1109/TCSVT.2021.3109895
- Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2022). U2Fusion: A unified unsupervised image fusion network. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(1), 502-518. https://doi.org/10.1109/TPAMI.2020.3012548
- Zhang, H., Yuan, J., Tian, X., & Ma, J. (2021). GAN-FM: Infrared and visible image fusion using GAN with full-scale skip connection and dual markovian discriminators. IEEE Transactions on Computational Imaging, 7, 1134-1147. https://doi.org/10.1109/TCI.2021.3119954
- Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020). IFCNN: A general image fusion framework based on convolutional neural network. Information Fusion, 54, 99-118. https://doi.org/10.1016/J.INFFUS.2019.07.011
- Zhao, W., Xie, S., Zhao, F., He, Y., & Lu, H. (2023). MetaFusion: Infrared and visible image fusion via meta-feature embedding from object detection [Paper presentation]. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.
- Zhou, Y., Fu, J., Chen, Z., Zhuge, F., Wang, Y., Yan, J., Ma, S., Xu, L., Yuan, H., Chan, M., Miao, X., He, Y., & Chai, Y. (2023). Computational event-driven vision sensors for in-sensor spiking neural networks. Nature Electronics, 6(11), 870-878. https://doi.org/10.1038/S41928-023-01055-2
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.