March 19, 2025

ikayaniaamirshahzad@gmail.com

VSS-SpatioNet: a multi-scale feature fusion network for multimodal image integrations


  • Yang, S., Qin, H., Dai, Y., Yan, X. & López-Baldomero, A. B. Temperature distribution inversion in infrared multispectral imaging based on ensemble network. Opt. Lett. 49, 5163–5166 (2024).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Wang, T. et al. Remote referencing strategy for high-resolution coded ptychographic imaging. Opt. Lett. 48, 485–488 (2023).

    Article 
    ADS 
    PubMed 
    MATH 

    Google Scholar
     

  • Umirzakova, S., Ahmad, S., Khan, L. U. & Whangbo, T. Medical image super-resolution for smart healthcare applications: A comprehensive survey. Inform. Fusion. 103, 102075 (2024).

    Article 
    MATH 

    Google Scholar
     

  • Jack, C. R. Jr et al. The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging. 27, 685–691 (2008).

    Article 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Schwenck, J. et al. Advances in PET imaging of cancer. Nat. Rev. Cancer. 23, 474–490 (2023).

    Article 
    CAS 
    PubMed 
    MATH 

    Google Scholar
     

  • Liu, C. et al. Multi-focus image fusion based on Spatial frequency in discrete cosine transform domain. IEEE. Signal. Process. Lett. 22, 220–224 (2014).

    MATH 

    Google Scholar
     

  • Yang, B., Yang, C. & Huang, G. Efficient image fusion with approximate sparse representation. Int. J. Wavelets Multiresolut Inf. Process. 14, 1650024 (2016).

    Article 
    MathSciNet 
    MATH 

    Google Scholar
     

  • Zhou, Y. et al. Real-infraredSR: real-world infrared image super-resolution via thermal imager. Opt. Express. 31, 36171–36187 (2023).

    Article 
    ADS 
    PubMed 

    Google Scholar
     

  • Kuncheva, L. I. & Faithfull, W. J. PCA feature extraction for change detection in multidimensional unlabeled data. IEEE Trans. Neural Networks Learn. Syst. 25, 69–80 (2013).

    Article 
    MATH 

    Google Scholar
     

  • Ermachenkova, M. K., Malashin, R. O. & Boiko, A. A. Neural network training for thermal image classification based on visible spectrum images. J. Opt. Technol. 90, 590–600 (2023).

    Article 

    Google Scholar
     

  • Zhang, H., Xu, H., Tian, X., Jiang, J. & Ma, J. Image fusion Meets deep learning: A survey and perspective. Inform. Fusion. 76, 323–336 (2021).

    Article 
    MATH 

    Google Scholar
     

  • Zhang, Y., Wu, Z., Xu, Y. & Huangfu, J. Dual-branch fusion model for lensless imaging. Opt. Express. 31, 19463–19477 (2023).

    Article 
    ADS 
    PubMed 

    Google Scholar
     

  • Ding, J., Du, Y., Li, W., Pei, L. & Cui, N. LG-Diff: learning to follow local class-regional guidance for nearshore image cross-modality high-quality translation. Inform. Fusion. 117, 102870 (2025).

    Article 

    Google Scholar
     

  • LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324. (1998).

  • Goodfellow, I. J. et al. Generative adversarial networks. Preprint at arXiv, (2014). https://doi.org/10.48550/arXiv.1406.2661

  • Ng, A. Sparse autoencoder. CS294A Lecture Notes. 72, 1–19 (2011).

    MATH 

    Google Scholar
     

  • Vaswani, A. et al. Attention is all you need. Preprint arXiv. https://doi.org/10.48550/arXiv.1706.03762 (2017).

    Article 
    MATH 

    Google Scholar
     

  • Pan, L., Zhou, X., Shi, R., Zhang, J. & Yan, C. Cross-modal feature extraction and integration based RGBD saliency detection. Image Vis. Comput. 101, 103964 (2020).

    Article 
    MATH 

    Google Scholar
     

  • Li, Y., Zhang, Y., Zeng, H., He, J. & Guo, J. Spatial relaxation transformer for image super-resolution. J. King Saud Univ. – Comput. Inform. Sci. 36 (7), 102150 (2024).

    Article 
    MATH 

    Google Scholar
     

  • Gu, A. & Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. Preprint arXiv. https://doi.org/10.48550/arXiv.2312.00752 (2023).

    Article 
    PubMed 
    MATH 

    Google Scholar
     

  • Liu, Y. et al. VMamba: Visual state space model. Preprint at arXiv, (2024). https://doi.org/10.48550/arXiv.2401.10166

  • Zhou, K., Zhang, M., Wang, H. & Tan, J. Ship detection in SAR images based on multi-scale feature extraction and adaptive feature fusion. Remote Sens. 14, 755 (2022).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Roberts, J. W., Van Aardt, J. A. & Ahmed, F. B. Assessment of image fusion procedures using entropy, image quality, and multispectral classification. J. Appl. Remote Sens. 2, 023522 (2008).

    Article 
    MATH 

    Google Scholar
     

  • Ma, K., Zeng, K. & Wang, Z. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24, 3345–3356 (2015).

    Article 
    ADS 
    MathSciNet 
    PubMed 
    MATH 

    Google Scholar
     

  • Aslantas, V. & Bendes, E. A new image quality metric for image fusion: the sum of the correlations of differences. AEU – Int. J. Electron. Commun. 69, 1890–1896 (2015).

    Article 
    MATH 

    Google Scholar
     

  • Wang, J. et al. Advancing infrared and visible image fusion with an enhanced multi-scale encoder and attention-based networks. iScience 27, 110915 (2024).

    Article 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Xie, X. et al. FusionMamba: dynamic feature enhancement for multimodal image fusion with Mamba. Preprint at arXiv, (2024). https://doi.org/10.48550/arXiv.2404.09498

  • Kumar, B. K. S. Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform. Signal. Image Video Process. 7, 1125–1143 (2013).

    Article 
    MATH 

    Google Scholar
     

  • Liu, Y., Chen, X., Ward, R. K. & Wang, Z. J. Image fusion with convolutional sparse representation. IEEE. Signal. Process. Lett. 23, 1882–1886 (2016).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Li, H., Wu, X. J. & Kittler, J. Infrared and visible image fusion using a deep learning framework. In Proceedings of the 24th International Conference on Pattern Recognition (ICPR), IEEE, pp. 2705–2710. (2018).

  • Li, H. & Wu, X. J. DenseFuse: A fusion approach to infrared and visible images. IEEE Trans. Image Process. 28, 2614–2623 (2018).

    Article 
    ADS 
    MathSciNet 
    MATH 

    Google Scholar
     

  • Zhang, Y. et al. IFCNN: A general image fusion framework based on convolutional neural network. Inform. Fusion. 54, 99–118 (2020).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Ding, J. et al. Novel pipeline integrating cross-modality and motion model for nearshore multi-object tracking in optical video surveillance. IEEE Trans. Intell. Transp. Syst. 25 (9), 12464–12476 (2024).

    Article 

    Google Scholar
     

  • Li, H., Wu, X. J. & Durrani, T. NestFuse: an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models. IEEE Trans. Instrum. Meas. 69, 9645–9656 (2020).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Li, H., Wu, X. J. & Kittler, J. RFN-Nest: an end-to-end residual fusion network for infrared and visible images. Inform. Fusion. 73, 72–86 (2021).

    Article 
    MATH 

    Google Scholar
     

  • Dong, Y., Chen, Z., Li, Z. & Gao, F. A Multi-Branch Multi-Scale deep learning image fusion algorithm based on densenet. Appl. Sci. 12 (21), 10989 (2022).

    Article 
    CAS 
    MATH 

    Google Scholar
     

  • Li, X., Li, X. & Liu, W. CBFM: contrast balance infrared and visible image fusion based on contrast-Preserving guided filter. Remote Sens. 15 (12), 2969 (2023).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Liu, W., Tan, H., Cheng, X. & Li, X. ESFuse: weak edge structure perception network for infrared and visible image fusion. Electronics 13 (20), 4115 (2024).

    Article 
    MATH 

    Google Scholar
     

  • Chen, L. & Han, J. Infrared and visible image fusion using salient decomposition based on a generative adversarial network. Appl. Opt. 60, 7017–7026 (2021).

    Article 
    ADS 
    PubMed 
    MATH 

    Google Scholar
     

  • Ma, J., Yu, W., Liang, P., Li, C. & Jiang, J. FusionGAN: A generative adversarial network for infrared and visible image fusion. Inform. Fusion. 48, 11–26 (2019).

    Article 
    CAS 
    MATH 

    Google Scholar
     

  • Li, K., Qi, M., Zhuang, S., Liu, Y. & Gao, J. Noise-aware infrared polarization image fusion based on salient prior with attention-guided filtering network. Opt. Express. 31, 25781–25796 (2023).

    Article 
    ADS 
    PubMed 

    Google Scholar
     

  • Vs, V., Valanarasu, J. M. J., Oza, P. & Patel, V. M. Image fusion transformer. In Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), pp. 3566–3570. IEEE. (2022).

  • Xu, H., Ma, J., Jiang, J., Guo, X. & Ling, H. U2Fusion: A unified unsupervised image fusion network. IEEE Trans. Pattern Anal. Mach. Intell. 44, 502–518 (2020).

    Article 
    MATH 

    Google Scholar
     

  • Ruan, J. & Xiang, S. VM-UNet: vision Mamba UNet for medical image segmentation. Preprint arXiv. https://doi.org/10.48550/arXiv.2402.02491 (2024).

    Article 
    MATH 

    Google Scholar
     

  • Liu, X., Zhang, C. & Zhang, L. Vision Mamba: A comprehensive survey and taxonomy. Preprint at arXiv, (2024). https://doi.org/10.48550/arXiv.2405.04404

  • Xu, R., Li, X. & Zhang, Y. and others. A survey on Vision Mamba: Models, applications and challenges. Preprint at arXiv, (2024). https://doi.org/10.48550/arXiv.2404.18861

  • Huang, S., Lu, Y., Wang, W. & others Multi-scale guided feature extraction and classification algorithm for hyperspectral images. Sci. Rep. 11, 18396 (2021).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Wang, D., Liu, J., Liu, R. & Fan, X. An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection. Inform. Fusion. 98, 101828 (2023).

    Article 
    MATH 

    Google Scholar
     

  • Qian, Y., Liu, G., Tang, H., Xing, M. & Chang, R. BTSFusion: fusion of infrared and visible image via a mechanism of balancing texture and salience. Opt. Lasers Eng. 173, 107925 (2024).

    Article 

    Google Scholar
     

  • Duan, W., Zhang, L., Colman, J., Gulli, G. & Ye, X. Multi-modal brain segmentation using hyper-fused convolutional neural network. In Machine Learning in Clinical Neuroimaging. MLCN 2021, A. Abdulkadir, et al., eds. (Springer, Cham), Lecture Notes in Computer Science, vol. 13001. (2021).

  • Zamir, S. W. et al. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5728–5739. (2022).

  • Huo, X. et al. HiFuse: hierarchical multi-scale feature fusion network for medical image classification. Biomed. Signal Process. Control. 87, 105534 (2024).

    Article 
    MATH 

    Google Scholar
     

  • Lu, D., Cheng, S., Wang, L. & others Multi-scale feature progressive fusion network for remote sensing image change detection. Sci. Rep. 12, 11968 (2022).

    Article 
    ADS 
    CAS 
    PubMed 
    PubMed Central 
    MATH 

    Google Scholar
     

  • Yi, Y., Li, Y., Du, J. & Wang, S. An infrared and visible image fusion method based on improved GAN with dropout layer. In The Proceedings of the 18th Annual Conference of China Electrotechnical Society. ACCES 2023, Q. Yang, Z. Li, and A. Luo, eds. (Springer, Singapore), Lecture Notes in Electrical Engineering, vol. 1168. (2024).

  • Pérez-García, F., Sparks, R. & Ourselin, S. TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Comput. Methods Programs Biomed. 208, 106236 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Qu, G., Zhang, D. & Yan, P. Information measure for performance of image fusion. Electron. Lett. 38, 1 (2002).

    Article 
    MATH 

    Google Scholar
     

  • Kumar, B. K. S. Image fusion based on pixel significance using cross bilateral filter. Signal. Image Video Process. 9, 1193–1204 (2015).

    Article 
    MATH 

    Google Scholar
     

  • Zhou, Z., Zhang, B. & others Fusion of infrared and visible images for night-vision context enhancement. Appl. Opt. 55, 23 (2016).

    Article 
    MATH 

    Google Scholar
     

  • Zhou, Z., Dong, M., Wang, P. & others Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Inform. Fusion. 30, 15–26 (2016).

    Article 
    MATH 

    Google Scholar
     

  • Yao, Y., Guo, P. & Xin, X. and others. Image fusion by hierarchical joint sparse representation. Cognitive Computation 6, 281–292. (2014).

  • Li, H., Wu, X. J. & Kittler, J. MDLatLRR: A novel decomposition method for infrared and visible image fusion. IEEE Trans. Image Process. 29, 4733–4746 (2020).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Xu, H., Ma, J. & Zhang, X. P. MEF-GAN: Multi-exposure image fusion via generative adversarial networks. IEEE Trans. Image Process. 29, 7203–7216 (2020).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Zhang, H., Xu, H., Xiao, Y., Guo, X. & Ma, J. Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity. In Proceedings of the AAAI Conference on Artificial Intelligence 34, 12797–12804. (2020).

  • Li, W. et al. Structure-aware image fusion. Optik 172, 1–11. (2014). Hybrid DDCT-PCA based multi-sensor image fusion. Journal of Optics 43, 48–61. (2018).

  • Li, W., Kang, X. & Zuo, W. and others. Structure-aware image fusion. Optik 172, 1–11. (2018).

  • Rao, Y. J. In-fibre Bragg grating sensors. Meas. Sci. Technol. 8, 355 (1997).

    Article 
    ADS 
    CAS 
    MATH 

    Google Scholar
     

  • Ma, J., Yu, W., Liang, P. & others DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE Trans. Image Process. 29, 4980–4995 (2020).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Xydeas, C. S. & Petrovic, V. Objective image fusion performance measure. Electron. Lett. 36, 308–309 (2000).

    Article 
    ADS 
    MATH 

    Google Scholar
     

  • Liu, Z. et al. Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Trans. Pattern Anal. Mach. Intell. 34 (1), 94–109 (2012).

    Article 
    CAS 
    PubMed 
    MATH 

    Google Scholar
     

  • Estevez, P. A., Tesmer, M., Perez, C. A. & Zurada, J. A. Normalized mutual information feature selection. IEEE Trans. Neural Networks. 20 (1), 189–201 (2009).

    Article 
    PubMed 

    Google Scholar
     

  • Cui, G., Feng, H., Xu, Z., Li, Q. & Chen, Y. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition. Opt. Commun. 341, 199–209 (2015).

    Article 
    CAS 
    MATH 

    Google Scholar
     

  • Zheng, Y., Essock, E. A., Hansen, B. C. & Haun, A. M. A new metric based on extended Spatial frequency and its application to DWT-based fusion algorithms. Inform. Fusion. 8 (2), 177–192 (2007).

    Article 
    MATH 

    Google Scholar
     

  • Chen, L., Lin, J., Bian, Q., Liu, Y. & Zhou, J. Inhomogeneous illumination image enhancement under extremely low visibility condition. Appl. Sci. 14, 10111 (2024).

    Article 
    CAS 

    Google Scholar
     

  • Intharachathorn, K., Jareemit, D. & Watcharapinchai, S. Potential use of an extended-distance thermal imaging camera for the assessment of thermal comfort in multi-occupant spaces. Build. Environ. 246, 110949 (2023).

    Article 
    MATH 

    Google Scholar
     

  • Ding, J. et al. SeaTrack: rethinking observation-centric SORT for robust nearshore multiple object tracking. Pattern Recogn. 159, 111091 (2025).

    Article 
    MATH 

    Google Scholar
     

  • Pan, Y., Yao, T., Li, Y. & Mei, T. X-linear attention networks for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10971–10980. (2020).

  • Li, Y., Zhao, J., Lv, Z. & Li, J. Medical image fusion method by deep learning. Int. J. Cogn. Comput. Eng. 2, 21–29 (2021).

    MATH 

    Google Scholar
     

  • Saragadam, V. et al. Foveated thermal computational imaging prototype using all-silicon meta-optics. Optica 11, 18–25 (2024).

    Article 
    ADS 
    CAS 

    Google Scholar
     

  • Luo, Y. & Luo, Z. Infrared and visible image fusion: methods, datasets, applications, and prospects. Appl. Sci. 13, 10891 (2023).

    Article 
    CAS 
    MATH 

    Google Scholar
     

  • Zhang, X. & Demiris, Y. Visible and infrared image fusion using deep learning. IEEE Trans. Pattern Anal. Mach. Intell. 45, 10535–10554 (2023).

    Article 
    PubMed 
    MATH 

    Google Scholar
     



  • Source link

    Leave a Comment