Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Multi-modal neural networks with multi-scale RGB-T fusion for semantic segmentation

A novel deep-learning-based method for semantic segmentation of RGB and thermal images is introduced. The proposed method employs a novel neural network design for multi-modal fusion based on multi-resolution patch processing. A novel decoder module is introduced to fuse the RGB and thermal features extracted by separate encoder streams. Experimental results on synthetic and real-world data demonstrate the efficiency of the proposed method compared with state-of-the-art methods.

References

    1. 1)
    2. 2)
      • 6. Zhu, J.-Y., Park, T., Isola, P., et al: ‘Unpaired image-to-image translation using cycle-consistent adversarial networks’. IEEE Int. Conf. Computer Vision, Venice, Italy, October 2017, pp. 22422251.
    3. 3)
    4. 4)
      • 12. Hwang, S., Park, J., Kim, N., et al: ‘Multispectral pedestrian detection: benchmark dataset and baseline’. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, October 2015, pp. 10371045.
    5. 5)
      • 2. Ha, Q., Watanabe, K., Karasawa, T., et al: ‘MFNet: towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes’. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, Vancouver, BC, Canada, December 2017, pp. 51085115.
    6. 6)
      • 11. Cordts, M., Omran, M., Ramos, S., et al: ‘The cityscapes dataset for semantic urban scene understanding’. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June 2016, pp. 32133223.
    7. 7)
      • 8. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, June 2016, pp. 770778.
    8. 8)
      • 7. Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., et al: ‘High-resolution image synthesis and semantic manipulation with conditional GANs’. IEEE/CV Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June 2018, pp. 87988807.
    9. 9)
    10. 10)
    11. 11)
      • 10. Xie, S., Girshick, R., Dollár, P., et al: ‘Aggregated residual transformations for deep neural networks’. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, July 2017, pp. 14921500.
    12. 12)
      • 5. Isola, P., Zhu, J.-Y., Zhou, T., et al: ‘Image-to-image translation with conditional adversarial networks’. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, HI, USA, July 2017, pp. 11251134.
http://iet.metastore.ingenta.com/content/journals/10.1049/el.2020.1635
Loading

Related content

content/journals/10.1049/el.2020.1635
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address