Skip to main content
SpringerLink
Log in

Multi-scale convolution underwater image restoration network

  • Short Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Due to the complex underwater imaging environment and illumination conditions, underwater images have some quality degradation problems, such as low contrast, color distortion, texture blur and uneven illumination, which seriously restrict the application in underwater work. In order to solve these problems, we proposed a multi-scale feature fusion CNN based on underwater imaging model in this paper called Multi-Scale Convolution Underwater Image Restoration Network (MSCUIR-Net). Unlike most previous models that estimated the background light and transmittance, respectively, our model unifies the two parameters into one, predicts the univariate linear physical model through lightweight CNN, and directly generates end-to-end clean images. Based on the underwater imaging model, we synthesized the underwater image training set can simulate the shallow water to deep water environment. Then, we do experiments on synthetic images and real underwater images, and prove the superiority of this method through image evaluation indexes. The experimental results show that MSCUIR-Net has a good effect on underwater image restoration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The error of a sample defined on a single sample.

  2. The average of all sample errors defined on the whole training set, the average of all loss function values.

  3. In the process of back propagation, the gradient is getting larger and larger, and each layer needs to update a large weight, resulting in the divergence of results.

References

  1. Hegerl, G.C.: Ocean science. warming the world’s oceans. Science 309(5732), 254–255 (2005)

    Article  Google Scholar 

  2. Cai, L., et al.: Multi-AUV collaborative target recognition based on transfer-reinforcement learning. IEEE Access 11, 39273–392114 (2020)

    Article  Google Scholar 

  3. Benoist, N.M.A., et al.: Monitoring mosaic biotopes in a marine conservation zone by autonomous underwater vehicle. Conserv. Biol. 33(5), 1174–11116 (2019)

    Article  Google Scholar 

  4. Shi, P., et al.: A detection and classification approach for underwater dam cracks. Struct. Health Monit 15(5), 541–554 (2016)

    Article  Google Scholar 

  5. Sun, Y., et al.: Deep submergence rescue vehicle docking based on parameter adaptive control with acoustic and visual guidance. Int. J. Adv. Robot. Syst. 17(2), 172911111420919970 (2020)

    Article  Google Scholar 

  6. Ancuti, C., Ancuti, C.O., Haber T., Bekaert P.: Enhancing underwater images and videos by fusion. In: 2012 IEEE conference on Computer vision and pattern recognition. IEEE, pp. 111–1111 (2012)

  7. Ghani, A.S.A., Isa, N.A.M.: Underwater image quality enhancement through integrated color model with rayleigh distribution. Appl. Soft Comput. 27, 219–230 (2015)

    Article  Google Scholar 

  8. Johnson一Roberson, M., Bryson, M., Friedman, A., et al.: High resolution underwater robotic vision based mapping and three一dimensional reconstruction for archaeology. J. Field Robot. 34(4), 625–643 (2017)

    Article  Google Scholar 

  9. Ghani, A.S.A., Isa, N.A.M.: Enhancement of low quality underwater image through integrated global and local contrast correction. Appl. Soft Comput. 37, 332–344 (2015)

    Article  Google Scholar 

  10. Mercado, MA., Ishii, K., Ahn, J.: Deep-sea image enhancement using multi-scale retinex with reverse color loss for autonomous underwater vehicles. In: //Proceedings of IEEE conference on oceans, (2017)

  11. Shahrizan, A., Ghani, A., Fakhri A, et al.: Integration of enhanced background filtering and wavelet fusion for high visibility and detection rate of deep sea underwater image of underwater vehicle. In: //Proceedings of international conference on information and communication technology, (2017)

  12. Treibitz, T., Schechner, YY: Instant 3Descatter. In: // Proceedings of IEEE computer society conference on computer vision and pattern recognition, 11161–111611 (2006)

  13. Treibitz, T., Schechner, Y.Y.: Active polarization descattering. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 3115–3399 (2009)

    Article  Google Scholar 

  14. Li, C.Y., Guo, J.C., Cong, R.M., et al.: Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25(12), 5664–5677 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  15. Dai, C.G., Lin, M.X., Wu, X.J., et al.: Single underwater image restoration by decomposing curves of attenuating color. Opt. Laser Technol. (2010). https://doi.org/10.1016/j.optlastec.2019.105947

    Article  Google Scholar 

  16. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)

    Article  Google Scholar 

  17. Ding, X., Wang, Y., Liang, Z., et al. Towards underwater image enhancement using super-resolution convolutional neural networks. In: //international conference on internet multimedia computing and Service. Springer, Singapore, 479–4116 (2017)

  18. Wang, Y., Zhang, J., Cao., Y., et al.: A deep CNN method for underwater image enhancement. In: // IEEE international conference on image processing. IEEE, 2017:13112–13116

  19. Wang, K.Y., Hu, Y., Chen, J., et al.: Underwater image restoration based on a parallel convolutional neural network. Remote Sens. 11(13), 1591 (2019)

    Article  Google Scholar 

  20. Jaffe, J.S.: Computer modeling and the design of optimal underwater imaging systems. J. Ocean. Eng. 15(2), 101–111 (1990)

    Article  Google Scholar 

  21. Xu Yan, Zeng Xiangbo.: Underwater image restoration based on a priori and inverse filtering of red dark channel (in chinses). Progress in laser and optoelectronics, 20111, 55 (2): 215–222

  22. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: European conference on computer vision, Springer, 2012, pp. 746–760

  23. Li, B., Peng, X., Wang, Z., et al.: AOD-Net: All-in-One Dehazing Network. In: // 2017 IEEE international conference on computer vision (ICCV). IEEE, 2017

  24. Nair, V., Hinton GE.: Rectified linear units improve restricted boltzmann machines. Icml. 2010:1107–1114

  25. Rafferty, J., Shellito, P., Hyman, N.H., et al.: Practice parameters for sigmoid diverticulitis. Dis. Colon Rectum 49(7), 939–944 (2006)

    Article  Google Scholar 

  26. Fan, E.: Extended tanh-function method and its applications to nonlinear equations. Phys. Lett. A 277(4–5), 212–2111 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  27. Berman, D., Treibitz, T., Avidan, S.: Diving into haze-lines: color restoration of underwater images. In: Proc. british machine vision conference (BMVC), vol. 1, no. 2, 2017

  28. Li, C., Anwar, S., Porikli, F.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 911, 1070311 (2020)

    Google Scholar 

  29. Nayak, GS., Nayak, D.: Back propagation algorithm. (2013)

  30. Ketkar, N.: Stochastic gradient descent, pp. 113–132. Deep learning with Python. Apress, Berkeley, CA (2017)

    Google Scholar 

  31. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 51117–51911 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  32. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  33. Drews, P.L., Nascimento, E.R., Botelho, S.S., Campos, M.F.M.: Underwater depth estimation and image restoration based on single images. IEEE Comput. Graphics Appl. 36(2), 24–35 (2016)

    Article  Google Scholar 

  34. Li, C.-Y., Guo, J.-C., Cong, R.-M., Pang, Y.-W., Wang, Bo.: Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25(12), 5664–5677 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  35. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Oceanic Eng. 41(3), 541–551 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (No. 51005142), the Innovation Program of Shanghai Municipal Education Commission (No.14YZ010), and the Natural Science Foundation of Shanghai (No. 14ZR1414900, No.19ZR1419300) for providing financial support for this work.

Author information

Authors and Affiliations

  1. School of Mechatronic Engineering and Automation, Shanghai University, No. 99 Shangda Road, Shanghai, China

    Zhijie Tang, Jianda Li, Jingke Huang, Zhanhua Wang & Zhihang Luo

Authors
  1. Zhijie Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianda Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingke Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanhua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhijie Tang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Z., Li, J., Huang, J. et al. Multi-scale convolution underwater image restoration network. Machine Vision and Applications 33, 85 (2022). https://doi.org/10.1007/s00138-022-01337-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01337-3

Keywords

Search

Navigation