Skip to main content
SpringerLink
Log in

Multi-resolution depth image restoration

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Depth degradation caused by the conditions and environment of depth sensor hardware restricts its application potential, and this limitation cannot be avoided simply by improving the design of sensor. To overcome this limitation, we propose a multi-resolution depth image restoration method. Firstly, the sub-images of depth image and color image at different scales are obtained by multi-resolution analysis based on two-dimensional discrete wavelet transform. The multi-resolution joint bilateral filtering is then applied to the approximation low-frequency sub-image of the decomposed image. At the same time, using color-guided filtering method to restore high-frequency sub-images can effectively suppress edge artifacts without adding extra time burden. The high-quality output image is finally reconstructed using two-dimensional inverse discrete wavelet transform. A color guide image with rich edge information is introduced into the depth sub-image restoration to improve the depth image edge detail. Extensive experiments with synthetic and real datasets demonstrate that the proposed algorithm can effectively reduce additive Gaussian noise without losing sharp details in the noisy images and reduce the time consumption of depth image restoration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Qiu, D., Pang, J., Sun, W., et al.: Deep end-to-end alignment and refinement for time-of-flight RGB-D module. IEEE Int. Conf. Comput. Vis. 9994–10003 (2019)

  2. Yu, Y., Song, Y., Zhang, Y., et al.: A shadow repair approach for Kinect depth images. Asian Conf. Comput. Vis. 727, 615–626 (2012)

    Google Scholar 

  3. Asokan, A., Anitha, J.: Adaptive Cuckoo Search based optimal bilateral filtering for denoising of satellite images. ISA Trans. 100, 308–321 (2020)

    Article  Google Scholar 

  4. Anand, C.S., Sahambi, J.S.: MRI denoising using bilateral filter in redundant wavelet domain. IEEE Region 10 Conf. 1–6 (2008)

  5. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. Int. Conf. Comput. Vis., 839–846 (1998)

  6. He, K., Sun, J., Tang, X.: Guided image filtering. IEEE T. Pattern Anal. 35(6), 1397–1409 (2013)

    Article  Google Scholar 

  7. Zhang, J., Lin, G., Wu, L., et al.: Speckle filtering of medical ultrasonic images using wavelet and guided filter. Ultrasonics 65, 177–193 (2016)

    Article  Google Scholar 

  8. Na, Y., Zhao, L., Yang, Y., et al.: Guided filter-based images fusion algorithm for CT and MRI medical images. IET Image Process 12(1), 138–148 (2017)

    Article  Google Scholar 

  9. Diebel, J., Thrun, S.: An application of Markov random fields to range sensing. Adv. IEPS 18, 291–298 (2005)

    Google Scholar 

  10. Ferstl, D., Reinbacher, C., Ranftl, R., et al.: Image guided depth upsampling using anisotropic total generalized variation. Int. Conf. Comput. Vis. IEEE, 993–1000 (2013)

  11. Min, D., Choi, S., Lu, J., et al.: Fast global image smoothing based on weighted least squares. IEEE T. Image Process. 23(12), 5638–5653 (2014)

    Article  MathSciNet  Google Scholar 

  12. Yang, J., Ye, X., Li, K., Hou, C., et al.: Color-guided depth recovery from RGB-D data using an adaptive auto-regressive model. IEEE T Image Process. 23(8), 3443–3458 (2014)

    Article  Google Scholar 

  13. Liu, W., Chen, X., Yang, J., et al.: Robust color guided depth image restoration. IEEE T Image Process. 26(1), 315–327 (2016)

    Article  Google Scholar 

  14. Ham, B., Cho, M., Ponce, J.: Robust image filtering using joint static and dynamic guidance. Comput. Vis. Pattern Recogn. 7, 4823–4831 (2015)

    Google Scholar 

  15. Ham, B., Cho, M., Ponce, J.: Robust guided image filtering using non-convex potentials. IEEE T. Pattern Anal. 40(1), 192–207 (2018)

    Article  Google Scholar 

  16. Kim, Y., Ham, B., Oh, C., et al.: Structure selective depth super-resolution for RGB-D cameras. Trans. Image Process. 25(11), 5227–5238 (2016)

    Article  MathSciNet  Google Scholar 

  17. Liu, X., Zhai, D., Chen, R., et al.: Depth restoration from rgb-d data via joint adaptive regularization and thresholding on manifolds. IEEE T Image Process. 28(3), 1068–1079 (2019)

    Article  MathSciNet  Google Scholar 

  18. Liu, X., Zhai, D., Chen, R., et al.: Depth super-resolution via joint color-guided internal and external regularizations. IEEE T. Image Process. 28(4), 1636–1645 (2019)

    Article  MathSciNet  Google Scholar 

  19. Zuo, Y., Wu, Q., Zhang, J., et al.: Explicit edge inconsistency evaluation model for color-guided depth image enhancement. IEEE T Circ. Syst. Vid. 28(2), 439–453 (2018)

    Google Scholar 

  20. Li, Y., Huang, J.B., Ahuja, N., et al.: Deep joint image filtering’, in: European conference on computer vision. ECCV, 154–169 (2016)

  21. Zhao, L., Bai, H., Liang, J., et al.: Single depth image Super-resoluion with multiple residual dictionary learning and refinement. Int. Conf. Multimedia Expo. ICME, 739–744 (2017)

  22. Karthikeyan, P., Vasuki, S.: Multiresolution joint bilateral filtering with modified adaptive shrinkage for image denoising. Multimed. Tools Appl. 75(23), 1–18 (2016)

    Google Scholar 

  23. Esakkirajan, S., Vimalraj, C.T., Muhammed, R., et al.: Adaptive wavelet packet-based despeckling of ultrasound images with bilateral filter. Ultrasound Med. Biol. 39(12), 2463–2476 (2013)

    Article  Google Scholar 

  24. Choi, H., Jeong, J.: Despeckling images using a preprocessing filter and discrete wavelet transform-based noise reduction techniques. IEEE Sens. J. 18(8), 3131–3139 (2018)

    Article  Google Scholar 

  25. Zhao, L., Bai, H., Liang, J., et al.: Local activity-driven structural-preserving filtering for noise removal and image smoothing. Signal Process. 157, 62–72 (2019)

    Article  Google Scholar 

  26. Singh, P., Jain, L.: Hybridization of adaptive wavelet shrinkage with guided filter for speckle reduction in ultrasound images. Procedia Comput. Sci. 132, 1718–1727 (2018)

    Article  Google Scholar 

  27. Wang, C., Chen, Z., Wang, Y., et al.: Denoising and 3D reconstruction of CT images in extracted tooth via wavelet and bilateral filtering. Int. J. Pattern Recogn. 32(05), 0218–0314 (2018)

    Google Scholar 

  28. Balure, C. S., Kini, M.R., Bhavsar, A.: Single depth image super-resolution via high-frequency subbands enhancement and bilateral filtering. ICIIS 523–528 (2016)

  29. Song, K., Wen, X., Zhao, Y., Dong, Z., Yan, Y.: Noise robust image matching using adjacent evaluation census transform and wavelet edge joint bilateral filter in stereo vision. J. Vis. Commun. Image R 38, 487–503 (2016)

    Article  Google Scholar 

  30. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors 12, 1437–1454 (2012)

    Article  Google Scholar 

  31. Silberman, N., Hoiem, D., Kohli, P., et al.: Indoor segmentation and support inference from RGBD images. Eur. Conf. Comput. Vis. ECCV 7576(1), 746–760 (2012)

    Google Scholar 

  32. Song, S., Lichtenberg, S.P., Xiao, J.A.: RGB-D: A RGB-D scene understanding benchmark suite. Comput. Vis. Pattern Recogn. IEEE, 5, 567–5766 (2015)

  33. Zhang, Q., Shen, X., Xu, L., et al.: Rolling guidance filter. Eur. Conf. Comput. Vis. Springer, ECCV, 8691: 815–830 (2014)

  34. Shen, X., Zhou, C., Xu, L., et al.: Mutual-structure for joint filtering. Int. Conf. Comput. Vis. IEEE, 3406–3414 (2015)

  35. Ramadan, Z.M.: Effect of kernel size on Wiener and Gaussian image filtering. Telkomnika 17(3), 1455–1460 (2019)

    Article  Google Scholar 

  36. Wang, N., Ma, S., Li, J., Zhang, Y., Zhang, L.: Multistage attention network for image inpainting. Pattern Recognit. 106, 107448 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

Dr. Qibing Zhu, Dr. Min Huang and Yue Zhang gratefully acknowledge the financial support from the National Natural Science Foundation of China (Grant no. 61772240, 61775086), the Fundamental Research Funds for the Central Universities (JUSRP51730A).

Author information

Authors and Affiliations

  1. Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), School of Internet of Things, Jiangnan University, 1800 Lihu Avenue, Wuxi, 214122, Jiangsu Province, People’s Republic of China

    Yue Zhang, Zhenfang Liu, Min Huang & Qibing Zhu

  2. Zhejiang Laboratory, Hangzhou, 310000, Zhejiang, China

    Bao Yang

Authors
  1. Yue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenfang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qibing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qibing Zhu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Liu, Z., Huang, M. et al. Multi-resolution depth image restoration. Machine Vision and Applications 32, 65 (2021). https://doi.org/10.1007/s00138-021-01189-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01189-3

Keywords

Search

Navigation