Skip to main content
SpringerLink
Log in

Image Reconstruction by Minimizing Curvatures on Image Surface

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

The curvature regularities are well-known for providing strong priors in the continuity of edges, which have been applied to a wide range of applications in image processing and computer vision. However, these models are usually non-convex, non-smooth, and highly nonlinear, the first-order optimality condition of which are high-order partial differential equations. Thus, numerical computation is extremely challenging. In this paper, we estimate the discrete mean curvature and Gaussian curvature on the local \(3\times 3\) stencil, based on the fundamental forms in differential geometry. By minimizing certain functions of curvatures over the image surface, it yields a kind of weighted image surface minimization problem, which can be efficiently solved by the alternating direction method of multipliers. Numerical experiments on image restoration and inpainting are implemented to demonstrate the effectiveness and superiority of the proposed curvature-based model compared to state-of-the-art variational approches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  1. Ambrosio, L., Masnou, S.: A direct variational approach to a problem arising in image reconstruction. Interfaces Free Boundaries 5(1), 63–81 (2003)

    MathSciNet  Google Scholar 

  2. Bae, E., Shi, J., Tai, X.C.: Graph cuts for curvature based image denoising. IEEE Trans. Image Process. 20(5), 1199–1210 (2011)

    MathSciNet  Google Scholar 

  3. Bae, E., Tai, X.C., Zhu, W.: Augmented Lagrangian method for an Euler’s elastica based segmentation model that promotes convex contours. Inverse Probl. Imaging 11(1), 1–23 (2017)

    MathSciNet  Google Scholar 

  4. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)

    MathSciNet  Google Scholar 

  5. Blomgren, P., Chan, T.: Color TV: total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304–309 (1998)

    Google Scholar 

  6. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2013)

    MathSciNet  Google Scholar 

  7. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)

    MathSciNet  Google Scholar 

  8. Bredies, K., Pock, T., Wirth, B.: A convex, lower semicontinuous approximation of Euler’s elastica energy. SIAM J. Math. Anal. 47(1), 566–613 (2015)

    MathSciNet  Google Scholar 

  9. Brito-Loeza, C., Chen, K., Uc-Cetina, V.: Image denoising using the Gaussian curvature of the image surface. Numer. Methods Partial Differ. Equ. 32(3), 1066–1089 (2016)

    MathSciNet  Google Scholar 

  10. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    MathSciNet  Google Scholar 

  11. Chambolle, A., Pock, T.: Total roto-translational variation. Numerische Mathematik 142(3), 611–666 (2019)

    MathSciNet  Google Scholar 

  12. Chan, T.F., Jianhong, S.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic. Society for Industrial and Applied Mathematics, Philadelphia (2005)

    Google Scholar 

  13. Ciomaga, A., Monasse, P., Morel, J.M.: The image curvature microscope: accurate curvature computation at subpixel resolution. Image Process. Line 7, 197–217 (2017)

    MathSciNet  Google Scholar 

  14. Cui, Y., Li, X., Sun, D., Toh, K.C.: On the convergence properties of a majorized alternating direction method of multipliers for linearly constrained convex optimization problems with coupled objective functions. J. Optim. Theory Appl. 169(3), 1013–1041 (2016)

    MathSciNet  Google Scholar 

  15. Deng, L.J., Glowinski, R., Tai, X.C.: A new operator splitting method for the Euler elastica model for image smoothing. SIAM J. Imaging Sci. 12(2), 1190–1230 (2019)

    MathSciNet  Google Scholar 

  16. Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2015)

    MathSciNet  Google Scholar 

  17. Dong, Y., Hintermüller, M., Neri, M.: An efficient primal-dual method for \({L}_{1}\) TV image restoration. SIAM J. Imaging Sci. 2(4), 1168–1189 (2009)

    MathSciNet  Google Scholar 

  18. Duan, Y., Huang, W., Zhou, J., Chang, H., Zeng, T.: A two-stage image segmentation method using Euler’s elastica regularized Mumford-Shah model. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 118–123 (2014)

  19. Duan, Y., Wang, Y., Hahn, J.: A fast augmented Lagrangian method for Euler’s elastica models. Numer. Math. Theory Methods Appl. 6(1), 47–71 (2013)

    MathSciNet  Google Scholar 

  20. El-Fallah, A., Ford, G.: Mean curvature evolution and surface area scaling in image filtering. IEEE Trans. Image Process. 6(5), 750–753 (1997)

    Google Scholar 

  21. El-Zehiry, N.Y., Grady, L.: Fast global optimization of curvature. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3257–3264 (2010)

  22. Esedoglu, S., March, R.: Segmentation with depth but without detecting junctions. J. Math. Imaging Vis. 18(1), 7–15 (2003)

    MathSciNet  Google Scholar 

  23. Goldluecke, B., Cremers, D.: Introducing total curvature for image processing. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1267–1274 (2011)

  24. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    MathSciNet  Google Scholar 

  25. Gong, Y., Sbalzarini, I.F.: Local weighted Gaussian curvature for image processing. In: Proceedings of the IEEE International Conference on Image Processing, pp. 534–538 (2013)

  26. Gong, Y., Sbalzarini, I.F.: Curvature filters efficiently reduce certain variational energies. IEEE Trans. Image Process. 26(4), 1786–1798 (2017)

    MathSciNet  Google Scholar 

  27. Huang, Y., Ng, M.K., Wen, Y.: A new total variation method for multiplicative noise removal. SIAM J. Imaging Sci. 2(1), 20–40 (2009)

    MathSciNet  Google Scholar 

  28. Kang, S.H., Zhu, W., Jianhong, J.: Illusory shapes via corner fusion. SIAM J. Imaging Sci. 7(4), 1907–1936 (2014)

    MathSciNet  Google Scholar 

  29. Le, T., Chartrand, R., Asaki, T.J.: A variational approach to reconstructing images corrupted by Poisson noise. J. Math. Imaging Vis. 27(3), 257–263 (2007)

    MathSciNet  Google Scholar 

  30. Lee, S.H., Seo, J.K.: Noise removal with Gauss curvature-driven diffusion. IEEE Trans. Image Process. 14(7), 904–909 (2005)

    MathSciNet  Google Scholar 

  31. Liu, J., Yan, M., Zeng, T.: Surface-aware blind image deblurring. IEEE Trans. Pattern Anal. Mach. Intell. 99, 1–1 (2019)

    Google Scholar 

  32. Lu, B., Wang, H., Lin, Z.: High order Gaussian curvature flow for image smoothing. In: Proceedings of the IEEE International Conference on Multimedia Technology, pp. 5888–5891 (2011)

  33. Ma, Q., Peng, J., Kong, D.: Image segmentation via mean curvature regularized Mumford–Shah model and thresholding. Neural Process. Lett. 48(2), 1227–1241 (2017)

    Google Scholar 

  34. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2007)

    MathSciNet  Google Scholar 

  35. Masnou, S., Morel, J.M.: Level lines based disocclusion. In: Proceedings of the IEEE International Conference on Image Processing, pp. 259–263 (1998)

  36. Mumford, D.: Elastica and Computer Vision, pp. 491–506. Springer, Berlin (1994)

    Google Scholar 

  37. Myllykoski, M., Glowinski, R., Karkkainen, T., Rossi, T.: A new augmented Lagrangian approach for \(l_{1}\)-mean curvature image denoising. SIAM J. Imaging Sci. 8(1), 95–125 (2015)

    MathSciNet  Google Scholar 

  38. Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004)

    MathSciNet  Google Scholar 

  39. Nitzberg, M., Mumford, D., Shiota, T.: Filtering, Segmentation and Depth. Springer, Berlin (1993)

    Google Scholar 

  40. Rosman, G., Dascal, L., Tai, X.C., Kimmel, R.: On semi-implicit splitting schemes for the Beltrami color image filtering. J. Math. Imaging Vis. 40(2), 199–213 (2011)

    MathSciNet  Google Scholar 

  41. Rosman, G., Tai, X.C., Dascal, L., Kimmel, R.: Polyakov action minimization for efficient color image processing. In: Proceedings of the Springer European Conference on Computer Vision, pp. 50–61 (2010)

  42. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)

    MathSciNet  Google Scholar 

  43. Schoenemann, T., Kahl, F., Cremers, D.: Curvature regularity for region-based image segmentation and inpainting: a linear programming relaxation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 17–23 (2009)

  44. Shefi, R., Teboulle, M.: Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    MathSciNet  Google Scholar 

  45. Shen, J., Kang, S.H., Chan, T.F.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2003)

    MathSciNet  Google Scholar 

  46. Sochen, N., Kimmel, R., Malladi, R.: A general framework for low level vision. IEEE Trans. Image Process. 7(3), 310–318 (1998)

    MathSciNet  Google Scholar 

  47. Tai, X.C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)

    MathSciNet  Google Scholar 

  48. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Google Scholar 

  49. Wen, Y.W., Ng, M.K., Huang, Y.M.: Efficient total variation minimization methods for color image restoration. IEEE Trans. Image Process. 17(11), 2081–2088 (2008)

    MathSciNet  Google Scholar 

  50. Wu, C., Tai, X.C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    MathSciNet  Google Scholar 

  51. Wu, C., Zhang, J., Tai, X.C.: Augmented Lagrangian method for total variation restoration with non-quadratic fidelity. Inverse Probl. Imaging 5(1), 237–261 (2011)

    MathSciNet  Google Scholar 

  52. Yan, M., Duan, Y.: Nonlocal elastica model for sparse reconstruction. J. Math. Imaging Vis. 62(4), 532–548 (2020)

    MathSciNet  Google Scholar 

  53. Yang, J., Zhang, Y., Yin, W.: An efficient TVL1 algorithm for deblurring multichannel images corrupted by impulsive noise. SIAM J. Sci. Comput. 31(4), 2842–2865 (2009)

    MathSciNet  Google Scholar 

  54. Yashtini, M., Kang, S.H.: A fast relaxed normal two split method and an effective weighted TV approach for Euler’s elastica image inpainting. SIAM J. Imaging Sci. 9(4), 1552–1581 (2016)

    MathSciNet  Google Scholar 

  55. Yezzi, A.: Modified curvature motion for image smoothing and enhancement. IEEE Trans. Image Process. 7(3), 345–352 (1998)

    MathSciNet  Google Scholar 

  56. Zhang, H., Wang, L., Duan, Y., Li, L., Hu, G., Yan, B.: Euler’s elastica strategy for limited-angle computed tomography image reconstruction. IEEE Trans. Nuclear Sci. 64(8), 2395–2405 (2017)

    Google Scholar 

  57. Zhong, Q., Li, Y., Yang, Y., Duan, Y.: Minimizing discrete total curvature for image processing. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9474–9482 (2020)

  58. Zhu, W., Chan, T.: Image denoising using mean curvature of image surface. SIAM J. Imaging Sci. 5(1), 1–32 (2012)

    MathSciNet  Google Scholar 

  59. Zhu, W., Chan, T., Esedolu, S.: Segmentation with depth: a level set approach. SIAM J. Sci. Comput. 28(5), 1957–1973 (2006)

    MathSciNet  Google Scholar 

  60. Zhu, W., Tai, X.C., Chan, T.: Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Probl. Imaging 7(4), 1409–1432 (2013)

    MathSciNet  Google Scholar 

  61. Zhu, W., Tai, X.C., Chan, T.: Image segmentation using Euler’s elastica as the regularization. J. Sci. Comput. 57(2), 414–438 (2013)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Gong and Prof. Sbalzarini for sharing the MATLAB code of curvature filter and the referees for providing us numerous valuable suggestions to revise the paper. The work was partially supported by National Natural Science Foundation of China (NSFC 12071345, 11701418), Major Science and Technology Project of Tianjin 18ZXRHSY00160 and Recruitment Program of Global Young Expert. The second author was supported by NSFC 11801200 and a startup Grant from HUST.

Author information

Authors and Affiliations

  1. Center for Applied Mathematics, Tianjin University, Tianjin, 300072, China

    Qiuxiang Zhong & Yuping Duan

  2. Center for Mathematical Sciences, Huazhong Univeristy of Science and Technology, Wuhan, Hubei, 430070, China

    Ke Yin

Authors
  1. Qiuxiang Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ke Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuping Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuping Duan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, Q., Yin, K. & Duan, Y. Image Reconstruction by Minimizing Curvatures on Image Surface. J Math Imaging Vis 63, 30–55 (2021). https://doi.org/10.1007/s10851-020-00992-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-020-00992-3

Keywords

Search

Navigation