Skip to main content
SpringerLink
Log in

An Inertial Algorithm for DC Programming

  • Published:
Set-Valued and Variational Analysis Aims and scope Submit manuscript

Abstract

We consider nonsmooth optimization problems whose objective function is defined by the Difference of Convex (DC) functions. With the aim of computing critical points that are also d(irectional)-stationary for such a class of nonconvex programs we propose an algorithmic scheme equipped with an inertial-force procedure. In contrast to the classical DC algorithm of P. D. Tao and L. T. H. An, the proposed inertial DC algorithm defines trial points whose sequence of functional values is not necessary monotonically decreasing, a property that proves useful to prevent the algorithm from converging to a critical point that is not d-stationary. Moreover, our method can handle inexactness in the solution of convex subproblems yielding trial points. This is another property of practical interest that substantially reduces the computational burden to compute d-stationary/critical points of DC programs. Convergence analysis of the proposed algorithm yields global convergence to critical points, and convergence rate is established for the considered class of problems. Numerical experiments on large-scale (nonconvex and nonsmooth) image denoising models show that the proposed algorithm outperforms the classic one in this particular application, specifically in the case of piecewise constant images with neat edges such as QR codes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19 (9), 2345–2356 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  2. Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in hilbert space. SIAM J. Optim. 14 (3), 773–782 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  3. Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Val. Anal. 9(1), 3–11 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  4. An, L.T.H., Tao, P.D.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1), 23–46 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bagirov, A.M., Yearwood, J.: A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170(2), 578–596 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18 (11), 2419–2434 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bello Cruz, J.Y.: On proximal subgradient splitting method for minimizing the sum of two nonsmooth convex functions. Set-Val. Var. Anal. 25(2), 245–263 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  8. Caselles, V., Chambolle, A., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theoretical Foundations and Numerical Methods for Sparse Recovery, De Gruyter, Radon Series. Comp. Appl. Math. 9, 263–340 (2010)

    MATH  Google Scholar 

  9. Condat, L.: A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  10. Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  11. de Oliveira, W.: Proximal Bundle Methods for Nonsmooth DC Programming. Technical report. Available at www.oliveira.mat.br (2017)

  12. Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  13. Gaudioso, M., Giallombardo, G., Miglionico, G.: Minimizing piecewise-concave functions over polyhedra. Math. Oper. Res. 43(2), 580–597 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Glob. Optim. 71, 37–55 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall, Inc., Upper Saddle River (2006)

  16. Gruzdeva, T., Strekalovsky, A.: A D.C. Programming Approach to Fractional Problems, pp 331–337. Springer International Publishing, Cham (2017)

    MATH  Google Scholar 

  17. Hiriart-Urruty, J.: Optimisation et analyse convexe. Presses Universitaires de France, France (1998)

    MATH  Google Scholar 

  18. Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I, 2nd edn. No. 305 in Grundlehren der mathematischen Wissenschaften. Springer, Berlin (1996)

    Google Scholar 

  19. Hiriart-Urruty, J.B.: Generalized Differentiability/Duality and Optimization for Problems Dealing with Differences of Convex Functions, pp 37–70. Springer, Berlin (1985)

    MATH  Google Scholar 

  20. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305-306 in Grund. der math. Wiss. Springer, Berlin (1993). (two volumes)

    Book  Google Scholar 

  21. Holmberg, K., Tuy, H.: A production-transportation problem with stochastic demand and concave production costs. Math. Program. 85(1), 157–179 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  22. Izmailov, A.F., Solodov, M.V.: Newton-type Methods for Optimization and Variational Problems, 1st edn. Springer Series in Operations Research and Financial Engineering Springer International Publishing (2014)

  23. Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth dc optimization utilizing nonconvex cutting planes. J. Glob. Optim. 68(3), 501–535 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  24. Khalaf, W., Astorino, A., d’Alessandro, P., Gaudioso, M.: A DC optimization-based clustering technique for edge detection. Optim. Lett. 11(3), 627–640 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization with parameter selection. J. Math. Imaging Vis. 56(2), 195–220 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Le Thi, H.A., Pham Dinh, T.: Dc programming in communication systems: challenging problems and methods. Vietnam J. Comput. Sci. 1(1), 15–28 (2014)

    Article  Google Scholar 

  27. Le Thi, H.A., Pham Dinh, T., Ngai, H.V.: Exact penalty and error bounds in dc programming. J. Glob. Optim. 52(3), 509–535 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  28. w. Lu, C.: Image restoration and decomposition using nonconvex non-smooth regularisation and negative hilbert-sobolev norm. IET Image Process. 6(6), 706–716 (2012)

    Article  MathSciNet  Google Scholar 

  29. Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155(2), 447–454 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  30. Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  32. Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  33. Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)

    Article  Google Scholar 

  34. Rockafellar, R.: Convex Analysis, 1st edn. Princeton University Press, Princeton (1970)

  35. Souza, J.C.O., Oliveira, P.R., Soubeyran, A.: Global convergence of a proximal linearized algorithm for difference of convex functions. Optim. Lett. 10(7), 1529–1539 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  36. Strekalovskiy, A.: An exact penalty method for d.c. optimization. AIP Conf. Proc. 1776(1), 060,003 (2016)

    Article  Google Scholar 

  37. Strekalovsky, A., Minarchenko, I.: On Local Search in d.c. Optimization. In: 2017 Constructive Nonsmooth Analysis and Related Topics (Dedicated to the Memory of V.F. Demyanov) (CNSA), pp. 1–4 (2017)

  38. Strekalovsky, A.S.: On local search in d.c. optimization problems. Appl. Math. Comput. 255(1), 73–83 (2015)

    MathSciNet  MATH  Google Scholar 

  39. Tao, P.D., An, L.T.H.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnamica 22(1), 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  40. Tao, P.D., Souad, E.B.: Algorithms for solving a class of nonconvex optimization problems. methods of subgradients. North-Holland Math. Stud. 129, 249–271 (1986). Fermat Days 85: Mathematics for Optimization

    Article  MathSciNet  MATH  Google Scholar 

  41. Tuy, H.: Convex Analysis and Global Optimization, 2nd edn. Springer Optimization and Its Applications. Springer (2016)

  42. van Ackooij, W., de Oliveira, W.: DC Programming Techniques with Inexact Subproblems’ Solution for General DC Programs. Technical Report, CMA. Available at www.oliveira.mat.br (2017)

  43. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  44. Zavriev, S.K., Kostyuk, F.V.: Heavy-ball method in nonconvex optimization problems. Comput. Math. Model. 4(4), 336–341 (1993)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors are grateful to the Reviewers and the Associate Editor for their remarks and constructive suggestions that considerably improved the original version of this article.

Author information

Authors and Affiliations

  1. MINES ParisTech, PSL – Research University, CMA – Centre de Mathématiques Appliquées, Sophia Antipolis, France

    Welington de Oliveira

  2. Rio de Janeiro State University (UERJ), Rio de Janeiro, Brazil

    Michel P. Tcheou

Authors
  1. Welington de Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michel P. Tcheou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Welington de Oliveira.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Oliveira, W., Tcheou, M.P. An Inertial Algorithm for DC Programming. Set-Valued Var. Anal 27, 895–919 (2019). https://doi.org/10.1007/s11228-018-0497-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11228-018-0497-0

Keywords

Mathematics Subject Classification (2010)

Search

Navigation