Skip to main content
SpringerLink
Log in

Subdomain Deflation Combined with Local AMG: A Case Study Using AMGCL Library

  • Published:
Lobachevskii Journal of Mathematics Aims and scope Submit manuscript

Abstract

The paper proposes a combination of the subdomain deflation method and local algebraic multigrid as a scalable distributed memory preconditioner that is able to solve large linear systems of equations. The implementation of the algorithm is made available for the community as part of an open source AMGCL library. The solution targets both homogeneous (CPU-only) and heterogeneous (CPU/GPU) systems, employing hybrid MPI/OpenMP approach in the former and a combination of MPI, OpenMP, and CUDA in the latter cases. The use of OpenMP minimizes the number of MPI processes, thus reducing the communication overhead of the deflation method and improving both weak and strong scalability of the preconditioner. The examples of scalar (single degree of freedom per grid node), Poisson-like, systems as well as non-scalar problems, stemming out of the discretization of the Navier-Stokes equations, are considered in order to estimate performance of the implemented algorithm. A comparison with a traditional global AMG preconditioner based on a well-established Trilinos ML package is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://github.com/ddemidov/amgcl.

  2. https://github.com/ddemidov/amgcl_benchmarks.

  3. Barcelona, Spain. https://www.bsc.es/marenostrum/.

  4. Lugano, Switzerland. http://www.cscs.ch/computers/piz_daint/.

  5. http://www.cimne.com/kratos/.

REFERENCES

  1. A. Alexandrescu, Modern \(C++\)Design: Generic Programming and Design Patterns Applied (Addison-Wesley, Reading, MA, 2001).

  2. R. Barrett, M. Berry, T. F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, Ch. Romine, and H. van der Vorst, Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods (SIAM, Philadelphia, 1994).

    Book  Google Scholar 

  3. A. Brandt, S. McCoruick, and J. Huge, ‘‘Algebraic multigrid (amg) for sparse matrix equations,’’ in Sparsity and its Applications, Ed. by D. J. Evans (Cambridge Univ. Press, Cambridge, 1985), p. 257.

    Google Scholar 

  4. O. Bröker and M. J. Grote, ‘‘Sparse approximate inverse smoothers for geometric and algebraic multigrid,’’ Appl. Numer. Math. 41, 61–80 (2002).

    Article  MathSciNet  Google Scholar 

  5. A. J. Cleary, R. D. Falgout, H. Van Emden, J. E. Jones, Th. A. Manteuffel, St. F. McCormick, G. N. Miranda, and J. W. Ruge, ‘‘Robustness and scalability of algebraic multigrid,’’ SIAM J. Sci. Comput. 21, 1886–1908 (2000).

    Article  MathSciNet  Google Scholar 

  6. P. Dadvand, R. Rossi, M. Gil, X. Martorell, J. Cotela, E. Juanpere, S. R. Idelsohn, and E. Oñate, ‘‘Migration of a generic multi-physics framework to HPC environments,’’ Comput. Fluids 80, 301–309 (2013).

    Article  Google Scholar 

  7. P. Dadvand, R. Rossi, and E. Oñate, ‘‘An object-oriented environment for developing finite element codes for multi-disciplinary applications,’’ Arch. Comput. Methods Eng. 17, 253–297 (2010).

    Article  Google Scholar 

  8. D. Demidov, ‘‘AMGCL: a C++ library for solution of large sparse linear systems with algebraic multigrid method’’ (2017).

  9. D. Demidov, ‘‘AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation,’’ Lobachevskii J. Math. 40 (5), 535–546 (2019).

    Article  MathSciNet  Google Scholar 

  10. J. Donéa and A. Huerta, Finite Element Methods for Flow Problems (Wiley, New York, 2003).

    Book  Google Scholar 

  11. H. C. Elman, ‘‘Preconditioning strategies for models of incompressible flow,’’ J. Sci. Comput. 25, 347–366 (2005).

    Article  MathSciNet  Google Scholar 

  12. J. Frank and C. Vuik, ‘‘On the construction of deflation-based preconditioners,’’ SIAM J. Sci. Comput. 23, 442–462 (2001).

    Article  MathSciNet  Google Scholar 

  13. Erich Gamma, Design Patterns: Elements of Reusable Object-Oriented Software (Pearson Education, India, 1995).

  14. M. W. Gee, C. M. Siefert, J. J. Hu, R. S. Tuminaro, and M. G. Sala, ‘‘ML 5.0 smoothed aggregation user’s guide,’’ Tech. Rep. SAND2006-2649 (Sandia Natl. Labor., 2006).

  15. B. Gmeiner, M. Huber, L. John, U. Rüde, and B. Wohlmuth, ‘‘A quantitative performance study for stokes solvers at the extreme scale,’’ J. Comput. Sci.17, 509–521 (2016).

    Article  MathSciNet  Google Scholar 

  16. P. Hénon, P. Ramet, and J. Roman, ‘‘PASTIX: a high-performance parallel direct solver for sparse symmetric positive definite systems,’’ Parallel Comput. 28, 301–321 (2002).

    Article  MathSciNet  Google Scholar 

  17. J. Hogg and J. Scott, ‘‘New parallel sparse direct solvers for multicore architectures,’’ Algorithms 6, 702–725 (2013).

    Article  MathSciNet  Google Scholar 

  18. P. Lin, M. Bettencourt, S. Domino, T. Fisher, M. Hoemmen, J. Hu, E. Phipps, A. Prokopenko, S. Rajamanickam, Ch. Siefert, et al., ‘‘Towards extreme-scale simulations for low mach fluids with second-generation trilinos,’’ Parallel Proces. Lett. 24, 1442005 (2014).

    Article  MathSciNet  Google Scholar 

  19. J. Mathews and R. L. Walker, Mathematical Methods of Physics (W.A. Benjamin, New York, 1970).

    MATH  Google Scholar 

  20. R. Nabben and C. Vuik, ‘‘A comparison of deflation and the balancing preconditioner,’’ SIAM J. Sci. Comput. 27, 1742–1759 (2006).

    Article  MathSciNet  Google Scholar 

  21. J. W. Ruge and K. Stüben, ‘‘Algebraic multigrid,’’ in Multigrid Methods (SIAM, Philadelphia, 1987), pp. 73–130.

    Google Scholar 

  22. Yousef Saad, Iterative Methods for Sparse Linear Systems (SIAM, Philadelphia, 2003).

  23. B. Schäling, The Boost \(C++\)Libraries (XML Press, Aufl. Laguna Hills, CA, 2014).

  24. G. L. G. Sleijpen and D. R. Fokkema, ‘‘BiCGstab (l) for linear equations involving unsymmetric matrices with complex spectrum,’’ Electron. Trans. Numer. Anal. 1, 11–32 (1993).

    MathSciNet  MATH  Google Scholar 

  25. B. Smith, P. Bjorstad, and W. Gropp, Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations (Cambridge Univ. Press, Cambridge, 2004).

    MATH  Google Scholar 

  26. K. Stuben, ‘‘Algebraic multigrid (AMG): an introduction with applications,’’ GMD Report №70 (GMD, St. Augustin, Germany, 1999).

    Google Scholar 

  27. J. M. Tang, R. Nabben, C. Vuik, and Y. A. Erlangga, ‘‘Comparison of two-level preconditioners derived from deflation, domain decomposition and multigrid methods,’’ SIAM J. Sci. Comput. 39, 340–370 (2009).

    Article  MathSciNet  Google Scholar 

  28. J. M. Tang and C. Vuik, New Variants of Deflation Techniques for Bubbly Flow Problems (Delft Univ. Technol., Delft, 2006).

  29. U. Trottenberg, C. Oosterlee, and A. Schüller, Multigrid (Academic, London, 2001).

    MATH  Google Scholar 

  30. Stefan Turek, Efficient Solvers for Incompressible Flow Problems: An Algorithmic and Computational Approach (Springer, Berlin, Heidelberg, 1999).

  31. C. Vuik, A. Segal, and J. A. Meijerink, ‘‘An efficient preconditioned CG method for the solution of a class of layered problems with extreme contrasts in the coefficients,’’ J. Comput. Phys. 152, 385–403 (1999).

    Article  Google Scholar 

  32. U. Meier Yang et al., ‘‘BoomerAMG: a parallel algebraic multigrid solver and preconditioner,’’ Appl. Numer. Math. 41, 155–177 (2002).

    Article  MathSciNet  Google Scholar 

Download references

Funding

Contribution of Dr. Demidov was funded by the state assignment to the Joint Supercomputer Center of the Russian Academy of Sciences for Scientific Research and Russian Foundation for Basic Research, grant no. 18-07-00964.

Dr. Rossi acknowledges the financial support to CIMNE via the CERCA Programme/Generalitat de Catalunya and the support of the ExaQUte FetHPC, project GA 800898.

The authors thankfully acknowledge the support of the PRACE program (project 2010PA4058), in providing access to the MareNostrum 4 and PizDaint clusters. Without such resources the testing would not have been possible. The help of Prof. Labarta of the POP Center of Excellence in improving the NUMA scalability of the solver is also gratefully acknowledged.

Author information

Authors and Affiliations

  1. Kazan Branch of Joint Supercomputer Center, Russian Academy of Sciences, 420111, Kazan, Russia

    D. Demidov

  2. Department of Civil and Environmental Engineering, Universitat Politècnica de Catalunya (UPC), 08034, Barcelona, Spain

    R. Rossi

  3. CIMNE—Centre Internacional de Mètodes Numèrics en Enginyeria, CIMNE, 08034, Barcelona, Spain

    R. Rossi

Authors
  1. D. Demidov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to D. Demidov or R. Rossi.

Additional information

(Submitted by A. V. Lapin)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Demidov, D., Rossi, R. Subdomain Deflation Combined with Local AMG: A Case Study Using AMGCL Library. Lobachevskii J Math 41, 491–511 (2020). https://doi.org/10.1134/S1995080220040071

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1995080220040071

Keywords and phrases:

Search

Navigation