We aim to factorize a completely positive matrix by using an optimization approach which consists in the minimization of a nonconvex smooth function over a convex and compact set. To solve this problem we propose a projected gradient algorithm with parameters that take into account the effects of relaxation and inertia. Both projection and gradient steps are simple in the sense that they have explicit

Space‐time methods are able to solve time‐dependent problems faster by exploiting the full power of high‐performance computer. In this article, a multilevel space‐time multiplicative Schwarz method is presented for solving parabolic equations in parallel on both spatial and temporal directions. In the implementation, the proposed Schwarz method is treated as preconditioner for GMRES, that is, a coupled

Tensor completion is a challenging problem with various applications, especially in recovering incomplete visual data. Considering a color image or gray video as a three‐dimensional tensor, many related models based on the low‐rank prior of the tensor have been proposed. However, the low‐rank prior may not be enough to recover the original tensor from the observed incomplete tensor. In this paper,

Overlapping block smoothers efficiently damp the error contributions from highly oscillatory components within multigrid methods for the Stokes equations but they are computationally expensive. This paper is concentrated on the development and analysis of new block smoothers for the Stokes equations that are discretized on staggered grids. These smoothers are nonoverlapping and therefore desirable

We discuss the use of a matrix‐oriented approach for numerically solving the dense matrix equation AX + XAT + M1XN1 + … + MℓXNℓ = F, with ℓ ≥ 1, and Mi, Ni, i = 1, … , ℓ of low rank. The approach relies on the Sherman–Morrison–Woodbury formula formally defined in the vectorized form of the problem, but applied in the matrix setting. This allows one to solve medium size dense problems with computational

Local Fourier analysis (LFA) serves a prominent role in the prediction of the convergence factor of multigrid methods for discretizations of PDEs. However, few discussions about the implementation of LFA for complicated discretizations of PDEs exist, such as higher order finite elements, as staggered meshes could lead to complex LFA representations of the grid‐transfer operator. In this work, we prove

Efficient parallel‐in‐time methods for hyperbolic partial differential equation problems remain scarce. Here we investigate an approach based on circulant preconditioned generalised minimal residual (GMRES) for the monolithic block Toeplitz equations which arise from constant time‐step discretizations. We present theoretical results which guarantee convergence in a number of iterations independent

We consider the numerical solution of large scale singular (continuous‐time) Lyapunov equations of the form AX + XA⊤ + BB⊤ = 0, where A is semistable, that is, its spectrum is contained in the left half plane, with the exception of a few semisimple eigenvalues at zero. We also consider the case of a few semisimple eigenvalues on the imaginary axis. We assume that we know these few eigenvalues (zero

In this paper, we consider the nonnegative tensor least squares problem, which arises in the color image restoration. Based on the BB stepsize technique, we design a nonmonotonic descent stepsize and then derive a new gradient projection algorithm to solve this problem. The convergence analysis of the new gradient projected algorithm is given. Some numerical examples show that the new method is feasible

This article shows that the bidiagonal decomposition of many important matrices of q‐integers can be constructed to high relative accuracy (HRA). This fact can be used to compute with HRA the eigenvalues, singular values, and inverses of these matrices. These results can be applied to collocation matrices of q‐Laguerre polynomials, q‐Pascal matrices, and matrices formed by q‐Stirling numbers. Numerical

This special issue “Multigrid Methods” in the Journal of Numerical Linear Algebra with Applications contains nine papers submitted by participants from the 19th Copper Mountain Conference on Multigrid Methods, held in the Colorado Rocky Mountains March 24–28, 2019. The papers cover a range of topics on Multigrid, including algorithm development for a variety of applications, convergence and complexity

Numerical methods for elliptic partial differential equations usually lead to systems of linear equations with sparse, symmetric, and positive definite matrices. In many methods, these matrices can be obtained as sums of local symmetric positive semidefinite matrices. In this article, we use this assumption and introduce a method that provides guaranteed lower and upper bounds to all individual eigenvalues

An algorithm for two‐variable rational interpolation is developed. The algorithm is suitable for interpolation cases where neither the number of interpolation points nor the final degrees of the rational interpolant are known a priori. Instead, a maximum degree for the interpolant's numerator and denominator is assumed. By testing the condition number of the interpolation system's matrix at each step

Problems arising in Earth's mantle convection involve finding the solution to Stokes systems with large viscosity contrasts. These systems contain localized features which, even with adaptive mesh refinement, result in linear systems that can be on the order of 109 or more unknowns. One common approach for preconditioning to the velocity block of these systems is to apply an Algebraic Multigrid (AMG)

The reduction of a large‐scale symmetric linear discrete ill‐posed problem with multiple right‐hand sides to a smaller problem with a symmetric block tridiagonal matrix can easily be carried out by the application of a small number of steps of the symmetric block Lanczos method. We show that the subdiagonal blocks of the reduced problem converge to zero fairly rapidly with increasing block number.

The solution of large sparse linear systems is required in many scientific fields such as computational fluid dynamics, computational electromagnetism, computational finance, etc. The computation of the solution of these systems is performed with preconditioned iterative methods, which rely on effective preconditioning schemes. A new class of approximate inverses is proposed, namely incomplete inverse

We propose an inexact coordinate descent method to solve a consistent equation of the type Ax = b where A is a real symmetric and positive semidefinite matrix. The admissible range of inexactitude allows to reduce every multiplication present in the computation to a scaling by a signed power of 2 for the sake of hardware simplification. Although the resulting descent is nonideal, the numerical experiments

Fixed‐point iteration is a popular method for the multilinear PageRank problem. In [B. Meini, F. Poloni, Perron‐based algorithms for the multilinear PageRank, Numer. Linear Algebra Appl., 25: e2177, 2018], a theorem was proved to determine the behavior of the fixed‐point iteration as the parameter α ∈ ( 1 2 , 1 ) . In this work, we point out that this theorem is incomplete, and revisit the convergence

This paper presents the two‐parameter scaling memoryless Broyden–Fletcher–Goldfarb–Shanno (BFGS) method for solving a system of monotone nonlinear equations. The optimal values of the scaling parameters are obtained by minimizing the measure function involving all the eigenvalues of the memoryless BFGS matrix. The optimal values can be used in the analysis of the quasi‐Newton method for ill‐conditioned

It is known that the solutions to space‐fractional diffusion equations exhibit singularities near the boundary. Therefore, numerical methods discretized on the composite mesh, in which the mesh size is refined near the boundary, provide more precise approximations to the solutions. However, the coefficient matrices of the corresponding linear systems usually lose the diagonal dominance and are ill‐conditioned

This paper is concerned with improving the empirical convergence speed of block‐coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in‐between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block‐coordinate algorithms for NTF

Solving linear systems is often the computational bottleneck in real‐life problems. Iterative solvers are the only option due to the complexity of direct algorithms or because the system matrix is not explicitly known. Here, we develop a two‐level preconditioner for regularized least squares linear systems involving a feature or data matrix. Variants of this linear system may appear in machine learning

In this paper, we present explicit expressions for the mixed and componentwise condition numbers of the truncated total least squares (TTLS) solution of A x ≈ b under the genericity condition, where A is a m × n real data matrix and b is a real m‐vector. Moreover, we reveal that normwise, componentwise, and mixed condition numbers for the TTLS problem can recover the previous corresponding counterparts

The growing discrepancy between CPU computing power and memory bandwidth drives more and more numerical algorithms into a bandwidth‐bound regime. One example is the overlapping Schwarz smoother, a highly effective building block for iterative multigrid solution of elliptic equations with higher order finite elements. Two options of reducing the required memory bandwidth are sparsity exploiting storage

In [Hayami K, Sugihara M. Numer Linear Algebra Appl. 2011; 18:449–469], the authors analyzed the convergence behavior of the generalized minimal residual (GMRES) method for the least squares problem min x ∈ R n ‖ b − A x ‖ 2 2 , where A ∈ Rn × n may be singular and b ∈ R n , by decomposing the algorithm into the range ℛ ( A ) and its orthogonal complement ℛ ( A ) ⊥ components. However, we found that

Parallel‐in‐time methods, such as multigrid reduction‐in‐time (MGRIT) and Parareal, provide an attractive option for increasing concurrency when simulating time‐dependent partial differential equations (PDEs) in modern high‐performance computing environments. While these techniques have been very successful for parabolic equations, it has often been observed that their performance suffers dramatically

Based on the spectral divide‐and‐conquer algorithm by Nakatsukasa and Higham [SIAM J. Sci. Comput., 35(3):A1325–A1349, 2013], we propose a new algorithm for computing all the eigenvalues and eigenvectors of a symmetric banded matrix with small bandwidth, with the eigenvectors given implicitly as a product of orthonormal matrices stored in the so‐called hierarchically off‐diagonal low‐rank (HODLR) format

The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large‐scale problems is challenging. Motivated by applications in hyper‐differential sensitivity analysis (HDSA), we propose new randomized algorithms for computing the GSVD which use randomized subspace iteration and weighted QR factorization.

The l parameterized generalized eigenvalue problems for the nonsquare matrix pencils, proposed by Chu and Golub [SIAM J. Matrix Anal. Appl., 28(2006), pp. 770‐787], can be formulated as an optimization problem on a corresponding complex product Stiefel manifold. Some early proposed algorithms are based on the first‐order information of the objective function, and fast convergence could not be expected

The radial point interpolation meshfree discretization is a very efficient numerical framework for the analysis of piezoelectricity, in which the fundamental electrostatic equations governing piezoelectric media are solved without mesh generation. Due to the mechanical‐electrical coupling property and the piezoelectric constant, the discrete linear system is sparse, of generalized saddle point form

Discrete ill‐posed inverse problems arise in many areas of science and engineering. Their solutions are very sensitive to perturbations in the data. Regularization methods aim at reducing this sensitivity. This article considers an iterative regularization method, based on iterated Tikhonov regularization, that was proposed in M. Donatelli and M. Hanke, Fast nonstationary preconditioned iterative methods

In [1], the following error was published on page 1. The affiliation of the second author, Pei Yuan, read: 2Institute of Computing Technology, Beijing, 100190, China. The affiliation of the second author, Pei Yuan, should read: 2Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China. 3University of Chinese Academy of Sciences, Beijing, 100049, China.

A new method is proposed to compute the eigenpairs of a parameter‐dependent nonlinear eigenvalue problem. We first analyze the properties on the analytic perturbation of the invariant pair of a nonlinear eigenvalue problem and provide a method to compute the first‐order and high‐order derivatives of the invariant pair. On these grounds, the subspaces independent of the parameter and containing the

In this article, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal‐dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill‐conditioned linear

Image denoising is a typical inverse problem and is hard to be solved. Fortunately, a powerful two‐phase method for restoring images corrupted by high‐level impulse noise has been proposed. The key point of the method is the computational efficiency of the second phase which requires the minimization of a smooth objective function defined on the terms of edge‐preserving potential function. In this

In this article, we present a reliable hybrid algorithm for solving convex quadratic minimization problems. At the kth iteration, two points are computed: first, an auxiliary point x ˙ k is generated by performing a gradient step using an optimal steplength, and second, the next iterate xk + 1 is obtained by means of weighted sum of x ˙ k with the penultimate iterate xk − 1. The coefficient of the

Scalable approaches for uncertainty quantification are necessary for characterizing prediction confidence in large‐scale subsurface flow simulations with uncertain permeability. To this end we explore a multilevel Monte Carlo approach for estimating posterior moments of a particular quantity of interest, where we employ an element‐agglomerated algebraic multigrid (AMG) technique to generate the hierarchy

In the past decades, multigrid methods for linear systems having multilevel Toeplitz coefficient matrices with scalar entries have been widely studied. On the other hand, only few papers have investigated the case of block entries, where the entries are small generic matrices of fixed size instead of scalars. In that case the efforts of the researchers have been mainly devoted to specific applications

Time–space fractional Fokker–Planck equations (TSFFPEs) are a class of very useful models for describing some practical phenomena in statistical physics. In the present article, we focus on the fast computation for semilinear two‐dimensional TSFFPEs. A stable implicit difference scheme with second‐order accuracy in time and space is derived. In order to accelerate the convergence rate of the scheme

The row iterative method is popular in solving the large‐scale ill‐posed problems due to its simplicity and efficiency. In this work we consider the randomized row iterative (RRI) method to tackle this issue. First, we present the semiconvergence analysis of RRI method for the overdetermined and inconsistent system, and derive upper bounds for the noise error propagation in the iteration vectors. To

We propose to reduce the (spectral) condition number of a given linear system by adding a suitable diagonal matrix to the system matrix, in particular by shifting its spectrum. Iterative procedures are then adopted to recover the solution of the original system. The case of real symmetric positive definite matrices is considered in particular, and several numerical examples are given. This approach

In this article, we consider the stochastic inverse singular value problem (ISVP) of constructing a stochastic matrix from the prescribed realizable singular values. We propose a Riemannian inexact Newton‐CG method with various choices of forcing terms for solving the stochastic ISVP. We show the proposed method converges linearly or superlinearly for different forcing terms under some assumptions

We investigate the method of conjugate gradients, exploiting inaccurate matrix‐vector products, for the solution of convex quadratic optimization problems. Theoretical performance bounds are derived, and the necessary quantities occurring in the theoretical bounds estimated, leading to a practical algorithm. Numerical experiments suggest that this approach has significant potential, including in the

High‐dimensionality reduction techniques are very important tools in machine learning and data mining. The method of generalized low rank approximations of matrices (GLRAM) is a popular technique for dimensionality reduction and image compression. However, it suffers from heavily computational overhead in practice, especially for data with high dimension. In order to reduce the cost of this algorithm

Block Krylov subspace methods (KSMs) comprise building blocks in many state‐of‐the‐art solvers for large‐scale matrix equations as they arise, for example, from the discretization of partial differential equations. While extended and rational block Krylov subspace methods provide a major reduction in iteration counts over polynomial block KSMs, they also require reliable solvers for the coefficient

Following the Perron theorem, the spectral radius of a primitive matrix is a simple eigenvalue. It is shown that for a primitive matrix A, there is a positive rank one matrix X such that B = A ∘ X, where ∘ denotes the Hadamard product of matrices, and such that the row (column) sums of matrix B are the same and equal to the Perron root. An iterative algorithm is presented to obtain matrix B without

During the past few decades, uncertainty quantification (UQ) techniques have been developed and applied to many applications. The majority of these techniques have been applied directly to specifically defined problems, that is, problems described by a mathematical operator and a specific source term, both which may be endowed with uncertainty. In this article, we take an alternative approach: applying

In this article, we propose some subspace methods such as the conjugate residual, generalized conjugate residual, biconjugate gradient, conjugate gradient squared and biconjugate gradient stabilized methods based on the tensor forms for solving the tensor equation involving the Einstein product. These proposed algorithms keep the tensor structure. The convergence analysis shows that the proposed methods

In this article, we study preconditioning techniques for the control of the Navier–Stokes equation, where the control only acts on a few parts of the domain. Optimization, discretization, and linearization of the control problem results in a generalized linear saddle‐point system. The Schur complement for the generalized saddle‐point system is very difficult or even impossible to approximate, which

Nonuniform rational B‐splines (NURBS) are the most common representation form in isogeometric analysis. In this article, we study the spectral behavior of discretization matrices arising from isogeometric Galerkin and collocation methods based on d‐variate NURBS of degrees (p1,…,pd), and applied to general second‐order partial differential equations defined on a d‐dimensional domain. The spectrum of

The greater arithmetic intensity of high‐order finite element discretizations makes them attractive for implementation on next‐generation hardware, but assembly of high‐order finite element operators as matrices is prohibitively expensive. As a result, the development of general algebraic solvers for such operators has been an open research challenge. Fast matrix‐free application of high‐order operators

In a recent article, one of the authors developed a multigrid technique for coarse‐graining dynamic powergrid models. A key component in this technique is a relaxation‐based coarsening of the graph Laplacian given by the powergrid network and its weighted graph, which is represented by the admittance matrix. In this article, we use this coarsening strategy to develop a multigrid method for solving

In this article, we study the numerical technique for variable‐order fractional reaction‐diffusion and subdiffusion equations that the fractional derivative is described in Caputo's sense. The discrete scheme is developed based on Lucas multiwavelet functions and also modified and pseudo‐operational matrices. Under suitable properties of these matrices, we present the computational algorithm with high

Bounds on the spectrum of Schur complements of subdomain stiffness matrices of the discretized Laplacian with respect to interior variables are important in the convergence analysis of finite element tearing and interconnecting (FETI)‐based domain decomposition methods. Here, we are interested in bounds on the regular condition number of Schur complements of “floating” clusters, that is, of matrices

In this paper, we propose a numerical scheme based on the iterative method for solving the fractional diffusion‐wave equation involving the Caputo–Weyl fractional derivative of order 0 < α ≤ 2 . The convergence of the approximate solutions of the fractional diffusion‐wave equation is rigourously established. Moreover, the approximate solutions of the fractional diffusion‐wave equation have been presented

The Gram–Schmidt process uses orthogonal projection to construct the A = QR factorization of a matrix. When Q has linearly independent columns, the operator P = I − Q(QTQ)−1QT defines an orthogonal projection onto Q⊥. In finite precision, Q loses orthogonality as the factorization progresses. A family of approximate projections is derived with the form P = I − QTQT, with correction matrix T. When T = (QTQ)−1

Algebraic multigrid (AMG) is a widely used scalable solver and preconditioner for large‐scale linear systems resulting from the discretization of a wide class of elliptic PDEs. While AMG has optimal computational complexity, the cost of communication has become a significant bottleneck that limits its scalability as processor counts continue to grow on modern machines. This article examines the design

We present a greedy algorithm for computing selected eigenpairs of a large sparse matrix H that can exploit localization features of the eigenvector. When the eigenvector to be computed is localized, meaning only a small number of its components have large magnitudes, the proposed algorithm identifies the location of these components in a greedy manner, and obtains approximations to the desired eigenpairs

The goal of the present paper is the design of embeddings of a general sparse graph into a set of points in ℝ d for appropriate d ≥ 2. The embeddings that we are looking at aim to keep vertices that are grouped in communities together and keep the rest apart. To achieve this property, we utilize coarsening that respects possible community structures of the given graph. We employ a hierarchical multilevel