Dimensionality reduction is an important technique in surrogate modeling and machine learning. In this article, we propose a supervised dimensionality reduction method, “least squares regression principal component analysis” (LSR-PCA), applicable to both classification and regression problems. To show the efficacy of this method, we present different examples in visualization, classification, and regression

Sparse principal component analysis (PCA), an important variant of PCA, attempts to find sparse loading vectors when conducting dimension reduction. This paper considers the nonsmooth Riemannian optimization problem associated with the ScoTLASS model 1 for sparse PCA which can impose orthogonality and sparsity simultaneously. A Riemannian proximal method is proposed in the work of Chen et al. 9 for

A hierarchical matrix approach for solving diffusion-dominated partial integro-differential problems is presented. The corresponding diffusion-dominated differential operator is discretized by a second-order accurate finite-volume scheme, while the Fredholm integral term is approximated by the trapezoidal rule. The hierarchical matrix approach is used to approximate the resulting algebraic problem

This article introduces a novel approach to algebraic multigrid methods for large systems of linear equations coming from finite element discretizations of certain elliptic second-order partial differential equations. Based on a discrete energy made up of edge and vertex contributions, we are able to develop coarsening criteria that guarantee two-level convergence even for systems of equations such

Extreme scale simulation requires fast and scalable algorithms, such as multigrid methods. To achieve asymptotically optimal complexity, it is essential to employ a hierarchy of grids. The cost to solve the coarsest grid system can often be neglected in sequential computings, but cannot be ignored in massively parallel executions. In this case, the coarsest grid can be large and its efficient solution

The symmetric tensor decomposition problem is a fundamental problem in many fields, which appealing for investigation. In general, greedy algorithm is used for tensor decomposition. That is, we first find the largest singular value and singular vector and subtract the corresponding component from tensor, then repeat the process. In this article, we focus on designing one effective algorithm and giving

Data assimilation algorithms combine prior and observational information, weighted by their respective uncertainties, to obtain the most likely posterior of a dynamical system. In variational data assimilation the posterior is computed by solving a nonlinear least squares problem. Many numerical weather prediction (NWP) centers use full observation error covariance (OEC) weighting matrices, which can

We introduce the tensor numerical method for solving optimal control problems that are constrained by fractional two- (2D) and three-dimensional (3D) elliptic operators with variable coefficients. We solve the governing equation for the control function which includes a sum of the fractional operator and its inverse, both discretized over large 3D n × n × n spacial grids. Using the diagonalization

In this article, we find necessary and sufficient conditions to identify pairs of matrices X and Y for which there exists Δ ∈ ℂ n , n such that Δ + Δ ∗ is positive semidefinite and Δ X = Y . Such a Δ is called a dissipative mapping taking X to Y. We also provide two different characterizations for the set of all dissipative mappings, and use them to characterize the unique dissipative mapping with

Robust Principal Component Analysis plays a key role in various fields such as image and video processing, data mining, and hyperspectral data analysis. In this paper, we study the problem of robust tensor train (TT) principal component analysis from partial observations, which aims to decompose a given tensor into the low TT rank and sparse components. The decomposition of the proposed model is used

This article is devoted to the computation of the solution to fractional linear algebraic systems using a differential-based strategy to evaluate matrix–vector products A α x , with α ∈ ℝ + ∗ . More specifically, we propose ODE-based preconditioners for efficiently solving fractional linear systems in combination with traditional sparse linear system preconditioners. Different types of preconditioners

The Fréchet derivative L f ( A , E ) of the matrix function f ( A ) plays an important role in many different applications, including condition number estimation and network analysis. We present several different Krylov subspace methods for computing low-rank approximations of L f ( A , E ) when the direction term E is of rank one (which can easily be extended to general low rank). We analyze the convergence

Incomplete factorization is a widely used preconditioning technique for Krylov subspace methods for solving large-scale sparse linear systems. Its multilevel variants, such as ILUPACK, are more robust for many symmetric or unsymmetric linear systems than the traditional, single-level incomplete LU (or ILU) techniques. However, the previous multilevel ILU techniques still lacked robustness and efficiency

In this article, the rank-1 approximation of a nonnegative tensor 𝒜 ∈ ℝ ⩾ 0 n 1 × … × n m is considered. Mathematically, the approximation problem can be formulated as an optimization problem. The Karush–Kuhn–Tucker (KKT) point of the optimization problem can be obtained by computing the nonnegative Z-eigenvector y of enlarged tensor 𝒢 . Therefore, we propose an iterative method with prediction and

We aim to factorize a completely positive matrix by using an optimization approach which consists in the minimization of a nonconvex smooth function over a convex and compact set. To solve this problem we propose a projected gradient algorithm with parameters that take into account the effects of relaxation and inertia. Both projection and gradient steps are simple in the sense that they have explicit

Space‐time methods are able to solve time‐dependent problems faster by exploiting the full power of high‐performance computer. In this article, a multilevel space‐time multiplicative Schwarz method is presented for solving parabolic equations in parallel on both spatial and temporal directions. In the implementation, the proposed Schwarz method is treated as preconditioner for GMRES, that is, a coupled

Tensor completion is a challenging problem with various applications, especially in recovering incomplete visual data. Considering a color image or gray video as a three‐dimensional tensor, many related models based on the low‐rank prior of the tensor have been proposed. However, the low‐rank prior may not be enough to recover the original tensor from the observed incomplete tensor. In this paper,

Overlapping block smoothers efficiently damp the error contributions from highly oscillatory components within multigrid methods for the Stokes equations but they are computationally expensive. This paper is concentrated on the development and analysis of new block smoothers for the Stokes equations that are discretized on staggered grids. These smoothers are nonoverlapping and therefore desirable

We discuss the use of a matrix-oriented approach for numerically solving the dense matrix equation AX + XAT + M1XN1 + … + MℓXNℓ = F, with ℓ ≥ 1, and Mi, Ni, i = 1, … , ℓ of low rank. The approach relies on the Sherman–Morrison–Woodbury formula formally defined in the vectorized form of the problem, but applied in the matrix setting. This allows one to solve medium size dense problems with computational

Local Fourier analysis (LFA) serves a prominent role in the prediction of the convergence factor of multigrid methods for discretizations of PDEs. However, few discussions about the implementation of LFA for complicated discretizations of PDEs exist, such as higher order finite elements, as staggered meshes could lead to complex LFA representations of the grid‐transfer operator. In this work, we prove

Efficient parallel‐in‐time methods for hyperbolic partial differential equation problems remain scarce. Here we investigate an approach based on circulant preconditioned generalised minimal residual (GMRES) for the monolithic block Toeplitz equations which arise from constant time‐step discretizations. We present theoretical results which guarantee convergence in a number of iterations independent

We consider the numerical solution of large scale singular (continuous-time) Lyapunov equations of the form AX + XA⊤ + BB⊤ = 0, where A is semistable, that is, its spectrum is contained in the left half plane, with the exception of a few semisimple eigenvalues at zero. We also consider the case of a few semisimple eigenvalues on the imaginary axis. We assume that we know these few eigenvalues (zero

In this paper, we consider the nonnegative tensor least squares problem, which arises in the color image restoration. Based on the BB stepsize technique, we design a nonmonotonic descent stepsize and then derive a new gradient projection algorithm to solve this problem. The convergence analysis of the new gradient projected algorithm is given. Some numerical examples show that the new method is feasible

This article shows that the bidiagonal decomposition of many important matrices of q-integers can be constructed to high relative accuracy (HRA). This fact can be used to compute with HRA the eigenvalues, singular values, and inverses of these matrices. These results can be applied to collocation matrices of q-Laguerre polynomials, q-Pascal matrices, and matrices formed by q-Stirling numbers. Numerical

This special issue “Multigrid Methods” in the Journal of Numerical Linear Algebra with Applications contains nine papers submitted by participants from the 19th Copper Mountain Conference on Multigrid Methods, held in the Colorado Rocky Mountains March 24–28, 2019. The papers cover a range of topics on Multigrid, including algorithm development for a variety of applications, convergence and complexity

Numerical methods for elliptic partial differential equations usually lead to systems of linear equations with sparse, symmetric, and positive definite matrices. In many methods, these matrices can be obtained as sums of local symmetric positive semidefinite matrices. In this article, we use this assumption and introduce a method that provides guaranteed lower and upper bounds to all individual eigenvalues

An algorithm for two-variable rational interpolation is developed. The algorithm is suitable for interpolation cases where neither the number of interpolation points nor the final degrees of the rational interpolant are known a priori. Instead, a maximum degree for the interpolant's numerator and denominator is assumed. By testing the condition number of the interpolation system's matrix at each step

Problems arising in Earth's mantle convection involve finding the solution to Stokes systems with large viscosity contrasts. These systems contain localized features which, even with adaptive mesh refinement, result in linear systems that can be on the order of 109 or more unknowns. One common approach for preconditioning to the velocity block of these systems is to apply an Algebraic Multigrid (AMG)

The reduction of a large-scale symmetric linear discrete ill-posed problem with multiple right-hand sides to a smaller problem with a symmetric block tridiagonal matrix can easily be carried out by the application of a small number of steps of the symmetric block Lanczos method. We show that the subdiagonal blocks of the reduced problem converge to zero fairly rapidly with increasing block number.

The solution of large sparse linear systems is required in many scientific fields such as computational fluid dynamics, computational electromagnetism, computational finance, etc. The computation of the solution of these systems is performed with preconditioned iterative methods, which rely on effective preconditioning schemes. A new class of approximate inverses is proposed, namely incomplete inverse

We propose an inexact coordinate descent method to solve a consistent equation of the type Ax = b where A is a real symmetric and positive semidefinite matrix. The admissible range of inexactitude allows to reduce every multiplication present in the computation to a scaling by a signed power of 2 for the sake of hardware simplification. Although the resulting descent is nonideal, the numerical experiments

Fixed-point iteration is a popular method for the multilinear PageRank problem. In [B. Meini, F. Poloni, Perron-based algorithms for the multilinear PageRank, Numer. Linear Algebra Appl., 25: e2177, 2018], a theorem was proved to determine the behavior of the fixed-point iteration as the parameter α ∈ ( 1 2 , 1 ) . In this work, we point out that this theorem is incomplete, and revisit the convergence

This paper presents the two-parameter scaling memoryless Broyden–Fletcher–Goldfarb–Shanno (BFGS) method for solving a system of monotone nonlinear equations. The optimal values of the scaling parameters are obtained by minimizing the measure function involving all the eigenvalues of the memoryless BFGS matrix. The optimal values can be used in the analysis of the quasi-Newton method for ill-conditioned

It is known that the solutions to space-fractional diffusion equations exhibit singularities near the boundary. Therefore, numerical methods discretized on the composite mesh, in which the mesh size is refined near the boundary, provide more precise approximations to the solutions. However, the coefficient matrices of the corresponding linear systems usually lose the diagonal dominance and are ill-conditioned

This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates the empirical convergence speed of most existing block-coordinate algorithms for NTF

Principal component analysis (PCA) is widely used for dimensionality reduction and unsupervised learning. The reconstruction error is sometimes large even when a large number of eigenmode is used. In this paper, we show that this unexpected error source is the pollution effect of a summation operation in the objective function of the PCA algorithm. The summation operator brings together unrelated parts

Solving linear systems is often the computational bottleneck in real-life problems. Iterative solvers are the only option due to the complexity of direct algorithms or because the system matrix is not explicitly known. Here, we develop a two-level preconditioner for regularized least squares linear systems involving a feature or data matrix. Variants of this linear system may appear in machine learning

In this paper, we present explicit expressions for the mixed and componentwise condition numbers of the truncated total least squares (TTLS) solution of A x ≈ b under the genericity condition, where A is a m × n real data matrix and b is a real m-vector. Moreover, we reveal that normwise, componentwise, and mixed condition numbers for the TTLS problem can recover the previous corresponding counterparts

The growing discrepancy between CPU computing power and memory bandwidth drives more and more numerical algorithms into a bandwidth-bound regime. One example is the overlapping Schwarz smoother, a highly effective building block for iterative multigrid solution of elliptic equations with higher order finite elements. Two options of reducing the required memory bandwidth are sparsity exploiting storage

In [Hayami K, Sugihara M. Numer Linear Algebra Appl. 2011; 18:449–469], the authors analyzed the convergence behavior of the generalized minimal residual (GMRES) method for the least squares problem min x ∈ R n ‖ b − A x ‖ 2 2 , where A ∈ Rn × n may be singular and b ∈ R n , by decomposing the algorithm into the range ℛ ( A ) and its orthogonal complement ℛ ( A ) ⊥ components. However, we found that

Parallel-in-time methods, such as multigrid reduction-in-time (MGRIT) and Parareal, provide an attractive option for increasing concurrency when simulating time-dependent partial differential equations (PDEs) in modern high-performance computing environments. While these techniques have been very successful for parabolic equations, it has often been observed that their performance suffers dramatically

Based on the spectral divide-and-conquer algorithm by Nakatsukasa and Higham [SIAM J. Sci. Comput., 35(3):A1325–A1349, 2013], we propose a new algorithm for computing all the eigenvalues and eigenvectors of a symmetric banded matrix with small bandwidth, with the eigenvectors given implicitly as a product of orthonormal matrices stored in the so-called hierarchically off-diagonal low-rank (HODLR) format

The generalized singular value decomposition (GSVD) is a valuable tool that has many applications in computational science. However, computing the GSVD for large-scale problems is challenging. Motivated by applications in hyper-differential sensitivity analysis (HDSA), we propose new randomized algorithms for computing the GSVD which use randomized subspace iteration and weighted QR factorization.

The l parameterized generalized eigenvalue problems for the nonsquare matrix pencils, proposed by Chu and Golub [SIAM J. Matrix Anal. Appl., 28(2006), pp. 770-787], can be formulated as an optimization problem on a corresponding complex product Stiefel manifold. Some early proposed algorithms are based on the first-order information of the objective function, and fast convergence could not be expected

The radial point interpolation meshfree discretization is a very efficient numerical framework for the analysis of piezoelectricity, in which the fundamental electrostatic equations governing piezoelectric media are solved without mesh generation. Due to the mechanical-electrical coupling property and the piezoelectric constant, the discrete linear system is sparse, of generalized saddle point form

Discrete ill‐posed inverse problems arise in many areas of science and engineering. Their solutions are very sensitive to perturbations in the data. Regularization methods aim at reducing this sensitivity. This article considers an iterative regularization method, based on iterated Tikhonov regularization, that was proposed in M. Donatelli and M. Hanke, Fast nonstationary preconditioned iterative methods

In [1], the following error was published on page 1. The affiliation of the second author, Pei Yuan, read: 2Institute of Computing Technology, Beijing, 100190, China. The affiliation of the second author, Pei Yuan, should read: 2Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China. 3University of Chinese Academy of Sciences, Beijing, 100049, China.

A new method is proposed to compute the eigenpairs of a parameter-dependent nonlinear eigenvalue problem. We first analyze the properties on the analytic perturbation of the invariant pair of a nonlinear eigenvalue problem and provide a method to compute the first-order and high-order derivatives of the invariant pair. On these grounds, the subspaces independent of the parameter and containing the

In this article, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal-dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill-conditioned linear

Image denoising is a typical inverse problem and is hard to be solved. Fortunately, a powerful two-phase method for restoring images corrupted by high-level impulse noise has been proposed. The key point of the method is the computational efficiency of the second phase which requires the minimization of a smooth objective function defined on the terms of edge-preserving potential function. In this

In this article, we present a reliable hybrid algorithm for solving convex quadratic minimization problems. At the kth iteration, two points are computed: first, an auxiliary point x ˙ k is generated by performing a gradient step using an optimal steplength, and second, the next iterate xk + 1 is obtained by means of weighted sum of x ˙ k with the penultimate iterate xk − 1. The coefficient of the

Scalable approaches for uncertainty quantification are necessary for characterizing prediction confidence in large‐scale subsurface flow simulations with uncertain permeability. To this end we explore a multilevel Monte Carlo approach for estimating posterior moments of a particular quantity of interest, where we employ an element‐agglomerated algebraic multigrid (AMG) technique to generate the hierarchy

In the past decades, multigrid methods for linear systems having multilevel Toeplitz coefficient matrices with scalar entries have been widely studied. On the other hand, only few papers have investigated the case of block entries, where the entries are small generic matrices of fixed size instead of scalars. In that case the efforts of the researchers have been mainly devoted to specific applications

Time–space fractional Fokker–Planck equations (TSFFPEs) are a class of very useful models for describing some practical phenomena in statistical physics. In the present article, we focus on the fast computation for semilinear two-dimensional TSFFPEs. A stable implicit difference scheme with second-order accuracy in time and space is derived. In order to accelerate the convergence rate of the scheme

The row iterative method is popular in solving the large‐scale ill‐posed problems due to its simplicity and efficiency. In this work we consider the randomized row iterative (RRI) method to tackle this issue. First, we present the semiconvergence analysis of RRI method for the overdetermined and inconsistent system, and derive upper bounds for the noise error propagation in the iteration vectors. To

We propose to reduce the (spectral) condition number of a given linear system by adding a suitable diagonal matrix to the system matrix, in particular by shifting its spectrum. Iterative procedures are then adopted to recover the solution of the original system. The case of real symmetric positive definite matrices is considered in particular, and several numerical examples are given. This approach

In this article, we consider the stochastic inverse singular value problem (ISVP) of constructing a stochastic matrix from the prescribed realizable singular values. We propose a Riemannian inexact Newton‐CG method with various choices of forcing terms for solving the stochastic ISVP. We show the proposed method converges linearly or superlinearly for different forcing terms under some assumptions

We investigate the method of conjugate gradients, exploiting inaccurate matrix‐vector products, for the solution of convex quadratic optimization problems. Theoretical performance bounds are derived, and the necessary quantities occurring in the theoretical bounds estimated, leading to a practical algorithm. Numerical experiments suggest that this approach has significant potential, including in the

High‐dimensionality reduction techniques are very important tools in machine learning and data mining. The method of generalized low rank approximations of matrices (GLRAM) is a popular technique for dimensionality reduction and image compression. However, it suffers from heavily computational overhead in practice, especially for data with high dimension. In order to reduce the cost of this algorithm

Block Krylov subspace methods (KSMs) comprise building blocks in many state‐of‐the‐art solvers for large‐scale matrix equations as they arise, for example, from the discretization of partial differential equations. While extended and rational block Krylov subspace methods provide a major reduction in iteration counts over polynomial block KSMs, they also require reliable solvers for the coefficient