New hybrid three-term spectral-conjugate gradient method for finding solutions of nonlinear monotone operator equations with applications

Abstract

In this paper, we present a new hybrid spectral-conjugate gradient (SCG) algorithm for finding approximate solutions to nonlinear monotone operator equations. The hybrid conjugate gradient parameter has the Polak–Ribière–Polyak (PRP), Dai–Yuan (DY), Hestenes–Stiefel (HS) and Fletcher–Reeves (FR) as special cases. Moreover, the spectral parameter is selected such that the search direction has the descent property. Also, the search directions are bounded and the sequence of iterates generated by the new hybrid algorithm converge globally. Furthermore, numerical experiments were conducted on some benchmark nonlinear monotone operator equations to assess the efficiency of the proposed algorithm. Finally, the algorithm is shown to have the ability to recover disturbed signals.

Introduction

Our aim in this paper is to propose a new hybrid algorithm for finding approximate solutions to the problem

$$\vartheta \left(y\right)=0,y\in \mathcal{D},$$

where the underlying operator $\vartheta :{\mathbf{R}}^n\to {\mathbf{R}}^n$ is monotone and Lipschitz continuous, $\mathcal{D}\subseteq {\mathbf{R}}^n$ is nonempty, closed and convex. A lot of researchers are now motivated and interested in solving problem (1) because of its applicability in many areas such as forecasting of financial market [18], constrained neural networks [17], economic and chemical equilibrium problems [20], [34], nonlinear compressed sensing [6], [10], [15], [27], [28], phase retrieval [16], [41], power flow equations [37], non-negative matrix factorization [13], [31] and lots more.

Among the trending methods for finding solutions to (1) are the conjugate gradient (CG) and spectral (SP) gradient methods. These methods are famous recently because of their simplicity in implementation and ability to solve large-scale problems. In addition, they require only the function value of the operator defined in (1). An important aspect of these methods is the definition of the parameters involved, that is the CG parameter (as in the case of CG methods) and the SP parameter (as in the case of SP methods). Some notable CG parameters are the Polak–Ribière–Polyak (PRP), Dai–Yuan (DY), Hestenes–Stiefel (HS) and Fletcher–Reeves (FR) defined as follows:

$${\beta }_k^{PRP}=\frac{{\vartheta }_k^T{z}_{k-1}}{{\Vert {\vartheta }_{k-1}\Vert }^2},{\beta }_k^{DY}=\frac{{\Vert {\vartheta }_k\Vert }^2}{d_{k-1}^T{z}_{k-1}},{\beta }_k^{HS}=\frac{{\vartheta }_k^T{z}_{k-1}}{d_{k-1}^T{z}_{k-1}}{\beta }_k^{FR}=\frac{{\Vert {\vartheta }_k\Vert }^2}{{\Vert {\vartheta }_{k-1}\Vert }^2},$$

where ${\vartheta }_k=\vartheta \left(y_k\right),z_{k-1}={\vartheta }_k-{\vartheta }_{k-1}$. For more on CG methods, we refer the reader to [1], [2], [3], [4], [8], [9], [11], [22], [23], [24], [25], [26], [36].

Several algorithms have been proposed using the above parameters or their modified version. For example, Ahookhosh et al. [12] proposed a three-term conjugate gradient algorithm to solve (1). The algorithm is a modified version of the PRP algorithm where an additional term was introduced to achieve a descent search direction. The global convergence was proved under some assumptions and some numerical examples were provided to support it. Liu and Feng [32] also proposed a modified DY algorithm by introducing a spectral parameter to achieve a descent direction. The algorithm was shown to be globally convergent and numerically efficient. In [39], Yan et al. proposed a modified three-term HS algorithm, where a new term was introduced and the HS parameter modified to achieve a descent direction. Recently Abubakar et al. [7] proposed a modified three-term FR algorithm. The search direction is descent and bounded. Also, the algorithm proves to be numerically competitive.

In recent times, researchers are becoming interested in hybrid algorithms for solving (1). The hybrid algorithms combine two or more CG parameters using a convex combination approach or other generalization approaches. For example, Koorapetse and Kaelo [29] proposed a new hybrid projection method for solving (1). The algorithm can be seen as a combination of the HS and DY algorithms. Sabiu et al. [35] proposed a hybrid conjugate gradient method that combines the FR and the PRP methods. Besides, Ibrahim et al. [25] proposed a hybrid conjugate gradient algorithm where the search direction is a convex combination of the Liu–Storey (LS) and the FR CG parameters. Likewise, a spectral gradient projection method that selects its search direction as a convex combination of two different spectral parameters was proposed by Abubakar et al. [5]. It is worth mentioning that the above hybrid algorithms were shown to be numerically more efficient than the classical algorithms. Furthermore, we note that not many of the hybrid algorithms for solving (1) are available in the literature.

Inspired by the above hybrid methods, we propose a new hybrid spectral-conjugate gradient (SCG) method for solving (1). The new hybrid method to the best of our knowledge is the first of its kind and has as special cases, the PRP method, the DY-type method [32], the HS-type method [39], and the FR method. The search direction is descent and bounded independent of the line search. The sequence generated by the algorithm converge globally under appropriate conditions. Using some benchmark test problems, numerical examples were given to assess the efficiency of the new algorithm. Among the advantages of the new hybrid method is that it possess the nice properties of the PRP, DY, HS and FR methods, which makes it computationally more efficient than existing methods. The rest of the paper is organized as follows. In Section 2, the new hybrid method is introduced. Theoretical results are provided in Section 3. Numerical experiments on some benchmark nonlinear monotone operator equations and application are given in Sections 4 Numerical experiments on some nonlinear monotone operator equations, 5 Application on sparse signal reconstruction, respectively. Section 6 contains the conclusion.

Notation. Unless otherwise stated, the symbol $\Vert \cdot \Vert$ stands for Euclidean norm on ${\mathbf{R}}^n$. $\vartheta \left(y_k\right)$ is abbreviated to ${\vartheta }_k$. Furthermore, $P_{\mathcal{D}}\left[\cdot \right]$ is the projection mapping from ${\mathbf{R}}^n$ onto $\mathcal{D}$ given by $P_{\mathcal{D}}\left[y\right]=argmin\{\Vert y-z\Vert :y\in {\mathbf{R}}^n,z\in \mathcal{D}\}$, for a nonempty closed and convex set $\mathcal{D}\in {\mathbf{R}}^n$.