Abstract
We use the concept of barrier-based smoothing approximations to extend the non-interior continuation method, which was proposed by B. Chen and N. Xiu for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions, to an inexact non-interior continuation method for variational inequalities over general closed convex sets. Newton equations involved in the method are solved inexactly to deal with high dimension problems. The method is proved to have global linear and local superlinear/quadratic convergence under suitable assumptions. We apply the method to non-negative orthants, positive semidefinite cones, polyhedral sets, epigraphs of matrix operator norm cone and epigraphs of matrix nuclear norm cone.
Similar content being viewed by others
References
Bai, Z.-Z., Golub, G.H., Lu, L.-Zh., Yin, J.-F.: Block triangular and skew-hermitian splitting methods for positive-definite linear systems. SIAM J. Sci. Comput. 26(3), 844–863 (2005)
Bai, Z-Z., Golub, G. H., Ng, M. K.: Hermitian and skew-hermitian splitting methods for non-hermitian positive definite linear systems. SIAM J. Matrix Anal. Appl. 24(3), 603–626 (2003)
Bellavia, S., Morini, B.: A globally convergent newton-gmres subspace method for systems of nonlinear equations. SIAM J. Sci. Comput. 23(3), 940–960 (2001)
Bhatia, R.: Matrix analysis. Springer, Berlin (1997)
Brown, P. N., Saad, Y.: Hybrid krylov methods for nonlinear systems of equations. SIAM J. Sci. Stat. Comput. 11(3), 450–481 (1990)
Burke, J., Xu, S.: The global linear convergence of a noninterior path-following algorithm for linear complementarity problem. Math. Oper. Res. 23, 719–734 (1998)
Chen, B., Harker, P. T.: A non-interior-point continuation method for linear complementarity problems. SIAM J. Matrix Anal. Appl. 14(4), 1168–1190 (1993)
Chen, B., Harker, P. T.: A continuation method for monotone variational inequalities. Math. Program. 69(1), 237–253 (1995)
Chen, B., Xiu, N.: A global linear and local quadratic noninterior continuation method for nonlinear complementarity problems based on Chen-Mangasarian smoothing functions. SIAM J. Optim. 9(3), 605–623 (1999)
Chen, C., Mangasarian, O. L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Comput. Optim. Appl. 5(2), 97–138 (1996)
Chen, C. H.: Numerical Algorithms for a Class of Matrix Norm Approximation Problems. PhD thesis, Nanjing University (2012)
Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing newton method and its application to general box constrained variational inequalities. Math. Comput. 67(222), 519–540 (1998)
Chen, X., Tseng, P.: Non-interior continuation methods for solving semidefinite complementarity problems. Math. Program. 95(3), 431–474 (2003)
Chen, X. J., Ye, Y.: On homotopy-smoothing methods for box-constrained variational inequalities. SIAM J. Control Optim. 37(2), 589–616 (1999)
Chua, C. B., Hien, L. T. K.: A superlinearly convergent smoothing newton continuation algorithm for variational inequalities over definable sets. SIAM J. Optim. 25(2), 1034–1063 (2015)
Chua, C. B., Li, Z.: A barrier-based smoothing proximal point algorithm for NCPsover closed convex cones. SIAM J. Optim. 23(2), 745–769 (2013)
Chua, C. B., Yi, P.: A continuation method for nonlinear complementarity problems over symmetric cones. SIAM J. Optim. 20, 2560–2583 (2010)
de Sá, E.: Faces of the unit ball of a unitarily invariant norm. Linear Algebra Appl. 197,198, 451–493 (1994)
Dembo, R. S., Eisenstat, S. C., Steihaug, T.: On newton-like methods. SIAM J. Numer. Anal. 19, 400–408 (1982)
Dennis, J. E. Jr: On newton-like methods. Numer. Math. 11(4), 324–330 (1968)
Ding, C., Sun, D., Toh, K.-Ch.: An introduction to a class of matrix cone programming. Math. Program. 144(1-2), 141–179 (2014)
Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer, New York (2003)
Facchinei, F., Pang, J. -S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. II. Springer, New York (2003)
Gabriel, S. A., Moré, J.J.: Smoothing of mixed complementarity problems (1995)
Kanzow, C.: Some noninterior continuation methods for linear complementarity problems. SIAM J. Matrix Anal. Appl. 17(4), 851–868 (1996)
Knoll, D. A., Keyes, D. E.: Jacobian-free newton–krylov methods: a survey of approaches and applications. J. Comput. Phys. 193(2), 357–397 (2004)
Li, D., Fukushima, M.: Smoothing newton and quasi-newton methods for mixed complementarity problems. Comput. Optim. Appl. 17(2), 203–230 (2000)
Nesterov, Y.E., Nemirovski, A. S.: Interior Point Polynomial Algorithms in Convex Programming. SIAM Stud. Appl. Math. SIAM Publication, Philadelphia (1994)
Pang, J.-S.: Newton’s method for B-differentiable equations. Math. Oper. Res. 15(2), 311–341 (1990)
Pang, J. S.: A B-differentiable equation based, globally, and locally quadratically convergent algorithm for nonlinear programs, complementarity and variational inequality problems. Math. Program. 51, 101–131 (1991)
Pataki, G., Tunċel, L.: On the generic properties of convex optimization problems in conic form. Math. Program. 89(3), 449–457 (2001)
Pernice, M., Walker, H. F.: Nitsol: A newton iterative solver for nonlinear systems. SIAM J. Sci. Comput. 19(1), 302–318 (1998)
Qi, L., Sun, D.: Smoothing functions and smoothing Newton method for complementarity and variational inequality problems. J. Optim. Theory Appl. 113(1), 121–147 (2002)
Qi, L., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58(1-3), 353–367 (1993)
Ralph, D.: Global convergence of damped Newton’s method for nonsmooth equations, via the path search. Math. Oper. Res. 19, 352–389 (1994)
Saad, Y., Schultz, M. H.: Gmres: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)
Stoer, J.: Solution of Large Linear Systems of Equations by Conjugate Gradient Type Methods, pp 540–565. Springer, Berlin (1983)
Xu, S.: The global linear convergence of an infeasible non-interior path-following algorithm for complementarity problems with uniform p-functions. Math. Program. 87 (3), 501–517 (2000)
Acknowledgments
We thank the anonymous reviewers for their meticulous and insightful comments, which help us improve the paper. LTKH gives special thanks to Prof. Nicolas Gillis for his support.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A: Technical proofs
1.1 A.1 Proof of Proposition 3
(i) From Eq. 9 we get
(ii) We have
where we have used the property \(\left \lVert {(a,b)} \right \rVert \leq \left \lVert {a} \right \rVert + \left \lVert {b} \right \rVert \) for the first inequality and Theorem 1 for the last inequality. Inequality (48) with (11) give us
1.2 A.2 Proof of Theorem 5
1.2.1 A.2.1 Non-negative orthant \(\mathbb {R}^{n}_{+}\)
Gradient and Hessian of the barrier function are
where ei denote the i −th standard unit vector of \(\mathbb {R}^{n}\). The corresponding barrier-based smoothing approximation is \(p_{\mu }(z)=\frac {1}{2}\sum \limits _{i=1}^{n}\left ({z_{i}+ \sqrt {{z_{i}^{2}} + 4\mu ^{2}}}\right ) e_{i}\). Its Jacobian is
The projection of z onto \(\mathbb {R}^{n}_{+}\) is \({\Pi }_{\mathbb {R}^{n}_{+}}(z)=[z]_{+}\). We observe that the projector is differentiable at z∗ if and only if \(z_{i}^{*}\ne 0, \forall i=1,\ldots ,n\). On the other hand, a pair (x∗,y∗) is strictly complimentary if and only if \(x_{i}^{*}+y_{i}^{*} >0\) for all i = 1,…,n, see [9]. Furthermore, \(x^{*}+y^{*}={\Pi }_{\mathbb {R}^{n}_{+}}(z^{*})+{\Pi }_{\mathbb {R}^{n}_{+}}(-z^{*})\). Hence, it is easy to see that differentiability of the projector at z∗ is equivalent to strict complimentarity of (x∗,y∗).
Now let \(z_{i}^{*}\ne 0\) for i = 1,…,n, then we observe that the Jacobian Jpμ(z) converges to
when \((z,\mu )\rightarrow (z^{*},0)\). Since the map (z,μ)↦Jpμ(z) is continuously differentiable at (z∗,0), thus is locally Lipschitz at this point. Consequently, \(\left \lVert {\text {\textbf {J}} p_{\mu }(z)-T^{*}}\right \rVert = O(\left \lVert {(z-z^{*},\mu )}\right \rVert )\). On the other hand, \(\left \lVert {\textup {D} p_{\mu }(z)-\textup {D} {\Pi }_{\mathbb {R}^{n}_{+}}(z^{*})}\right \rVert =\left \lVert {\text {\textbf {J}} p_{\mu }(z)-\text {\textbf {J}} {\Pi }_{\mathbb {R}^{n}_{+}}(z^{*})}\right \rVert \). Thus, we get the result.
1.2.2 A.2.2 Positive semidefinite cone \(\mathbb {S}_{+}^{n}\)
We have ∇f(x) = −x− 1.
From the equation x + μ2∇f(x) = z, we deduce that the corresponding barrier-based smoothing approximation is
Denote \(\mathfrak {g}: u\in \mathbb {R} \mapsto \mathfrak {g}(u)= u+ \sqrt {u^{2} + 4\mu ^{2}}\), \(\mathfrak {g}^{\prime }(u)=1+ \frac {u}{\sqrt {u^{2} + 4\mu ^{2}}}\) and \(\mathfrak {g}^{(1)}\) is a matrix whose (i,j)-th entry with respect to a vector d is
Let z = q Diag(λf(z))qT, then \(\textup {D} p_{\mu }(z)[h]= \frac {1}{2} q \left [ \mathfrak {g}^{(1)} (\lambda _{f}(z)) \circ (q^{T} h q)\right ] q^{T}. \) Projection of z onto \(\mathbb {S}^{n}_{+}\) is \({\Pi }_{\mathbb {S}^{n}_{+}}(z)=q\textup {Diag}\left ({[\lambda _{f}(z)]_{+}} \right )q^{T}\). We see that \({\Pi }_{\mathbb {S}^{n}_{+}}(\cdot )\) is differentiable at z∗ if and only if all eigenvalues \(\lambda _{i}^{*}\), for i = 1,…,n, of z∗ are non-zeroes. Furthermore, strict complementarity of (x∗,y∗) is equivalent to the condition that all eigenvalues of x∗ + y∗ is positive. We now let \(z^{*}=\hat {q}\textup {Diag}(\lambda _{f}(z^{*})) \hat {q}^{T} \) be the eigenvalue decomposition of z∗. Then,
Hence differentiability of \({\Pi }_{\mathbb {S}^{n}_{+}}(\cdot )\) at z∗ is equivalent to strict complementarity of (x∗,y∗).
Now we consider z∗ whose eigenvalues are non-zeros. Let (z,μ) go to (z∗,0), then λf(z) converges to λ∗. Let \(\bar {q}\) be a limit point of q. We then have \(z^{*}=\bar {q} \textup {Diag}(\lambda ^{*}) \bar {q}^{T}\) with \(\bar {q} \in \mathcal {O}^{n}(z^{*}). \) We deduce from \(\lambda _{i}^{*} \ne 0\), i = 1,…,n that
Therefore, by Theorem 2, when (z,μ) → (z∗,0), where z∗ are differential points of \({\Pi }_{\mathbb {S}^{n}_{+}}(\cdot )\), Dpμ(z) converges to \(\textup {D} {\Pi }_{\mathbb {S}^{n}_{+}}(z^{*})\) with
where \(\bar {\mathfrak {g}}^{(1)}(\lambda ^{*})_{ij}= 1+ \frac {\lambda _{i}^{*} + \lambda _{j}^{*}}{|\lambda _{i}^{*}| + | \lambda _{j}^{*}|}\). Note that Formula (49) is independent of the choice \(\bar {q}\). Finally, similarly to the case \(\mathbb {R}_{+}^{n}\), we have \(\left \lVert {\textup {D} p_{\mu }(z) - \textup {D} {\Pi }_{\mathbb {S}^{n}_{+}}(z^{*})}\right \rVert =O(\left \lVert {z-z^{*},\mu }\right \rVert )\) since (z,μ)↦Dpμ(z) is locally Lipschitz around (z∗,0).
1.3 A.3 Proof of Proposition 8
We note that \(\mathcal {F}_{I^{*}}\) is the unique neighbour face of z∗ and \(z^{*} \in \textup {int} (\mathcal {F}_{I} + \mathcal {N}_{I})\) as the projector is differentiable at z∗ (see Proposition 7). When \((z,\mu )\rightarrow (z^{*},0)\), we have \(x\rightarrow {\Pi }_{K}(z^{*})=\bar {z}^{*},\) satisfying \(A_{i}\bar {z}^{*} =b_{i}, \forall i\in \mathcal {I}^{*}\) and \(A_{i}\bar {z}^{*} > b_{i}, \forall i\not \in \mathcal {I}^{*}.\) From Eq. 22 we have
If there exists \(j\in \mathcal {I}^{*}\) and a subsequence \((z,\mu )_{k}\rightarrow (z^{*},0)\) such that \(\frac {{\mu _{k}^{2}}}{A_{j} x_{k} - b_{j}} \rightarrow 0\) then we take the limit of this subsequence in (50) to get
On the other hand, \(\bar {z}^{*}\in \mathcal {F}_{\mathcal {I}^{*}\setminus \{j\}}\) as \(A_{i}\bar {z}^{*}=b_{i} \forall i\in \mathcal {I}^{*}\setminus \{j\}\). Hence \(z^{*} = \bar {z}^{*} + z^{*} -\bar {z}^{*} \in \mathcal {F}_{I^{*}\setminus \{j\}} + \mathcal {N}_{\mathcal {I}^{*}\setminus \{j\}}\), which implies \(\mathcal {I}^{*}\setminus \{j\}\) is a neighbour face of z∗. This contradicts to the fact \(\mathcal {I}^{*}\) is the unique neighbour face of z∗. Therefore, for all \(i\in \mathcal {I}^{*}, \frac {\mu ^{2}}{A_{i} x -b_{i}}\) only have nonzero limit points. The result follows then.
1.4 A.4 Proof of Proposition 9
By re-indexing z∗ if necessary, we can assume that π is the identity permutation. For each i ∈{1,…,n}, if \(|x_{i}^{*}| < t^{*}\) then
Together with sgn(xi) = sgn(zi) and t > |xi|, we deduce that \(x_{i}^{*}=\textup {sgn}(z_{i}^{*}) t^{*} \text {or} z_{i}^{*} \text {for} i=1,\ldots ,n. \) Moreover, \(|x_{i}| < {\min \limits } \{t,|z_{i}| \}\) further implies that
Thus, there exists a unique positive integer k∗ such that
and \(|z_{\mathbf k^{*}}^{*} |> t^{*} \geq |z_{\mathbf k^{*}+1}^{*}|\). Summing up (n + 1) equations in (26) gives
If t∗ > 0 then taking limnit gives, \(z_{o}^{*}+\sum \limits _{i=1}^{n} |z_{i}^{*}| = t^{*} + \sum \limits _{i=1}^{n}|x_{i}^{*}|\), and hence
If t∗ = 0 then |xi| < t implies that \(x_{i}^{*}=0\) for i = 1,…,n. Thus
Subsequently, \(z_{o}^{*} + \sum \limits _{i=1}^{\mathbf k^{*}}|z_{i}^{*}| \leq z_{o}^{*} + \sum \limits _{i=1}^{n}|z_{i}^{*}|\leq 0\). Hence,
Appendix B: Example
We consider the second order cone in \(\mathbb {R}^{3}\)
Firstly, we use the barrier \(f^{(1)}(t,z)=-\log (t^{2}-\|z\|^{2}).\) Denote \(\mathbf M=t^{2} - {z_{1}^{2}} - {z_{2}^{2}}\). The gradient of f(1) is
The smoothing approximation \(p^{(1)}_{\mu }(t^{o},z^{o})=(t,z)\) regarding to f(1)(t,z) is computed by
The unique solution of (51) is
Denote \(s_{1}=\sqrt {(t^{o}-\left \lVert {z^{o}} \right \rVert )^{2} + 8\mu ^{2}}, s_{2}=\sqrt {(t^{o}+\left \lVert {z^{o}} \right \rVert )^{2} + 8\mu ^{2}}\). We have
where
We choose \((t^{o},{z_{1}^{o}},{z_{2}^{o}})\) such that \( (t^{o},{z_{1}^{o}},{z_{2}^{o}}) \to (t^{*},z_{1}^{*},z_{2}^{*})=(0,1,0)\); then \(s_{1}\to 1, s_{2} \to 1, \left \lVert {z^{o}} \right \rVert \to 1\). We imply
which equals to \(\text {\textbf {J}}{\Pi }_{K_{2}}(0,1,0)\) by Theorem 2. Now we use another barrier
Denote Mμ = u2 − v2 − w2. Gradient ∇f(2)(u,v1,v2) of f(2) is
Let \(p_{\mu }^{(2)}(t^{*},z_{1}^{*},z_{2}^{*})=p_{\mu }^{(2)}(0,1,0)=(u,v_{1},v_{2})\), which is defined by
The third equation implies v2 = 0, hence \(\mathbf M_{\mu }=u^{2} - {v_{1}^{2}} =(u-v_{1})(u+v_{1}).\) Thus, the first and the second equation imply
which give \(u=\frac {1}{2}\sqrt {1+ 16\mu ^{2}}, v_{1}=\frac {1}{2}, \mathbf M_{\mu } = 4\mu ^{2}\). Denote \(a=\sqrt {1+ 16\mu ^{2}}\). Hessian matrix ∇2f(2)(u,v1,v2) of the barrier f(2) at (u,v1,v2) is
We remind that \( \text {\textbf {J}} p^{(2)}_{\mu }(0,1,0)=[I + \mu ^{2} \nabla ^{2} f^{(2)}(u,v_{1},v_{2})]^{-1}=\left (\begin {array}{ccc} 1/2 & \frac {1}{2a} &0 \\ \frac {1}{2a} & 1/2&0\\ 0 & 0 & 2/3 \end {array}\right ). \) Thus, \(\lim \limits _{\mu \rightarrow 0}\text {\textbf {J}} p^{(2)}_{\mu }(0,1,0)\) equals \(\left (\begin {array}{ccc} 1/2&1/2&0\\ 1/2&1/2&0\\ 0&0&2/3 \end {array}\right ).\) It does not coincide with the Jacobian of \({\Pi }_{K_{2}}\) at (0,1,0). This shows that the limit \(\lim \limits _{(t^{o},{z_{1}^{o}},{z_{2}^{o}},\mu )\rightarrow (0,1,0,0)}\textup {D} p^{(2)}_{\mu }(t^{o},z^{o})\) does not exist.
Rights and permissions
About this article
Cite this article
Hien, L.T.K., Chua, C.B. A Global Linear and Local Superlinear (Quadratic) Inexact Non-Interior Continuation Method for Variational Inequalities Over General Closed Convex Sets. Set-Valued Var. Anal 29, 109–144 (2021). https://doi.org/10.1007/s11228-020-00540-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11228-020-00540-6
Keywords
- Inexact non-interior continuation method
- Variational inequality
- Smoothing approximation
- Polyhedral set
- Epigraph of matrix operator norm
- Epigraph of matrix nuclear norm
- Strict complementarity