SIAM Journal on Optimization, Volume 31, Issue 2, Page 1215-1241, January 2021. We present a forward-backward-based algorithm to minimize a sum of a differentiable function and a nonsmooth function, both being possibly nonconvex. The main contribution of this work is to consider the challenging case where the nonsmooth function corresponds to a sum of nonconvex functions, resulting from composition

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1184-1214, January 2021. This paper aims at developing two versions of the generalized Newton method to compute local minimizers for nonsmooth problems of unconstrained and constrained optimization that satisfy an important stability property known as tilt stability. We start with unconstrained minimization of continuously differentiable cost functions

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1158-1183, January 2021. The primary objective of this paper is to study a large class of differential variational-hemivariational inequalities involving history-dependent operators and constraints in a Banach space. First, we establish a well-posedness result, which includes existence, uniqueness, and continuous dependence on the initial data

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1131-1157, January 2021. We consider multilevel composite optimization problems where each mapping in the composition is the expectation over a family of randomly chosen smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient method where the approximate gradients are obtained via

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1108-1130, January 2021. Stochastic gradient descent (SGD) is one of the most widely used algorithms for large-scale optimization problems. While classical theoretical analysis of SGD for convex problems studies (suffix) averages of iterates and obtains information theoretically optimal bounds on suboptimality, the last point of SGD is, by far

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1079-1107, January 2021. Structured optimization problems are ubiquitous in fields like data science and engineering. The goal in structured optimization is using a prescribed set of points, called atoms, to build up a solution that minimizes or maximizes a given function. In the present paper, we want to minimize a black-box function over the

SIAM Journal on Optimization, Volume 31, Issue 1, Page 1049-1077, January 2021. We consider the local sensitivity of least-squares formulations of inverse problems. The sets of inputs and outputs of these problems are assumed to have the structures of Riemannian manifolds. The problems we consider include the approximation problem of finding the nearest point on a Riemannian embedded submanifold from

SIAM Journal on Optimization, Volume 31, Issue 1, Page 1017-1048, January 2021. We present adaptive sequential SAA (sample average approximation) algorithms to solve large-scale two-stage stochastic linear programs. The iterative algorithm framework we propose is organized into outer and inner iterations as follows: during each outer iteration, a sample-path problem is implicitly generated using a

SIAM Journal on Optimization, Volume 31, Issue 1, Page 991-1016, January 2021. The conditions of relative smoothness and relative strong convexity were recently introduced for the analysis of Bregman gradient methods for convex optimization. We introduce a generalized left-preconditioning method for gradient descent and show that its convergence on an essentially smooth convex objective function can

SIAM Journal on Optimization, Volume 31, Issue 1, Page 972-990, January 2021. Given a piecewise-defined function, checking whether it is convex is a nontrivial task. While it may be easy to check whether the restriction of the function to each piece is convex, ensuring the entire function is convex seems to require global conditions. However, it is known that one only needs to ensure the (convex) subdifferential

SIAM Journal on Optimization, Volume 31, Issue 1, Page 945-971, January 2021. We consider linear fractional programming problems in a form of which the linear fractional program and its stochastic and distributionally robust counterparts with finite support are special cases. We introduce a novel reformulation that involves differences of square terms in the constraint, subsequently using a piecewise

SIAM Journal on Optimization, Volume 31, Issue 1, Page 915-944, January 2021. We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient descent and Polyak's heavy ball method to a broad class of momentum methods in the setting of (possibly) constrained minimization in Euclidean and non-Euclidean normed vector spaces. Our perspective leads to a generic and unifying nonasymptotic

SIAM Journal on Optimization, Volume 31, Issue 1, Page 887-914, January 2021. Solving the trust-region subproblem (TRS) plays a key role in numerical optimization and many other applications. The generalized Lanczos trust-region (GLTR) method is a well-known Lanczos type approach for solving a large-scale TRS. The method projects the original large-scale TRS onto a sequence of lower dimensional Krylov

SIAM Journal on Optimization, Volume 31, Issue 1, Page 866-886, January 2021. We propose a necessary and sufficient test to determine whether a solution for a general quadratic program with two quadratic constraints (QC2QP) can be computed from that of a specific convex semidefinite relaxation, in which case we say that there is no optimality gap. Originally intended to solve a nonconvex optimal control

SIAM Journal on Optimization, Volume 31, Issue 1, Page 837-865, January 2021. This paper deals with particular families of DC optimization problems involving suprema of convex functions. We show that the specific structure of this type of function allows us to cover a variety of problems in nonconvex programming. Necessary and sufficient optimality conditions for these families of DC optimization problems

SIAM Journal on Optimization, Volume 31, Issue 1, Page 812-836, January 2021. In semidefinite programming a proposed optimal solution may be quite poor in spite of having sufficiently small residual in the optimality conditions. This issue may be framed in terms of the discrepancy between forward error (the unmeasurable “true error'') and backward error (the measurable violation of optimality conditions)

SIAM Journal on Optimization, Volume 31, Issue 1, Page 785-811, January 2021. In this paper, we study greedy variants of quasi-Newton methods. They are based on the updating formulas from a certain subclass of the Broyden family. In particular, this subclass includes the well-known DFP, BFGS, and SR1 updates. However, in contrast to the classical quasi-Newton methods, which use the difference of successive

SIAM Journal on Optimization, Volume 31, Issue 1, Page 754-784, January 2021. We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with strongly convex regularization and general data-fit functions. We develop an inertial approach of which we analyze both convergence and stability properties. Using tools from inexact proximal calculus

SIAM Journal on Optimization, Volume 31, Issue 1, Page 727-753, January 2021. Smooth convex minimization over the unit trace-norm ball is an important optimization problem in machine learning, signal processing, statistics, and other fields that underlies many tasks in which one wishes to recover a low-rank matrix given certain measurements. While first-order methods for convex optimization enjoy optimal

SIAM Journal on Optimization, Volume 31, Issue 1, Page 702-726, January 2021. We address the bilevel optimization problem of identifying the most critical attacks to an alternating current (AC) power flow network. The upper-level binary maximization problem consists of choosing an attack that is treated as a parameter in the lower-level defender minimization problem. Instances of the lower-level global

SIAM Journal on Optimization, Volume 31, Issue 1, Page 686-701, January 2021. We consider a linear problem over a finite set of integer vectors and assume that there is a verification oracle, which is an algorithm being able to verify whether a given vector optimizes a given linear function over the feasible set. Given an initial solution, the algorithm proposed in this paper finds an optimal solution

SIAM Journal on Optimization, Volume 31, Issue 1, Page 653-685, January 2021. We introduce Bella, a locally superlinearly convergent Bregman forward-backward splitting method for minimizing the sum of two nonconvex functions, one of which satisfies a relative smoothness condition and the other one is possibly nonsmooth. A key tool of our methodology is the Bregman forward-backward envelope (BFBE),

SIAM Journal on Optimization, Volume 31, Issue 1, Page 626-652, January 2021. The split equality problem (SEP) seeks a pair of points $(x^{\ast },y^{\ast })\in (C,D)$ with the property that $Ax^{\ast }=By^{\ast }$, where $C,D$ are nonempty closed convex subsets of Hilbert spaces $\mathcal{H}_{1}$ and $% \mathcal{H}_{2}$, respectively, and $A:\mathcal{H}_{1}\rightarrow \mathcal{H}% _{3}$ and $B:\ma

SIAM Journal on Optimization, Volume 31, Issue 1, Page 604-625, January 2021. We study a class of countably infinite linear programs (CILPs) whose feasible sets are bounded subsets of appropriately defined spaces of measures. The optimal value, optimal points, and minimal points of these CILPs can be approximated by solving finite-dimensional linear programs. We show how to construct finite-dimensional

SIAM Journal on Optimization, Volume 31, Issue 1, Page 569-603, January 2021. This paper is devoted to developing and applications of a generalized differential theory of variational analysis that allows us to work in incomplete normed spaces, without employing conventional variational techniques based on completeness and limiting procedures. The main attention is paid to generalized derivatives and

SIAM Journal on Optimization, Volume 31, Issue 1, Page 545-568, January 2021. This paper mainly studies the relationship between quadratic growth and strong metric subregularity of the subdifferential in finite dimensional settings by using the subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous

SIAM Journal on Optimization, Volume 31, Issue 1, Page 518-544, January 2021. Worst-case complexity guarantees for nonconvex optimization algorithms have been a topic of growing interest. Multiple frameworks that achieve the best known complexity bounds among a broad class of first- and second-order strategies have been proposed. These methods have often been designed primarily with complexity guarantees

SIAM Journal on Optimization, Volume 31, Issue 1, Page 489-517, January 2021. In the paper, a Newton-type method for the solution of generalized equations (GEs) is derived, where the linearization concerns both the single-valued and the multivalued part of the considered GE. The method is based on the new notion of semismoothness${}^*$, which, together with a suitable regularity condition, ensures

SIAM Journal on Optimization, Volume 31, Issue 1, Page 461-488, January 2021. We consider a distributionally robust partially observable Markov decision process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations can be inferred using side information at the end of each period after an action being

SIAM Journal on Optimization, Volume 31, Issue 1, Page 448-460, January 2021. Several works have shown linear speedup is achieved by an asynchronous parallel implementation of stochastic coordinate descent so long as there is not too much parallelism. More specifically, it is known that if all updates are of similar duration, then linear speedup is possible with up to $\Theta(L_{\max}\sqrt n/L_{\overline{{res}}})$

SIAM Journal on Optimization, Volume 31, Issue 1, Page 425-447, January 2021. We consider a generalization of the Bauer maximum principle. We work with tensorial products of convex measures set that are not necessarily compact but generated by their extreme points. We show that the maximum of a quasi-convex lower semicontinuous function on this product space is reached on the tensorial product of finite

SIAM Journal on Optimization, Volume 31, Issue 1, Page 404-424, January 2021. Recently, a new kind of distance has been introduced for the graphs of two point-to-set operators, one of which is maximally monotone. When both operators are the subdifferential of a proper lower semicontinuous convex function, this kind of distance specializes under modest assumptions to the classical Bregman distance.

SIAM Journal on Optimization, Volume 31, Issue 1, Page 377-403, January 2021. We consider the generalized Nash equilibrium problem (GNEP) with $ N $ players in a Hilbert space setting. The joint constraints are eliminated by an augmented Lagrangian-type approach, leading to an ADMM (alternating direction method of multipliers) algorithm. In contrast to standard optimization problems, however, the direct

SIAM Journal on Optimization, Volume 31, Issue 1, Page 348-376, January 2021. In this work, we present a novel approach for solving stochastic shape optimization problems. Our method is the extension of the classical stochastic gradient method to infinite-dimensional shape manifolds. We prove convergence of the method on Riemannian manifolds and then make the connection to shape spaces. The method

SIAM Journal on Optimization, Volume 31, Issue 1, Page 331-347, January 2021. We provide a polynomial time cutting plane algorithm based on split cuts to solve integer programs in the plane. We also prove that the split closure of a polyhedron in the plane has polynomial size.

SIAM Journal on Optimization, Volume 31, Issue 1, Page 307-330, January 2021. We propose a new family of multilevel methods for unconstrained minimization. The resulting strategies are multilevel extensions of high-order optimization methods based on $q$th-order Taylor models (with $q\geq 1$) that have been recently proposed in the literature. The use of high-order models, while decreasing the worst-case

SIAM Journal on Optimization, Volume 31, Issue 1, Page 275-306, January 2021. In this study, we consider a rich class of mathematical programs with equilibrium constraints (MPECs) involving both integer and continuous variables. Such a class, which subsumes mathematical programs with complementarity constraints, as well as bilevel programs involving lower level convex programs is, in general, extremely

SIAM Journal on Optimization, Volume 31, Issue 1, Page 244-274, January 2021. Adam is a popular variant of stochastic gradient descent for finding a local minimizer of a function. In the constant stepsize regime, assuming that the objective function is differentiable and nonconvex, we establish the convergence in the long run of the iterates to a stationary point under a stability condition. The key

SIAM Journal on Optimization, Volume 31, Issue 1, Page 217-243, January 2021. This paper presents an accelerated composite gradient (ACG) variant, referred to as the AC-ACG method, for solving nonconvex smooth composite minimization problems. As opposed to well-known ACG variants that are based on either a known Lipschitz gradient constant or a sequence of maximum observed curvatures, the current one

SIAM Journal on Optimization, Volume 31, Issue 1, Page 200-216, January 2021. We obtain a transference bound for vertices of corner polyhedra that connects two well-established areas of research: proximity and sparsity of solutions to integer programs. In the knapsack scenario, it implies that for any vertex ${x}^*$ of an integer feasible knapsack polytope ${P}({a}, { b})=\{{x} \in {\mathbb R}^n_{\ge

SIAM Journal on Optimization, Volume 31, Issue 1, Page 172-199, January 2021. We study the convergence issue for the gradient algorithm (employing general step sizes) for optimization problems on general Riemannian manifolds (without curvature constraints). Under the assumption of the local convexity/quasi-convexity (resp., weak sharp minima), local/global convergence (resp., linear convergence) results

SIAM Journal on Optimization, Volume 31, Issue 1, Page 142-171, January 2021. We consider the global optimization of nonconvex (mixed-integer) quadratic programs. We present a family of convex quadratic relaxations derived by convexifying nonconvex quadratic functions through perturbations of the quadratic matrix. We investigate the theoretical properties of these relaxations and show that they are

SIAM Journal on Optimization, Volume 31, Issue 1, Page 114-141, January 2021. This work is a follow-up and a complement to [J. Wang, V. Magron and J. B. Lasserre, preprint, arXiv:1912.08899, 2019] where the TSSOS hierarchy was proposed for solving polynomial optimization problems (POPs). The chordal-TSSOS hierarchy that we propose is a new sparse moment-SOS framework based on term sparsity and chordal

SIAM Journal on Optimization, Volume 31, Issue 1, Page 91-113, January 2021. In this article, the concept of the single-objective sequential quadratically constrained quadratic programming method is extended to the multiobjective case and a new line search technique is developed for nonlinear multiobjective optimization problems. The proposed method ensures global convergence as well as spreading of

SIAM Journal on Optimization, Volume 31, Issue 1, Page 59-90, January 2021. Recently, there has been a surge of interest in designing variants of the classical Newton-CG in which the Hessian of a (strongly) convex function is replaced by suitable approximations. This is mainly motivated by large-scale finite-sum minimization problems that arise in many machine learning applications. Going beyond convexity

SIAM Journal on Optimization, Volume 31, Issue 1, Page 30-58, January 2021. This paper is concerned with polynomial optimization problems. We show how to exploit term (or monomial) sparsity of the input polynomials to obtain a new converging hierarchy of semidefinite programming relaxations. The novelty (and distinguishing feature) of such relaxations is to involve block-diagonal matrices obtained

SIAM Journal on Optimization, Volume 31, Issue 1, Page 1-29, January 2021. The prevalence of uncertainty in models of engineering and the natural sciences necessitates the inclusion of random parameters in the underlying partial differential equations (PDEs). The resulting decision problems governed by the solution of such random PDEs are infinite dimensional stochastic optimization problems. In order

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3387-3414, January 2020. This paper is devoted to the study of generalized subdifferentials of spectral functions over Euclidean Jordan algebras. Spectral functions appear often in optimization problems playing the role of “regularizer,” ``barrier,” ``penalty function,” and many others. We provide formulae for the regular, approximate, and horizon

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3359-3386, January 2020. Sparse regression and variable selection for large-scale data have been rapidly developed in the past decades. This work focuses on sparse ridge regression, which enforces the sparsity by use of the $L_{0}$ norm. We first prove that the continuous relaxation of the mixed integer second order conic (MISOC) reformulation

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3345-3358, January 2020. Cubic regularization methods have several favorable properties. In particular under mild assumptions, they are globally convergent towards critical points with second-order necessary conditions satisfied. Their adoption among practitioners, however, does not yet match the strong theoretical results. One of the reasons for

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3315-3344, January 2020. In this paper, we consider the general nonoblivious stochastic optimization where the underlying stochasticity may change during the optimization procedure and depends on the point at which the function is evaluated. We develop Stochastic Frank--Wolfe++ (SFW++), an efficient variant of the conditional gradient method for

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3284-3314, January 2020. Fair resource allocation is a fundamental optimization problem with applications in operations research, networking, and economic and game theory. Research in these areas has led to the general acceptance of a class of $\alpha$-fair utility functions parameterized by $\alpha \in [0, \infty]$. We consider $\alpha$-fair packing---the

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3252-3283, January 2020. The introduction of the Hessian damping in the continuous version of Nesterov's accelerated gradient method provides, by temporal discretization, fast proximal gradient algorithms where the oscillations are significantly attenuated. We will extend these results to the maximally monotone case. We rely on the technique introduced

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3230-3251, January 2020. We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extragradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method.

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3198-3229, January 2020. A spectral risk measure (SRM) is a weighted average of value at risk where the weighting function (also known as risk spectrum or distortion function) characterizes a decision maker's risk attitude. In this paper, we consider the case where the decision maker's risk spectrum is ambiguous and introduce a robust SRM model

SIAM Journal on Optimization, Volume 30, Issue 4, Page 3170-3197, January 2020. We consider the application of the type-I Anderson acceleration to solving general nonsmooth fixed-point problems. By interleaving with safeguarding steps and employing a Powell-type regularization and a restart checking for strong linear independence of the updates, we propose the first globally convergent variant of Anderson

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2659-2686, January 2020. A multiobjective optimization problem is simplicial if the Pareto set and front are homeomorphic to a simplex and, under the homeomorphisms, each face of the simplex corresponds to the Pareto set and front of a subproblem that treats a subset of objective functions. In this paper, we show that strongly convex problems are

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2628-2658, January 2020. We introduce a new feasible corrector-predictor (CP) interior-point algorithm (IPA), which is suitable for solving linear complementarity problem (LCP) with $P_{*} (\kappa)$-matrices. We use the method of algebraically equivalent transformation (AET) of the nonlinear equation of the system which defines the central path

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2603-2627, January 2020. The paper is devoted to an analysis of a new constraint qualification and a derivation of the strongest existing optimality conditions for nonsmooth mathematical programming problems with equality and inequality constraints in terms of Demyanov--Rubinov--Polyakova quasidifferentials under the minimal possible assumptions

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2577-2602, January 2020. When solving large-scale semidefinite programs that admit a low-rank solution, an efficient heuristic is the Burer--Monteiro factorization: instead of optimizing over the full matrix, one optimizes over its low-rank factors. This reduces the number of variables to optimize but destroys the convexity of the problem, thus