SIAM Journal on Optimization, Volume 30, Issue 3, Page 2659-2686, January 2020. A multiobjective optimization problem is simplicial if the Pareto set and front are homeomorphic to a simplex and, under the homeomorphisms, each face of the simplex corresponds to the Pareto set and front of a subproblem that treats a subset of objective functions. In this paper, we show that strongly convex problems are

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2628-2658, January 2020. We introduce a new feasible corrector-predictor (CP) interior-point algorithm (IPA), which is suitable for solving linear complementarity problem (LCP) with $P_{*} (\kappa)$-matrices. We use the method of algebraically equivalent transformation (AET) of the nonlinear equation of the system which defines the central path

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2603-2627, January 2020. The paper is devoted to an analysis of a new constraint qualification and a derivation of the strongest existing optimality conditions for nonsmooth mathematical programming problems with equality and inequality constraints in terms of Demyanov--Rubinov--Polyakova quasidifferentials under the minimal possible assumptions

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2577-2602, January 2020. When solving large-scale semidefinite programs that admit a low-rank solution, an efficient heuristic is the Burer--Monteiro factorization: instead of optimizing over the full matrix, one optimizes over its low-rank factors. This reduces the number of variables to optimize but destroys the convexity of the problem, thus

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2559-2576, January 2020. The Douglas--Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2530-2558, January 2020. This paper studies the class of two-stage stochastic programs with a linearly bi-parameterized recourse function defined by a convex quadratic program. A distinguishing feature of this new class of nonconvex stochastic programs is that the objective function in the second stage is linearly parameterized by the first-stage

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2470-2500, January 2020. In this paper, we study active set identification results for the away-step Frank--Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, for nonconvex objectives, a novel

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2501-2529, January 2020. Motivated by the TRACE algorithm [F. E. Curtis, D. P. Robinson, and M. Samadi, Math. Program., 162 (2017), pp. 1--32], we propose a trust region algorithm for finding second-order stationary points of a linearly constrained nonconvex optimization problem. We show the convergence of the proposed algorithm to ($\epsilon_g

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2441-2469, January 2020. In this work, we present a method to compute the Kantorovich--Wasserstein distance of order 1 between a pair of two-dimensional histograms. Recent works in computer vision and machine learning have shown the benefits of measuring Wasserstein distances of order 1 between histograms with $n$ bins by solving a classical transportation

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2410-2440, January 2020. Powerful interior-point methods (IPM) based commercial solvers, such as Gurobi and Mosek, have been hugely successful in solving large-scale linear programming (LP) problems. The high efficiency of these solvers depends critically on the sparsity of the problem data and advanced matrix factorization techniques. For a large

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2379-2409, January 2020. The paper is devoted to the study of the twice epi-differentiablity of extended-real-valued functions, with an emphasis on functions satisfying a certain composite representation. This will be conducted under parabolic regularity, a second-order regularity condition that was recently utilized in [A. Mohammadi, B. Mordukhovich

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2355-2378, January 2020. We study a type of network flow problem that we call the minimum-cost F-graph flow problem. This problem generalizes the typical minimum-cost network flow problem by allowing the underlying network to be a directed hypergraph rather than a directed graph. This new problem is pertinent because it can be used to model network

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2337-2354, January 2020. Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow. Using geometric singular perturbation theory, we show that, under certain conditions

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2303-2336, January 2020. We consider a resource allocation problem involving a large number of agents with individual constraints subject to privacy, and a central operator whose objective is to optimize a global, possibly nonconvex, cost while satisfying the agents' constraints, for instance, an energy operator in charge of the management of energy

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2272-2302, January 2020. Consider the minimization of a nonconvex differentiable function over a bounded polyhedron. A popular primal-dual first-order method for this problem is to perform a gradient projection iteration for the augmented Lagrangian function and then update the dual multiplier vector using the constraint residual. However, numerical

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2251-2271, January 2020. We propose a methodology for studying the performance of common splitting methods through semidefinite programming. We prove tightness of the methodology and demonstrate its value by presenting two applications of it. First, we use the methodology as a tool for computer-assisted proofs to prove tight analytical contraction

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2221-2250, January 2020. We introduce a new method for solving nonlinear continuous optimization problems with chance constraints. Our method is based on a reformulation of the probabilistic constraint as a quantile function. The quantile function is approximated via a differentiable sample average approximation. We provide theoretical statistical

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2197-2220, January 2020. Undirected graphical models have been especially popular for learning the conditional independence structure among a large number of variables where the observations are drawn independently and identically from the same distribution. However, many modern statistical problems would involve categorical data or time-varying

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2163-2196, January 2020. We propose a general trust-region method for the minimization of nonsmooth and nonconvex, locally Lipschitz continuous functions that can be applied, e.g., to optimization problems constrained by elliptic variational inequalities. The convergence of the considered algorithm to C-stationary points is verified in an abstract

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2134-2162, January 2020. In a Hilbert space ${\mathcal H}$, we introduce a new class of proximal-gradient algorithms with finite convergence properties. These algorithms naturally occur as discrete temporal versions of an inertial differential inclusion which is stabilized under the joint action of three dampings: dry friction, viscous friction

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2103-2133, January 2020. In this paper, we study a class of stochastic optimization problems, referred to as the conditional stochastic optimization (CSO), in the form of $\min_{x \in \mathcal{X}} \mathbb{E}_{\xi}f_\xi({\mathbb{E}_{\eta|\xi}[g_\eta(x, \xi)]})$, which finds a wide spectrum of applications including portfolio selection, reinforcement

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2083-2102, January 2020. In some applications the considered multistage stochastic programs have a periodical behavior. We show that in such cases it is possible to drastically reduce the number of stages by introducing a periodical analogue of the so-called Bellman equations for discounted infinite horizon problems used in Markov decision processes

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2053-2082, January 2020. We provide new tools for worst-case performance analysis of the gradient (or steepest descent) method of Cauchy for smooth strongly convex functions, and Newton's method for self-concordant functions, including the case of inexact search directions. The analysis uses semidefinite programming performance estimation, as pioneered

SIAM Journal on Optimization, Volume 30, Issue 3, Page 2026-2052, January 2020. When there are infinitely many scenarios, the current studies of two-stage stochastic programming problems rely on the relatively complete recourse assumption. However, such an assumption can be unrealistic for many real-world problems. This motivates us to study general stochastic programming problems where the sample

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1996-2025, January 2020. We consider distributionally robust optimization problems (DROPs) with nonlinear and nonconcave dependence on uncertain parameters. The DROP can be written as a nonsmooth, nonlinear program with a bilevel structure; the objective function and each of the constraint functions are suprema of expected values of parametric

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1980-1995, January 2020. The trust region subproblem has at most one local nonglobal minimizer. In characterizing this local solution, there is a clear gap between necessary and sufficient conditions. In this paper, we surprisingly show that the sufficient second-order optimality condition remains necessary. As an application, we improve the state-of-the-art

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1954-1979, January 2020. We introduce a class of positive definite preconditioners for the solution of large symmetric indefinite linear systems or sequences of such systems, in optimization frameworks. The preconditioners are iteratively constructed by collecting information on a reduced eigenspace of the indefinite matrix by means of a Krylov-subspace

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1756-1775, January 2020. For the rank regularized minimization problem, we introduce several classes of stationary points by the problem itself and its equivalent reformulations including the mathematical program with an equilibrium constraint (MPEC), the global exact penalty of the MPEC, and the difference-of-convex surrogate yielded by eliminating

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1724-1755, January 2020. This paper studies generalized differentiability properties of the marginal function of parametric optimal control problems governed by semilinear elliptic partial differential equations. We establish some upper estimates for the regular and the limiting subgradients of the marginal function for Hilbert parametric spaces

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1693-1723, January 2020. We present in this paper a novel deterministic algorithmic framework that enables the computation of a directional stationary solution of the empirical deep neural network training problem formulated as a multicomposite optimization problem with coupled nonconvexity and nondifferentiability. This is the first time to our

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1664-1692, January 2020. The stochastic gradient method (SGM) has been popularly applied to solve optimization problems with an objective that is stochastic or an average of many functions. Most existing works on SGMs assume that the underlying problem is unconstrained or has an easy-to-project constraint set. In this paper, we consider problems

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1638-1663, January 2020. Regarding the approximation of Nash equilibria in games where the players have a continuum of strategies, there exist various algorithms based on best response dynamics and on its relaxed variants: from one step to the next, a player's strategy is updated by using explicitly a best response to the strategies of the other

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1610-1637, January 2020. We introduce a sequential optimality condition for locally Lipschitz constrained nonsmooth optimization, verifiable just using derivative information, and which holds even in the absence of any constraint qualification. We present a practical algorithm that generates iterates either fulfilling the new necessary optimality

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1582-1609, January 2020. We investigate new convex relaxations for the pooling problem, a classic nonconvex production planning problem in which input materials are mixed in intermediate pools, with the outputs of these pools further mixed to make output products meeting given attribute percentage requirements. Our relaxations are derived by considering

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1555-1581, January 2020. It is a well-known phenomenon that the presence of critical Lagrange multipliers in constrained optimization problems may cause a deterioration of the convergence speed of primal-dual Newton-type methods. Regardless of the method under consideration, we develop a new local technique for avoiding convergence to critical

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1527-1554, January 2020. In this paper, we study the sensitivity of discrete-time dynamic programs with nonlinear dynamics and objective to perturbations in the initial conditions and reference parameters. Under uniform controllability and boundedness assumptions for the problem data, we prove that the directional derivative of the optimal state

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1501-1526, January 2020. In this paper, we employ advanced techniques of variational analysis and generalized differentiation to examine robust optimality conditions and robust duality for an uncertain nonsmooth multiobjective optimization problem under arbitrary uncertainty nonempty sets. We establish necessary and sufficient optimality conditions

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1473-1500, January 2020. Stochastic gradient-based optimization has been a core enabling methodology in applications to large-scale problems in machine learning and related areas. Despite this progress, the gap between theory and practice remains significant, with theoreticians pursuing mathematical optimality at the cost of obtaining specialized

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1451-1472, January 2020. In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1421-1450, January 2020. In this work, we analyze the regularizing property of the stochastic gradient descent for the numerical solution of a class of nonlinear ill-posed inverse problems in Hilbert spaces. At each step of the iteration, the method randomly chooses one equation from the nonlinear system to obtain an unbiased stochastic estimate

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1391-1420, January 2020. We study inertial versions of primal-dual proximal splitting, also known as the Chambolle--Pock method. Our starting point is the preconditioned proximal point formulation of this method. By adding correctors corresponding to the antisymmetric part of the relevant monotone operator, using a FISTA-style gap unrolling argument

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1922-1953, January 2020. In general, standard necessary optimality conditions cannot be formulated in a straightforward manner for semismooth shape optimization problems. In this paper, we consider shape optimization problems constrained by variational inequalities of the first kind, so-called obstacle-type problems. Under appropriate assumptions

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1905-1921, January 2020. The tight sublinear convergence rate of the proximal point algorithm for maximal monotone inclusion problems is established based on the squared fixed point residual. By using the performance estimation framework, the tight sublinear convergence rate problem is written as an infinite dimensional nonconvex optimization problem

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1878-1904, January 2020. We analyze the coordinate descent method with a new coordinate selection strategy, called volume sampling. This strategy prescribes selecting subsets of variables of certain size proportionally to the determinants of principal submatrices of the matrix, which bounds the curvature of the objective function. In the particular

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1850-1877, January 2020. In this article a family of second-order ODEs associated with the inertial gradient descent is studied. These ODEs are widely used to build trajectories converging to a minimizer $x^*$ of a function $F$, possibly convex. This family includes the continuous version of the Nesterov inertial scheme and the continuous heavy

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1822-1849, January 2020. This paper focuses on the design of sequential quadratic optimization (commonly known as SQP) methods for solving large-scale nonlinear optimization problems. The most computationally demanding aspect of such an approach is the computation of the search direction during each iteration, for which we consider the use of matrix-free

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1795-1821, January 2020. EXTRA is a popular method for dencentralized distributed optimization and has broad applications. This paper revisits EXTRA. First, we give a sharp complexity analysis for EXTRA with the improved $O\big(\big(\frac{L}{\mu}+\frac{1}{1-\sigma_2({W})}\big)\log\frac{1}{\epsilon(1-\sigma_2({W}))}\big)$ communication and computation

SIAM Journal on Optimization, Volume 30, Issue 3, Page 1777-1794, January 2020. Consider a semialgebraic function $f\colon\mathbb{R}^n \to {\mathbb{R}},$ which is continuous around a point $\bar{x} \in \mathbb{R}^n.$ Using the so-called tangency variety of $f$ at $\bar{x},$ we first provide necessary and sufficient conditions for $\bar{x}$ to be a local minimizer of $f,$ and then in the case where

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1366-1390, January 2020. We construct a proximal average for two prox-bounded functions, which recovers the classical proximal average for two convex functions. The new proximal average transforms continuously in epi-topology from one proximal hull to the other. When one of the functions is differentiable, the new proximal average is differentiable

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1339-1365, January 2020. One of the most fundamental ingredients in mixed-integer nonlinear programming solvers is the well-known McCormick relaxation for a product of two variables $x$ and $y$ over a box-constrained domain. The starting point of this paper is the fact that the convex hull of the graph of $xy$ can be much tighter when computed

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1327-1338, January 2020. We construct examples of Lipschitz continuous functions, with pathological subgradient dynamics in both continuous and discrete time. In both settings, the iterates generate bounded trajectories and yet fail to detect any (generalized) critical points of the function.

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1300-1326, January 2020. In 1988, Barzilai and Borwein published a pioneering paper which opened the way to inexpensively accelerate first-order. In more detail, in the framework of unconstrained optimization, Barzilai and Borwein developed two strategies to select the step length in gradient descent methods with the aim of encoding some second-order

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1274-1299, January 2020. We propose a subgradient-based method for finding the maximum feasible subsystem in a collection of closed sets with respect to a given closed set $C$ (MFS$_C$). In this method, we reformulate the MFS$_C$ problem as an $\ell_0$ optimization problem and construct a sequence of continuous optimization problems to approximate

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1251-1273, January 2020. We present a unified geometrical analysis on the completely positive programming (CPP) reformulations of quadratic optimization problems (QOPs) and their extension to polynomial optimization problems (POPs) based on a class of geometrically defined nonconvex conic programs and their convexification. The class of nonconvex

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1223-1250, January 2020. The stochastic dual dynamic programming (SDDP) algorithm has become one of the main tools used to address convex multistage stochastic optimal control problems. Recently a large amount of work has been devoted to improving the convergence speed of the algorithm through cut selection and regularization, and to extending

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1191-1222, January 2020. In this paper, we study network linear equations subject to digital communications with a finite data rate, where each node is associated with one equation from a system of linear equations. Each node holds a dynamic state and interacts with its neighbors through an undirected connected graph, where along each link the

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1173-1190, January 2020. In subset selection we search for the best linear predictor that involves a small subset of variables. From a computational complexity viewpoint, subset selection is NP-hard and few classes are known to be solvable in polynomial time. Using mainly tools from discrete geometry, we show that some sparsity conditions on the

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1144-1172, January 2020. Motivated by applications arising from large-scale optimization and machine learning, we consider stochastic quasi-Newton (SQN) methods for solving unconstrained convex optimization problems. Much of the convergence analysis of SQN methods, in both full and limited-memory regimes, requires the objective function to be strongly

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1119-1143, January 2020. We treat the so-called scenario approach, a popular probabilistic approximation method for robust minmax optimization problems via independent and identically distributed (i.i.d.) sampling from the uncertainty set, from various perspectives. The scenario approach is well studied in the important case of convex robust optimization

SIAM Journal on Optimization, Volume 30, Issue 2, Page 1094-1118, January 2020. Based on concepts like the $k$th convex hull and finer characterization of nonconvexity of a function, we propose a refinement of the Shapley--Folkman lemma and derive a new estimate for the duality gap of nonconvex optimization problems with separable objective functions. We apply our result to the network utility maximization