SIAM Journal on Optimization, Volume 31, Issue 3, Page 1926-1946, January 2021. Using the Attouch--Théra duality, we study the cycles, gap vectors, and fixed point sets of compositions of proximal mappings. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1897-1925, January 2021. We consider the problem of estimating the true values of a Wiener process given noisy observations corrupted by outliers. In this paper we show how to improve existing mixed-integer quadratic optimization formulations for this problem. Specifically, we convexify the existing formulations via lifting, deriving new mixed-integer

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1877-1896, January 2021. We consider a nonconvex optimization problem consisting of maximizing the difference of two convex functions. We present a randomized method that requires low computational effort at each iteration. The described method is a randomized coordinate descent method employed on the so-called Toland-dual problem. We prove subsequence

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1850-1876, January 2021. An algorithm is proposed to solve robust control problems constrained by partial differential equations with uncertain coefficients, based on the so-called MG/OPT framework. The levels in the MG/OPT hierarchy correspond to discretization levels of the PDE, as usual. For stochastic problems, the relevant quantities (such

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1827-1849, January 2021. Symmetries of a set are linear transformations that map the set to itself. A fundamental domain of a symmetric set is a subset that contains at least one representative from each of the symmetric equivalence classes (orbits) in the set. This paper contributes a novel polynomial algorithm for constructing minimal polytopic

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1797-1826, January 2021. In this paper, through an intuitive vanilla proximal method perspective, we derive a concise unified acceleration framework (UAF) for minimizing a convex function that has Hölder continuous derivatives with respect to general (non-Euclidean) norms. The UAF reconciles two different high-order acceleration approaches, one

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1774-1796, January 2021. Various characterizations of convexity for images of a vector mapping where some of its components are quadratic and the remaining ones are linear are established. In a certain sense, one might conclude that convexity of the full image is reduced to the convexity of an image in a lower dimension by deleting the linear components

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1748-1773, January 2021. In this paper, we adopt the augmented Lagrangian method (ALM) to solve convex quadratic second-order cone programming problems (SOCPs). Fruitful results on the efficiency of the ALM have been established in the literature. Recently, it has been shown in [Cui, Sun, and Toh, Math. Program., 178 (2019), pp. 381--415] that

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1722-1747, January 2021. Fundamental in matrix algebra and its applications, a generalized inverse of a real matrix $A$ is a matrix $H$ that satisfies the Moore--Penrose (M--P) property $AHA=A$. If $H$ also satisfies the additional useful M--P property, $HAH=H$, it is called a reflexive generalized inverse. Reflexivity is equivalent to minimum

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1690-1721, January 2021. Recently, there has been a growing interest in distributionally robust optimization (DRO) as a principled approach to data-driven decision making. In this paper, we consider a distributionally robust two-stage stochastic optimization problem with discrete scenario support. While much research effort has been devoted to

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1658-1689, January 2021. Shape optimization based on shape calculus has received a lot of attention in recent years, particularly regarding the development, analysis, and modification of efficient optimization algorithms. In this paper we propose and investigate nonlinear conjugate gradient methods based on Steklov--Poincaré-type metrics for the

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1635-1657, January 2021. In this paper, we investigate the Moreau envelope of the supremum of a family of convex, proper, and lower semicontinuous functions. Under mild assumptions, we prove that the Moreau envelope of a supremum is the supremum of Moreau envelopes, which allows us to approximate possibly nonsmooth supremum functions by smooth

SIAM Journal on Optimization, Volume 31, Issue 3, Page 1605-1634, January 2021. We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are ubiquitous in engineering applications but still largely unexplored. We present a family of Riemannian subgradient-type methods---namely Riemannian

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1576-1603, January 2021. Recently, in a series of papers [Y. Rahimi, C. Wang, H. Dong, and Y. Lou, SIAM J. Sci. Comput., 41 (2019), pp. A3649--A3672; C. Wang, M. Tao, J. Nagy, and Y. Lou, SIAM J. Imaging Sci., 14 (2021), pp. 749--777; C. Wang, M. Yan, and Y. Lou, IEEE Trans. Signal Process., 68 (2020), pp. 2660--2669; P. Yin, E. Esser, and J. Xin

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1546-1575, January 2021. The symplectic Stiefel manifold, denoted by ${Sp}(2p,2n)$, is the set of linear symplectic maps between the standard symplectic spaces $\mathbb{R}^{2p}$ and $\mathbb{R}^{2n}$. When $p=n$, it reduces to the well-known set of $2n\times 2n$ symplectic matrices. Optimization problems on ${Sp}(2p,2n)$ find applications in various

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1519-1545, January 2021. Multiple-input multiple-output (MIMO) detection is a fundamental problem in wireless communications and it is strongly NP-hard in general. Massive MIMO has been recognized as a key technology in fifth generation (5G) and beyond communication networks, which on one hand can significantly improve the communication performance

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1489-1518, January 2021. In this paper, we develop convergence analysis of a modified line search method for objective functions whose value is computed with noise and whose gradient estimates are inexact and possibly random. The noise is assumed to be bounded in absolute value without any additional assumptions. We extend the framework based on

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1459-1488, January 2021. We show that the Mordukhovich-stationarity system associated with a mathematical program with complementarity constraints (MPCC) can be equivalently written as a system of discontinuous equations which can be tackled with a semismooth Newton method. It will be demonstrated that the resulting algorithm can be interpreted

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1433-1458, January 2021. In this paper we give an algorithm to round the floating point output of a semidefinite programming solver to a solution over the rationals or a quadratic extension of the rationals. This algorithm does not require the solution to be strictly feasible and works for large problems. We apply this to get sharp bounds for packing

SIAM Journal on Optimization, Volume 31, Issue 1, Page 1049-1077, January 2021. We consider the local sensitivity of least-squares formulations of inverse problems. The sets of inputs and outputs of these problems are assumed to have the structures of Riemannian manifolds. The problems we consider include the approximation problem of finding the nearest point on a Riemannian embedded submanifold from

SIAM Journal on Optimization, Volume 31, Issue 1, Page 1017-1048, January 2021. We present adaptive sequential SAA (sample average approximation) algorithms to solve large-scale two-stage stochastic linear programs. The iterative algorithm framework we propose is organized into outer and inner iterations as follows: during each outer iteration, a sample-path problem is implicitly generated using a

SIAM Journal on Optimization, Volume 31, Issue 1, Page 991-1016, January 2021. The conditions of relative smoothness and relative strong convexity were recently introduced for the analysis of Bregman gradient methods for convex optimization. We introduce a generalized left-preconditioning method for gradient descent and show that its convergence on an essentially smooth convex objective function can

SIAM Journal on Optimization, Volume 31, Issue 1, Page 972-990, January 2021. Given a piecewise-defined function, checking whether it is convex is a nontrivial task. While it may be easy to check whether the restriction of the function to each piece is convex, ensuring the entire function is convex seems to require global conditions. However, it is known that one only needs to ensure the (convex) subdifferential

SIAM Journal on Optimization, Volume 31, Issue 1, Page 945-971, January 2021. We consider linear fractional programming problems in a form of which the linear fractional program and its stochastic and distributionally robust counterparts with finite support are special cases. We introduce a novel reformulation that involves differences of square terms in the constraint, subsequently using a piecewise

SIAM Journal on Optimization, Volume 31, Issue 1, Page 915-944, January 2021. We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient descent and Polyak's heavy ball method to a broad class of momentum methods in the setting of (possibly) constrained minimization in Euclidean and non-Euclidean normed vector spaces. Our perspective leads to a generic and unifying nonasymptotic

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1410-1432, January 2021. For an inequality system defined by an infinite family of proper lower semicontinuous convex functions in normed linear space, we consider the Farkas--Minkowski (FM for short) type qualification and the basic constraint qualification (BCQ for short). By employing a new approach based on some new results established here

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1380-1409, January 2021. Metric regularity and a stability theorem with a square-root distance estimate are investigated for smooth mappings which act in Hilbert spaces. The main result concerns a sufficient condition for this type of metric regularity and stability, which is formulated without a priori normality assumptions. As an application

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1352-1379, January 2021. Sequential quadratic optimization algorithms are proposed for solving smooth nonlinear optimization problems with equality constraints. The main focus is an algorithm proposed for the case when the constraint functions are deterministic, and constraint function and derivative values can be computed explicitly, but the objective

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1330-1351, January 2021. We obtain new properties and formulas for the horizon map and graphical convergence for set-valued maps. We calculate horizon maps and establish the graphical convergence of various important maps from variational analysis. We also establish the continuity of several operations with set-valued maps. We compare these notions

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1299-1329, January 2021. In this paper, we propose a primal-dual algorithm with a novel momentum term using the partial gradients of the coupling function that can be viewed as a generalization of the method proposed by Chambolle and Pock in [Math. Program., 159 (2016), pp. 253--287] for solving saddle point problems defined by a convex-concave

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1276-1298, January 2021. Convex cones play an important role in nonlinear analysis and optimization theory. In particular, specific normal cones and tangent cones are known to be convex cones, and it is a crucial fact that they are useful geometric objects for describing optimality conditions. As important applications (especially, in the fields

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1242-1275, January 2021. This paper develops a novel limited-memory method to solve dynamic optimization problems. The memory requirements for such problems often present a major obstacle, particularly for problems with PDE constraints such as optimal flow control, full waveform inversion, and optical tomography. In these problems, PDE constraints

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1215-1241, January 2021. We present a forward-backward-based algorithm to minimize a sum of a differentiable function and a nonsmooth function, both being possibly nonconvex. The main contribution of this work is to consider the challenging case where the nonsmooth function corresponds to a sum of nonconvex functions, resulting from composition

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1184-1214, January 2021. This paper aims at developing two versions of the generalized Newton method to compute local minimizers for nonsmooth problems of unconstrained and constrained optimization that satisfy an important stability property known as tilt stability. We start with unconstrained minimization of continuously differentiable cost functions

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1158-1183, January 2021. The primary objective of this paper is to study a large class of differential variational-hemivariational inequalities involving history-dependent operators and constraints in a Banach space. First, we establish a well-posedness result, which includes existence, uniqueness, and continuous dependence on the initial data

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1131-1157, January 2021. We consider multilevel composite optimization problems where each mapping in the composition is the expectation over a family of randomly chosen smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient method where the approximate gradients are obtained via

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1108-1130, January 2021. Stochastic gradient descent (SGD) is one of the most widely used algorithms for large-scale optimization problems. While classical theoretical analysis of SGD for convex problems studies (suffix) averages of iterates and obtains information theoretically optimal bounds on suboptimality, the last point of SGD is, by far

SIAM Journal on Optimization, Volume 31, Issue 2, Page 1079-1107, January 2021. Structured optimization problems are ubiquitous in fields like data science and engineering. The goal in structured optimization is using a prescribed set of points, called atoms, to build up a solution that minimizes or maximizes a given function. In the present paper, we want to minimize a black-box function over the

SIAM Journal on Optimization, Volume 31, Issue 1, Page 887-914, January 2021. Solving the trust-region subproblem (TRS) plays a key role in numerical optimization and many other applications. The generalized Lanczos trust-region (GLTR) method is a well-known Lanczos type approach for solving a large-scale TRS. The method projects the original large-scale TRS onto a sequence of lower dimensional Krylov

SIAM Journal on Optimization, Volume 31, Issue 1, Page 866-886, January 2021. We propose a necessary and sufficient test to determine whether a solution for a general quadratic program with two quadratic constraints (QC2QP) can be computed from that of a specific convex semidefinite relaxation, in which case we say that there is no optimality gap. Originally intended to solve a nonconvex optimal control

SIAM Journal on Optimization, Volume 31, Issue 1, Page 837-865, January 2021. This paper deals with particular families of DC optimization problems involving suprema of convex functions. We show that the specific structure of this type of function allows us to cover a variety of problems in nonconvex programming. Necessary and sufficient optimality conditions for these families of DC optimization problems

SIAM Journal on Optimization, Volume 31, Issue 1, Page 812-836, January 2021. In semidefinite programming a proposed optimal solution may be quite poor in spite of having sufficiently small residual in the optimality conditions. This issue may be framed in terms of the discrepancy between forward error (the unmeasurable “true error'') and backward error (the measurable violation of optimality conditions)

SIAM Journal on Optimization, Volume 31, Issue 1, Page 785-811, January 2021. In this paper, we study greedy variants of quasi-Newton methods. They are based on the updating formulas from a certain subclass of the Broyden family. In particular, this subclass includes the well-known DFP, BFGS, and SR1 updates. However, in contrast to the classical quasi-Newton methods, which use the difference of successive

SIAM Journal on Optimization, Volume 31, Issue 1, Page 754-784, January 2021. We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with strongly convex regularization and general data-fit functions. We develop an inertial approach of which we analyze both convergence and stability properties. Using tools from inexact proximal calculus

SIAM Journal on Optimization, Volume 31, Issue 1, Page 727-753, January 2021. Smooth convex minimization over the unit trace-norm ball is an important optimization problem in machine learning, signal processing, statistics, and other fields that underlies many tasks in which one wishes to recover a low-rank matrix given certain measurements. While first-order methods for convex optimization enjoy optimal

SIAM Journal on Optimization, Volume 31, Issue 1, Page 702-726, January 2021. We address the bilevel optimization problem of identifying the most critical attacks to an alternating current (AC) power flow network. The upper-level binary maximization problem consists of choosing an attack that is treated as a parameter in the lower-level defender minimization problem. Instances of the lower-level global

SIAM Journal on Optimization, Volume 31, Issue 1, Page 686-701, January 2021. We consider a linear problem over a finite set of integer vectors and assume that there is a verification oracle, which is an algorithm being able to verify whether a given vector optimizes a given linear function over the feasible set. Given an initial solution, the algorithm proposed in this paper finds an optimal solution

SIAM Journal on Optimization, Volume 31, Issue 1, Page 653-685, January 2021. We introduce Bella, a locally superlinearly convergent Bregman forward-backward splitting method for minimizing the sum of two nonconvex functions, one of which satisfies a relative smoothness condition and the other one is possibly nonsmooth. A key tool of our methodology is the Bregman forward-backward envelope (BFBE),

SIAM Journal on Optimization, Volume 31, Issue 1, Page 626-652, January 2021. The split equality problem (SEP) seeks a pair of points $(x^{\ast },y^{\ast })\in (C,D)$ with the property that $Ax^{\ast }=By^{\ast }$, where $C,D$ are nonempty closed convex subsets of Hilbert spaces $\mathcal{H}_{1}$ and $% \mathcal{H}_{2}$, respectively, and $A:\mathcal{H}_{1}\rightarrow \mathcal{H}% _{3}$ and $B:\ma

SIAM Journal on Optimization, Volume 31, Issue 1, Page 604-625, January 2021. We study a class of countably infinite linear programs (CILPs) whose feasible sets are bounded subsets of appropriately defined spaces of measures. The optimal value, optimal points, and minimal points of these CILPs can be approximated by solving finite-dimensional linear programs. We show how to construct finite-dimensional

SIAM Journal on Optimization, Volume 31, Issue 1, Page 569-603, January 2021. This paper is devoted to developing and applications of a generalized differential theory of variational analysis that allows us to work in incomplete normed spaces, without employing conventional variational techniques based on completeness and limiting procedures. The main attention is paid to generalized derivatives and

SIAM Journal on Optimization, Volume 31, Issue 1, Page 545-568, January 2021. This paper mainly studies the relationship between quadratic growth and strong metric subregularity of the subdifferential in finite dimensional settings by using the subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous

SIAM Journal on Optimization, Volume 31, Issue 1, Page 518-544, January 2021. Worst-case complexity guarantees for nonconvex optimization algorithms have been a topic of growing interest. Multiple frameworks that achieve the best known complexity bounds among a broad class of first- and second-order strategies have been proposed. These methods have often been designed primarily with complexity guarantees

SIAM Journal on Optimization, Volume 31, Issue 1, Page 489-517, January 2021. In the paper, a Newton-type method for the solution of generalized equations (GEs) is derived, where the linearization concerns both the single-valued and the multivalued part of the considered GE. The method is based on the new notion of semismoothness${}^*$, which, together with a suitable regularity condition, ensures

SIAM Journal on Optimization, Volume 31, Issue 1, Page 461-488, January 2021. We consider a distributionally robust partially observable Markov decision process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations can be inferred using side information at the end of each period after an action being

SIAM Journal on Optimization, Volume 31, Issue 1, Page 448-460, January 2021. Several works have shown linear speedup is achieved by an asynchronous parallel implementation of stochastic coordinate descent so long as there is not too much parallelism. More specifically, it is known that if all updates are of similar duration, then linear speedup is possible with up to $\Theta(L_{\max}\sqrt n/L_{\overline{{res}}})$

SIAM Journal on Optimization, Volume 31, Issue 1, Page 425-447, January 2021. We consider a generalization of the Bauer maximum principle. We work with tensorial products of convex measures set that are not necessarily compact but generated by their extreme points. We show that the maximum of a quasi-convex lower semicontinuous function on this product space is reached on the tensorial product of finite

SIAM Journal on Optimization, Volume 31, Issue 1, Page 404-424, January 2021. Recently, a new kind of distance has been introduced for the graphs of two point-to-set operators, one of which is maximally monotone. When both operators are the subdifferential of a proper lower semicontinuous convex function, this kind of distance specializes under modest assumptions to the classical Bregman distance.

SIAM Journal on Optimization, Volume 31, Issue 1, Page 377-403, January 2021. We consider the generalized Nash equilibrium problem (GNEP) with $ N $ players in a Hilbert space setting. The joint constraints are eliminated by an augmented Lagrangian-type approach, leading to an ADMM (alternating direction method of multipliers) algorithm. In contrast to standard optimization problems, however, the direct

SIAM Journal on Optimization, Volume 31, Issue 1, Page 348-376, January 2021. In this work, we present a novel approach for solving stochastic shape optimization problems. Our method is the extension of the classical stochastic gradient method to infinite-dimensional shape manifolds. We prove convergence of the method on Riemannian manifolds and then make the connection to shape spaces. The method