We consider the distributionally robust optimization and show that computing the distributional worst-case is equivalent to computing the projection onto the canonical simplex with additional linear inequality. We consider several distance functions to measure the distance of distributions. We write the projections as optimization problems and show that they are equivalent to finding a zero of real-valued

Mathematical programs with vanishing constraints (MPVCs) are a class of nonlinear optimization problems with applications to various engineering problems such as truss topology design and robot motion planning. MPVCs are difficult problems from both a theoretical and numerical perspective: the combinatorial nature of the vanishing constraints often prevents standard constraint qualifications and optimality

A method for evaluating lexicographical directional (LD)-derivatives of functional programs is presented, extending previous methods to programs containing conditional branches and loops. A language for imperative programs is given, and conditions under which LD-derivatives can be calculated automatically for conditional branches and loops are described, along with a full description of the source

In this paper, we consider a joint-inversion problem using different types of geophysical data: gravity and magnetism. We first formulate two kinds of inverse problems in the famework of the first kind Fredholm integral equations, and then build up a sparse inversion model combining the two inverse problems as well as the cross-gradient term. The cyclic gradient method for quadratic function minimization

The complexity of solving feasibility problems is considered in this work. It is assumed that the constraints that define the problem can be divided into expensive and cheap constraints. At each iteration, the introduced method minimizes a regularized pth-order model of the sum of squares of the expensive constraints subject to the cheap constraints. Under a Hölder continuity property with constant

Extended formulations of ( k , l ) -sparsity matroids defined on graphs with n vertices and m edges are investigated by Iwata et al. [Extended formulations for sparsity matroids, Math. Program. 158 (2016), pp. 565–574]. This note interprets results on ( k , l ) -lower-truncated transversal polymatroids by the first author in 1983, from the viewpoint of extended formulations, and shows the same O (

For the Lagrangian-DNN relaxation of quadratic optimization problems (QOPs), we propose a Newton-bracketing method to improve the performance of the bisection-projection method implemented in BBCPOP [ACM Tran. Softw., 45(3):34 (2019)]. The relaxation problem is converted into the problem of finding the largest zero y ∗ of a continuously differentiable (except at y ∗ ) convex function g : R → R such

In this paper, we propose a new multilevel Levenberg–Marquardt optimizer for the training of artificial neural networks with quadratic loss function. This setting allows us to get further insight into the potential of multilevel optimization methods. Indeed, when the least squares problem arises from the training of artificial neural networks, the variables subject to optimization are not related by

The paper presents a new approach to solve multifacility location problems, which is based on mixed integer programming and algorithms for minimizing differences of convex (DC) functions. The main challenges for solving the multifacility location problems under consideration come from their intrinsic discrete, nonconvex, and nondifferentiable nature. We provide a reformulation of these problems as

An outerplanar graph is a graph that can be drawn on the plane without crossing edges so that all vertices are on the infinite face. Most organic compounds have outerplanar graph structures. The number of matchings in the skeleton graphs of organic compounds is known as the topological index Z, introduced in the early 70s to investigate correlation between molecular structures and physical properties

Sum-of-squares (SOS) tensors plays an important role in tensor positive definiteness and polynomial optimization. So it is important to figure out what kind of tensors are SOS tensors. In this paper, we first show that several types of even order symmetric tensors are SOS tensors. The inclusive relation between several types of existing SOS tensors are developed under suitable conditions. By exploring

In this work, motivated by the challenging task of learning a deep neural network, we consider optimization problems that consist of minimizing a finite-sum of non-convex and non-smooth functions, where the non-smoothness appears as the maximum of non-convex functions with Lipschitz continuous gradient. Due to the large size of the sum, in practice, we focus here on stochastic first-order methods and

Finding the symmetric rank-1 approximation to a given symmetric tensor is an important problem due to its wide applications and its close relationship to the Z-eigenpair of a tensor. In this paper, we propose a method based on the proximal alternating linearized minimization to directly solve the optimization problem. Global convergence of our algorithm is established. Numerical experiments show that

In this paper, we compute Z-eigenvalues of a class of tensors by studying the properties of semi-symmetric tensor. We prove that Axm−1 is identical to zero if and only if As=0, where As is the associated semi-symmetric tensor of A. Based on the semi-symmetric property, an almost nonnegative irreducible tensor is defined. And we use Newton method to compute Z-eigenvalues of this kind of tensor. The

We consider a nonconvex and nonsmooth group sparse optimization problem where the penalty function is the sum of compositions of a folded concave function and the ℓ2 vector norm for each group variable. We show that under some mild conditions a first-order directional stationary point is a strict local minimizer that fulfils the first-order growth condition, and a second-order directional stationary

This paper studies stochastic optimization problems with polynomials. We propose an optimization model with sample averages and perturbations. The Lasserre-type Moment-SOS relaxations are used to solve the sample average optimization. Properties of the optimization and its relaxations are studied. Numerical experiments are presented.

We derive bounds for the objective errors and gradient residuals when finding approximations to the solution of common regularized quadratic optimization problems within evolving Krylov spaces. These provide upper bounds on the number of iterations required to achieve a given stated accuracy. We illustrate the quality of our bounds on given test examples.

We expand the scope of the alternating direction method of multipliers (ADMM). Specifically, we show that ADMM, when employed to solve problems with multiaffine constraints that satisfy certain verifiable assumptions, converges to the set of constrained stationary points if the penalty parameter in the augmented Lagrangian is sufficiently large. When the Kurdyka–Łojasiewicz (K–Ł) property holds, this

An adaptive regularization algorithm is proposed that uses Taylor models of the objective of order p, p≥2, of the unconstrained objective function, and that is guaranteed to find a first- and second-order critical point in at most O(max{ϵ1−p+1p,ϵ2−p+1p−1}) function and derivatives evaluations, where ϵ1 and ϵ2 are prescribed first- and second-order optimality tolerances. This is a simple algorithm and

It has long been known that the gradient (steepest descent) method may fail on non-smooth problems, but the examples that have appeared in the literature are either devised specifically to defeat a gradient or subgradient method with an exact line search or are unstable with respect to perturbation of the initial point. We give an analysis of the gradient method with steplengths satisfying the Armijo

(2020). Preface. Optimization Methods and Software: Vol. 35, Part I of the special issue dedicated to the 60th birthday of Professor Ya-xiang Yuan. Guest-Editors: Yu-Hong Dai, Xin Liu, Jiawang Nie, Zaiwen Wen, pp. 221-222.

ABSTRACT In this paper, we study the auxiliary problems that appear in p-order tensor methods for unconstrained minimization of convex functions with ν-Hölder continuous pth derivatives. This type of auxiliary problems corresponds to the minimization of a (p+ν)-order regularization of the pth-order Taylor approximation of the objective. For the case p = 3, we consider the use of Gradient Methods with

We introduce exponential augmented Lagrangian methods for solving equilibrium problems in finite-dimensional spaces, extending the so-called quadratic augmented Lagrangian methods. Unlike the quadratic augmented Lagrangian methods that are at most first-order differentiable, our exponential augmented Lagrangian methods can be differentiable at any order. Therefore, second-order methods, such as Newton's

In this paper, we introduce a class of SOR-like iteration methods for solving the systems of the weakly nonlinear equation, which is by reformulating equivalently the weakly nonlinear equation as a two-by-two block nonlinear equation. Two types of the global convergence theorems are given under suitable choices of the involved splitting matrix and parameter. Numerical results for the three-dimensional

We study dual-based algorithms for distributed convex optimization problems over networks, where the objective is to minimize a sum ∑i=1mfi(z) of functions over in a network. We provide complexity bounds for four different cases, namely: each function fi is strongly convex and smooth, each function is either strongly convex or smooth, and when it is convex but neither strongly convex nor smooth. Our

Training generative adversarial networks (GANs) often suffers from cyclic behaviours of iterates. Based on a simple intuition that the direction of centripetal acceleration of an object moving in uniform circular motion is toward the centre of the circle, we present the Simultaneous Centripetal Acceleration (SCA) method and the Alternating Centripetal Acceleration (ACA) method to alleviate the cyclic

This work considers the minimization of a sum of an expectation-valued coordinate-wise smooth nonconvex function and a nonsmooth block-separable convex regularizer. We propose an asynchronous variance-reduced algorithm, where in each iteration, a single block is randomly chosen to update its estimates by a proximal variable sample-size stochastic gradient scheme, while the remaining blocks are kept

Algencan is a well established safeguarded Augmented Lagrangian algorithm introduced in [R. Andreani, E. G. Birgin, J. M. Martínez, and M. L. Schuverdt, On Augmented Lagrangian methods with general lower-level constraints, SIAM J. Optim. 18 (2008), pp. 1286–1309]. Complexity results that report its worst-case behaviour in terms of iterations and evaluations of functions and derivatives that are necessary

A class of direct search methods for locally minimizing a Lipschitz continuous black-box function f subject to locally Lipschitz constraints is presented. A sequence of smoothed ℓ1 penalty functions is used. Each smoothed penalty function is approximately minimized in turn. The smoothing is reduced after each minimization, exposing the ℓ1 exact penalty function in the limit. Convergence to a constrained

Professor Masao Iri and his students had several important contributions to the theory of electric networks. He was one of the first scientists who recognized the importance of matroid theory in studying the qualitative problems of electric networks. The present paper summarizes his results in the unique solvability of linear active networks and in the existence of hybrid immittance description of

The one-to-one correspondence between finite distributive lattices and finite partially ordered sets (posets) is a well-known theorem of G. Birkhoff. This implies a nice representation of any distributive lattice by its corresponding poset, where the size of the former (distributive lattice) is often exponential in the size of the underlying set of the latter (poset). A lot of engineering and economic

We propose and analyse a nonmonotone quasi-Newton algorithm for unconstrained strongly convex multiobjective optimization. In our method, we allow for the decrease of a convex combination of recent function values. We establish the global convergence and local superlinear rate of convergence under reasonable assumptions. We implement our scheme in the context of BFGS quasi-Newton method for solving

This paper begins with a brief review of affine invariance and its significance for iterative algorithms. It then explores the existence of affine invariant descent directions for unconstrained minimization. While there may exist several affine invariant descent directions for smooth functions at a given point, it is shown that for quadratic functions, there exists exactly one invariant descent direction

The paper illustrates connections between classical results of Analysis and Optimization. The focus is on new elementary proofs of Implicit Function Theorem, Lusternik's Theorem, and optimality conditions for equality constrained optimization problems. The proofs are based on Fermat's Theorem and the Weierstrass Theorem and do not use the contraction mapping principle or other advanced results of Real

We analyze the rate of convergence of the proximal method of multipliers for non-convex nonlinear programming problems. First, we prove, under the strict complementarity condition, that the rate of convergence of the proximal method of multipliers is linear and the ratio constant is proportional to 1/c when the ratio ‖(μ0,λ0)−(μ¯,λ¯)‖/c is small enough, which implies that the rate of convergence of

We introduce an Alternating Direction Method of Multipliers (ADMM) for finding a solution of the nonsymmetric Eigenvalue Complementarity Problem (EiCP). A simpler version of this method is proposed for the symmetric EiCP, that is, for the computation of a Stationary Point (SP) of a Standard Fractional Quadratic Program. The algorithm is also extended for the computation of an SP of a Standard Quadratic

In this paper, we propose a projection-free accelerated method for solving convex optimization problems with unbounded feasible set. The method is an accelerated gradient scheme such that each projection subproblem is approximately solved by means of a conditional gradient scheme. Under reasonable assumptions, it is shown that an ϵ-approximate solution (concept related to the optimal value of the problem)

We present a feasible full step interior-point algorithm to solve the P∗(κ) horizontal linear complementarity problem defined on a Cartesian product of symmetric cones, which is not based on a usual barrier function. The full steps are scaled utilizing the Nesterov-Todd (NT) scaling point. Our approach generates the search directions leading to the full-NT steps by algebraically transforming the centring

We consider two-machine routing open shop problem on a tree. In this problem, a transportation network with a tree-like structure is given, and each node contains some jobs to be processed by two mobile machines. Machines are initially located in the predefined node called the depot, have to traverse the network to perform their operations on each job (and each job has to be processed by both machines

This paper proposes two algorithms that are based on a subgradient and an inertial scheme with the explicit iterative method for solving pseudomonotone equilibrium problems. The weak convergence of both algorithms is well-established under standard assumptions on the cost bifunction. The advantage of these algorithms is that they did not require any line search procedure or any knowledge about bifunction

The ★-congruence Sylvester equation is the matrix equation AX+X⋆B=C, where A∈Fm×n, B∈Fn×m and C∈Fm×m are given, whereas X∈Fn×m is to be determined. Here, F=R or C, and ⋆=T (transposed) or ∗ (conjugate transposed). Very recently, Satake et al. showed that under some conditions, the matrix equation for the case ⋆=T is equivalent to the generalized Sylvester equation. In this paper, we demonstrate that

Support vector machine (SVM) has proved to be a successful approach for machine learning. Two typical SVM models are the L1-loss model for support vector classification (SVC) and ε-L1-loss model for support vector regression (SVR). Due to the non-smoothness of the L1-loss function in the two models, most of the traditional approaches focus on solving the dual problem. In this paper, we propose an augmented

The sparse optimization problems arise in many areas of science and engineering, such as compressed sensing, image processing, statistical and machine learning. The ℓ 0 -minimization problem is one of such optimization problems, which is typically used to deal with signal recovery. The ℓ 1 -minimization method is one of the popular approaches for solving the ℓ 0 -minimization problems, and the stability

In this paper, a new variant of accelerated gradient descent is proposed. The proposed method does not require any information about the objective function, uses exact line search for the practical accelerations of convergence, converges according to the well-known lower bounds for both convex and non-convex objective functions, possesses primal–dual properties and can be applied in the non-euclidian

We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73–88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method

(2020). An investigation of Newton-Sketch and subsampled Newton methods. Optimization Methods and Software: Vol. 35, Special issue dedicated to Professor Ya-xiang Yuan on the occasion of his 60th birthday, Part II. Guest-Editors: Yu-Hong Dai, Xin Liu, Jiawang Nie, Zaiwen Wen, pp. 661-680.

In this paper, we consider resource allocation problem stated as a convex minimization problem with linear constraints. To solve this problem, we use gradient and accelerated gradient descent applied to the dual problem and prove the convergence rate both for the primal iterates and the dual iterates. We obtain faster convergence rates than the ones known in the literature. We also provide economic

A p-order regularization method for finding weak stationary points of multiobjective optimization problems with constraints is introduced. Under Hölder conditions on the derivatives of the objective functions, complexity results are obtained that generalize properties recently proved for scalar optimization.

Supply chain integration has become one of the most attractive topics for researchers in recent years. One of the advantages of this integration is in improving overall profit in comparison to separate decisions. In this study, an integrated scheduling and distribution problem is investigated. One of the contributions of this paper is to study this problem from a multi-agent viewpoint. In this case

We introduce LMLS and LMQR, two globally convergent Levenberg–Marquardt methods for finding zeros of Hölder metrically subregular mappings that may have non-isolated zeros. The first method unifies the Levenberg–Marquardt direction and an Armijo-type line search, while the second incorporates this direction with a non-monotone quadratic regularization technique. For both methods, we prove the global

Canonical correlation analysis (CCA) is a cornerstone of linear dimensionality reduction techniques that jointly maps two datasets to achieve maximal correlation. CCA has been widely used in applications for capturing data features of interest. In this paper, we establish a range constrained orthogonal CCA (OCCA) model and its variant and apply them for three data analysis tasks of datasets in real-life

Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g. sparsity) efficiently. In this paper, we propose a novel accelerated dual-averaging primal–dual algorithm for minimizing a composite convex function. We also derive a stochastic version of the proposed method that solves empirical risk minimization, and

In this paper, we propose new first-order methods for minimization of a convex function on a simple convex set. We assume that the objective function is a composite function given as a sum of a simple convex function and a convex function with inexact Hölder-continuous subgradient. We propose Universal Intermediate Gradient Method. Our method enjoys both the universality and intermediateness properties

In the inverse optimization problem, we modify parameters of the original problem at minimum total cost so as to make a prespecified solution optimal with respect to new parameters. We extend in this paper a class of inverse single facility problems on trees, including inverse balance point, inverse 1-median and inverse 1-center problem, and call it the inverse stable point problem. For the general

In the network scheduling, jobs (tasks) must be scheduled on uniform machines (processors) connected by a complete graph so as to minimize the total weighted completion time. This setting can be applied in distributed multi-processor computing environments and also in operations research. In this paper, we study the design of randomized decentralized mechanism in the setting where a set of non-preemptive

In this paper, we propose a framework of Inexact Proximal Stochastic Second-order (IPSS) method for solving nonconvex optimization problems, whose objective function consists of an average of finitely many, possibly weakly, smooth functions and a convex but possibly nonsmooth function. At each iteration, IPSS inexactly solves a proximal subproblem constructed by using some positive definite matrix

The primary focus of this paper is on designing an inexact first-order algorithm for solving constrained nonlinear optimization problems. By controlling the inexactness of the subproblem solution, we can significantly reduce the computational cost needed for each iteration. A penalty parameter updating strategy during the process of solving the subproblem enables the algorithm to automatically detect

The paper concerns the elimination of a set of variables from a system of linear inequalities. We employ the widely used Fourier–Motzkin elimination method extended with the Chernikov rules. A straightforward implementation of the algorithm results in extensive enumeration during the most computationally demanding stage. We propose a new way of checking Chernikov rules using bit pattern trees as an

We consider a conflict-free minimum latency data aggregation problem that occurs in different wireless networks. Given a network that is presented as an undirected graph with one selected vertex (a sink), the goal is to find a spanning aggregation tree rooted in the sink and to define a conflict-free aggregation minimum length schedule along the arcs of the tree directed to the sink. Herewith, at the

Mathematical programs with equilibrium constraints is a difficult class of constrained optimization problems. The feasible set has a very special structure and violates most of the standard constraint qualifications. Thus, the Karush–Kuhn–Tucker conditions are not necessarily satisfied at minimizers, and the convergence assumptions of many methods for solving constrained optimization problems are not