Recently, a new approach to tackle cardinality-constrained optimization problems based on a continuous reformulation of the problem was proposed. Following this approach, we derive a problem-tailored sequential optimality condition, which is satisfied at every local minimizer without requiring any constraint qualification. We relate this condition to an existing M-type stationary concept by introducing

This paper presents two new bidirectional heuristic search algorithms for solving the shortest path problem on graphs: consistent-heuristic bucket-based bidirectional search (CBBS) and front-to-front GPU bidirectional search (FFGBS). CBBS uses a consistent heuristic and groups nodes into buckets that organize nodes based on estimated path cost and known heuristic errors. FFGBS splits the work between

We prove a-posteriori error-estimates for reduced-order modeling of quasilinear parabolic PDEs with non-monotone nonlinearity. We consider the solution of a semi-discrete in space equation as reference, and therefore incorporate reduced basis-, empirical interpolation-, and time-discretization-errors in our consideration. Numerical experiments illustrate our results.

For general large-scale optimization problems compact representations exist in which recursive quasi-Newton update formulas are represented as compact matrix factorizations. For problems in which the objective function contains additional structure, recent structured quasi-Newton methods exploit available second-derivative information and approximate unavailable second derivatives. This article develops

In this paper we consider the weighted nonlinear complementarity problem (denoted by wNCP) which contains a wide class of optimization problems. We introduce a family of new weighted complementarity functions and show that it is continuously differentiable everywhere and has several favorable properties. Based on this function, we reformulate the wNCP as a smooth nonlinear equation and propose a nonmonotone

The weighted essentially non-oscillatory (WENO) methods are popular and effective spatial discretization methods for nonlinear hyperbolic partial differential equations. Although these methods are formally first-order accurate when a shock is present, they still have uniform high-order accuracy right up to the shock location. In this paper, we propose a novel third-order numerical method for solving

In the present paper we propose to rewrite a nonsmooth problem subjected to convex constraints as an unconstrained problem. We show that this novel formulation shares the same global and local minima with the original constrained problem. Moreover, the reformulation can be solved with standard nonsmooth optimization methods if we are able to make projections onto the feasible sets. Numerical evidence

In this paper we consider convex feasibility problems where the feasible set is given as the intersection of a collection of closed convex sets. We assume that each set is specified algebraically as a convex inequality, where the associated convex function is general (possibly non-differentiable). For finding a point satisfying all the convex inequalities we design and analyze random projection algorithms

The classical convex feasibility problem in a finite dimensional Euclidean space consists of finding a point in the intersection of two convex sets. In the present paper we are interested in two particular instances of this problem. First, we assume to know how to compute an exact projection onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method

In this paper, a class of stochastic mathematical programs with probabilistic complementarity constraints is considered. We first investigate convergence properties of sample average approximation (SAA) approach to the corresponding chance constrained relaxed complementarity problem. Our discussion can be not only applied to the specific model in this paper, but also viewed as a supplementary for the

The worst-case linear optimization (WCLO) with uncertainties in the right-hand-side of the constraints often arises from numerous applications such as systemic risk estimate in finance and stochastic optimization, which is known to be NP-hard. In this paper, we investigate the efficient global algorithm for WCLO based on its nonlinear semidefinite relaxation (SDR). We first derive an enhanced nonlinear

In this paper, the problem of best subset selection in logistic regression is addressed. In particular, we take into account formulations of the problem resulting from the adoption of information criteria, such as AIC or BIC, as goodness-of-fit measures. There exist various methods to tackle this problem. Heuristic methods are computationally cheap, but are usually only able to find low quality solutions

In this paper, we analyze optimal control problems governed by an elliptic partial differential equation, in which the control acts as the Dirichlet data. Box constraints for the controls are imposed and the cost functional involves the state and possibly a sparsity-promoting term, but not a Tikhonov regularization term. Two different discretizations are investigated: the variational approach and a

Sparse multinomial logistic regression has recently received widespread attention. It provides a useful tool for solving multi-classification problems in various fields, such as signal and image processing, machine learning and disease diagnosis. In this paper, we first study the group sparse multinomial logistic regression model and establish its optimality conditions. Based on the theoretical results

Relative smoothness—a notion introduced in Birnbaum et al. (Proceedings of the 12th ACM conference on electronic commerce, ACM, pp 127–136, 2011) and recently rediscovered in Bauschke et al. (Math Oper Res 330–348, 2016) and Lu et al. (Relatively-smooth convex optimization by first-order methods, and applications, arXiv:1610.05708, 2016)—generalizes the standard notion of smoothness typically used

We introduce and analyze BPALM and A-BPALM, two multi-block proximal alternating linearized minimization algorithms using Bregman distances for solving structured nonconvex problems. The objective function is the sum of a multi-block relatively smooth function (i.e., relatively smooth by fixing all the blocks except one) and block separable (nonsmooth) nonconvex functions. The sequences generated by

One of the commonly used techniques for tackling the nonconvex optimization problems in which all the nonlinear terms are univariate is the piecewise linear approximation by which the nonlinear terms are reformulated. The performance of the linearization technique primarily depends on the quantities of variables and constraints required in the formulation of a piecewise linear function. The state-of-the-art

We develop a dual simplex-type algorithm for computing the smallest enclosing ball of a set of balls and other closely related problems. Our algorithm employs a pivoting scheme resembling the simplex method for linear programming, in which a sequence of exact curve searches is performed until a new dual feasible solution with a strictly smaller objective function value is found. We utilize the Cholesky

In this paper we address game theory problems arising in the context of network security. In traditional game theory problems, given a defender and an attacker, one searches for mixed strategies which minimize a linear payoff functional. In the problems addressed in this paper an additional quadratic term is added to the minimization problem. Such term represents switching costs, i.e., the costs for

In this paper, we describe and establish iteration-complexity of two accelerated composite gradient (ACG) variants to solve a smooth nonconvex composite optimization problem whose objective function is the sum of a nonconvex differentiable function f with a Lipschitz continuous gradient and a simple nonsmooth closed convex function h. When f is convex, the first ACG variant reduces to the well-known

Jordan Algebras are an important tool for dealing with semidefinite programming and optimization over symmetric cones in general. In this paper, a judicious use of Jordan Algebras in the context of sequential optimality conditions is done in order to generalize the global convergence theory of an Augmented Lagrangian method for nonlinear semidefinite programming. An approximate complementarity measure

The main focus in this paper is exact linesearch methods for minimizing a quadratic function whose Hessian is positive definite. We give a class of limited-memory quasi-Newton Hessian approximations which generate search directions parallel to those of the BFGS method, or equivalently, to those of the method of preconditioned conjugate gradients. In the setting of reduced Hessians, the class provides

Euclidean embedding from noisy observations containing outlier errors is an important and challenging problem in statistics and machine learning. Many existing methods would struggle with outliers due to a lack of detection ability. In this paper, we propose a matrix optimization based embedding model that can produce reliable embeddings and identify the outliers jointly. We show that the estimators

Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in

Rank minimization is of interest in machine learning applications such as recommender systems and robust principal component analysis. Minimizing the convex relaxation to the rank minimization problem, the nuclear norm, is an effective technique to solve the problem with strong performance guarantees. However, nonconvex relaxations have less estimation bias than the nuclear norm and can more accurately

We study the convergence rate of the Circumcentered-Reflection Method (CRM) for solving the convex feasibility problem and compare it with the Method of Alternating Projections (MAP). Under an error bound assumption, we prove that both methods converge linearly, with asymptotic constants depending on a parameter of the error bound, and that the one derived for CRM is strictly better than the one for

We present a stochastic descent algorithm for unconstrained optimization that is particularly efficient when the objective function is slow to evaluate and gradients are not easily obtained, as in some PDE-constrained optimization and machine learning problems. The algorithm maps the gradient onto a low-dimensional random subspace of dimension \(\ell\) at each iteration, similar to coordinate descent

In this work we introduce the concept of an Underestimate Sequence (UES), which is motivated by Nesterov’s estimate sequence. Our definition of a UES utilizes three sequences, one of which is a lower bound (or under-estimator) of the objective function. The question of how to construct an appropriate sequence of lower bounds is addressed, and we present lower bounds for strongly convex smooth functions

We consider the problem of minimizing the sum of two convex functions: one is differentiable and relatively smooth with respect to a reference convex function, and the other can be nondifferentiable but simple to optimize. We investigate a triangle scaling property of the Bregman distance generated by the reference convex function and present accelerated Bregman proximal gradient (ABPG) methods that

We apply proof mining methods to analyse a result of Boikanyo and Moroşanu on the strong convergence of a Halpern-type proximal point algorithm. As a consequence, we obtain quantitative versions of this result, providing uniform effective rates of asymptotic regularity and metastability.

The optimization problems arising in modern engineering practice are increasingly simulation-based, characterized by extreme types of nonsmoothness, the inaccessibility of derivatives, and high computational expense. While generating set searches (GSS) generally offer a satisfying level of robustness and converge to stationary points, the convergence rates may be slow. In order to accelerate the solution

The context of this research is multiobjective optimization where conflicting objectives are present. In this work, these objectives are only available as the outputs of a blackbox for which no derivative information is available. This work proposes a new extension of the mesh adaptive direct search (MADS) algorithm to multiobjective derivative-free optimization with bound constraints. This method

We propose a first-order method to solve the cubic regularization subproblem (CRS) based on a novel reformulation. The reformulation is a constrained convex optimization problem whose feasible region admits an easily computable projection. Our reformulation requires computing the minimum eigenvalue of the Hessian. To avoid the expensive computation of the exact minimum eigenvalue, we develop a surrogate

We propose a novel differentiable reformulation of the linearly-constrained \(\ell _1\) minimization problem, also known as the basis pursuit problem. The reformulation is inspired by the Laplacian paradigm of network theory and leads to a new family of gradient-based methods for the solution of \(\ell _1\) minimization problems. We analyze the iteration complexity of a natural solution approach to

A computational method is developed for solving time consistent distributionally robust multistage stochastic linear programs with discrete distribution. The stochastic structure of the uncertain parameters is described by a scenario tree. At each node of this tree, an ambiguity set is defined by conditional moment constraints to guarantee time consistency. This method employs the idea of nested Benders

We present a stochastic extension of the mesh adaptive direct search (MADS) algorithm originally developed for deterministic blackbox optimization. The algorithm, called StoMADS, considers the unconstrained optimization of an objective function f whose values can be computed only through a blackbox corrupted by some random noise following an unknown distribution. The proposed method is based on an

We consider a nonconvex mixed-integer nonlinear programming (MINLP) model proposed by Goldberg et al. (Comput Optim Appl 58:523–541, 2014. https://doi.org/10.1007/s10589-014-9647-y) for piecewise linear function fitting. We show that this MINLP model is incomplete and can result in a piecewise linear curve that is not the graph of a function, because it misses a set of necessary constraints. We provide

The focus in this paper is interior-point methods for bound-constrained nonlinear optimization, where the system of nonlinear equations that arise are solved with Newton’s method. There is a trade-off between solving Newton systems directly, which give high quality solutions, and solving many approximate Newton systems which are computationally less expensive but give lower quality solutions. We propose

This study considers a proximal Newton-type method to solve the minimization of a composite function that is the sum of a smooth nonconvex function and a nonsmooth convex function. In general, the method uses the Hessian matrix of the smooth portion of the objective function or its approximation. The uniformly positive definiteness of the matrix plays an important role in establishing the global convergence

We consider the problem of designing a cloak for waves described by the Helmholtz equation from an integer programming point of view. The problem can be modeled as a PDE-constrained optimization problem with integer-valued control inputs that are distributed in the computational domain. A first-discretize-then-optimize approach results in a large-scale mixed-integer nonlinear program that is in general

We analyze the conditional gradient method, also known as Frank–Wolfe method, for constrained multiobjective optimization. The constraint set is assumed to be convex and compact, and the objectives functions are assumed to be continuously differentiable. The method is considered with different strategies for obtaining the step sizes. Asymptotic convergence properties and iteration-complexity bounds

This article introduces a new algorithm for computing the set of supported non-dominated points in the objective space and all the corresponding efficient solutions in the decision space for the multi-objective spanning tree (MOST) problem. This algorithm is based on the connectedness property of the set of efficient supported solutions and uses a decomposition of the weight set in the weighting space

The minimum cut problem, MC, and the special case of the vertex separator problem, consists in partitioning the set of nodes of a graph G into k subsets of given sizes in order to minimize the number of edges cut after removing the k-th set. Previous work on approximate solutions uses, in increasing strength and expense: eigenvalue, semidefinite programming, SDP, and doubly nonnegative, DNN, bounding

In optimization, one of the main challenges of the widely used family of Quasi-Newton methods is to find an estimate of the Hessian matrix as close as possible to the real matrix. In this paper, we develop a new update formula for the estimate of the Hessian starting from the Powell-Symetric-Broyden (PSB) formula and adding pieces of information from the previous steps of the optimization path. This

For finite-dimensional problems, stochastic approximation methods have long been used to solve stochastic optimization problems. Their application to infinite-dimensional problems is less understood, particularly for nonconvex objectives. This paper presents convergence results for the stochastic proximal gradient method applied to Hilbert spaces, motivated by optimization problems with partial differential

This article studies Gauss–Newton-type methods for over-determined systems to find solutions to bilevel programming problems. To proceed, we use the lower-level value function reformulation of bilevel programs and consider necessary optimality conditions under appropriate assumptions. First, under strict complementarity for upper- and lower-level feasibility constraints, we prove the convergence of

We consider a certain class of hierarchical decision problems that can be viewed as single-leader multi-follower games, and be represented by a virtual market coordinator trying to set a price system for traded goods, according to some criterion that balances supply and demand. The objective function of the market coordinator involves the decisions of many agents, which are taken independently by solving

Considering saddle-point systems of the Karush–Kuhn–Tucker (KKT) form, we propose approximations of the “ideal” block diagonal preconditioner based on the exact Schur complement proposed by Murphy et al. (SIAM J Sci Comput 21(6):1969–1972, 2000). We focus on the case where the (1,1) block is symmetric and positive definite, but with a few very small eigenvalues that possibly affect the convergence

This paper studies tensor Z-eigenvalue complementarity problems. We formulate the tensor Z-eigenvalue complementarity problem as constrained polynomial optimization, and propose a semidefinite relaxation algorithm for solving the complementarity Z-eigenvalues of tensors. For every tensor that has finitely many complementarity Z-eigenvalues, we can compute all of them and show that our algorithm has

We develop techniques to construct a series of sparse polyhedral approximations of the semidefinite cone. Motivated by the semidefinite (SD) bases proposed by Tanaka and Yoshise (Ann Oper Res 265:155–182, 2018), we propose a simple expansion of SD bases so as to keep the sparsity of the matrices composing it. We prove that the polyhedral approximation using our expanded SD bases contains the set of

The Karush–Kuhn–Tucker and value function (lower-level value function, to be precise) reformulations are the most common single-level transformations of the bilevel optimization problem. So far, these reformulations have either been studied independently or as a joint optimization problem in an attempt to take advantage of the best properties from each model. To the best of our knowledge, these reformulations

Consider a nonempty finite set of nonzero vectors \(S \subset \mathbb {R}^n\). The angle between a nonzero vector \(v \in \mathbb {R}^n\) and S is the smallest angle between v and an element of S. The cosine measure of S is the cosine of the largest possible angle between a nonzero vector \(v \in \mathbb {R}^n\) and S. The cosine measure provides a way of quantifying the positive spanning property

We consider vector equilibrium problems in real Banach spaces and study their regularized problems. Based on cone continuity and generalized convexity properties of vector-valued mappings, we propose general conditions that guarantee existence of solutions to such problems in cases of monotonicity and nonmonotonicity. First, our study indicates that every Tikhonov trajectory converges to a solution

In this paper, we show that the quadratic assignment problem (QAP) can be reformulated to an equivalent rank constrained doubly nonnegative (DNN) problem. Under the framework of the difference of convex functions (DC) approach, a semi-proximal DC algorithm is proposed for solving the relaxation of the rank constrained DNN problem whose subproblems can be solved by the semi-proximal augmented Lagrangian

The combinatorial integral approximation (CIA) decomposition suggests solving mixed-integer optimal control problems by solving one continuous nonlinear control problem and one mixed-integer linear program (MILP). Unrealistic frequent switching can be avoided by adding a constraint on the total variation to the MILP. Within this work, we present a fast heuristic way to solve this CIA problem and investigate

This work considers nonsmooth and nonconvex optimization problems whose objective and constraint functions are defined by difference-of-convex (DC) functions. We consider an infeasible bundle method based on the so-called improvement functions to compute critical points for problems of this class. Our algorithm neither employs penalization techniques nor solves subproblems with linearized constraints

Convex programming based robust optimization is an active research topic in the past two decades, partially because of its computational tractability for many classes of optimization problems and uncertainty sets. However, many problems arising from modern operations research and statistical learning applications are nonconvex even in the nominal case, let alone their robust counterpart. In this paper

Optimization problems with composite functions consist of an objective function which is the sum of a smooth and a (convex) nonsmooth term. This particular structure is exploited by the class of proximal gradient methods and some of their generalizations like proximal Newton and quasi-Newton methods. The current literature on these classes of methods almost exclusively considers the case where also

This paper concerns the generalized Nash equilibrium problem of polynomials (GNEPP). We apply the Gauss–Seidel method and Moment-SOS relaxations to solve GNEPPs. The convergence of the Gauss–Seidel method is known for some special GNEPPs, such as generalized potential games (GPGs). We give a sufficient condition for GPGs and propose a numerical certificate, based on Putinar’s Positivstellensatz. Numerical

This work describes a new variant of projective splitting for solving maximal monotone inclusions and complicated convex optimization problems. In the new version, cocoercive operators can be processed with a single forward step per iteration. In the convex optimization context, cocoercivity is equivalent to Lipschitz differentiability. Prior forward-step versions of projective splitting did not fully