An infeasible-start framework for convex quadratic optimization, with application to constraint-reduced interior-point and other methods

Abstract

A framework is proposed for solving general convex quadratic programs (CQPs) from an infeasible starting point by invoking an existing feasible-start algorithm tailored for inequality-constrained CQPs. The central tool is an exact penalty function scheme equipped with a penalty-parameter updating rule. The feasible-start algorithm merely has to satisfy certain general requirements, and so is the updating rule. Under mild assumptions, the framework is proved to converge on CQPs with both inequality and equality constraints and, at a negligible additional cost per iteration, produces an infeasibility certificate, together with a feasible point for an (approximately) $\ell _1$-least relaxed feasible problem, when the given problem does not have a feasible solution. The framework is applied to a feasible-start constraint-reduced interior-point algorithm previously proved to be highly performant on problems with many more inequality constraints than variables (“imbalanced”). Numerical comparison with popular codes (OSQP, qpOASES, MOSEK) is reported on both randomly generated problems and support-vector machine classifier training problems. The results show that the former typically outperforms the latter on imbalanced problems. Finally, application of the proposed infeasible-start framework to other feasible-start algorithms is briefly considered, and is tested on a simplex iteration.

Appendices

Appendix

Two algorithms considered in this paper are presented here for the sake of completeness. Some notation used in the appendix is “local”, i.e., only applies to this appendix; indeed, some of the same symbols are used in the main body of the paper for unrelated quantities.

A Feasible-start algorithm CR-MPC

In this appendix, we present a brief description of Algorithm CR-MPC in [28], as applied to a generic inequality-constrained CQP of the form (6). We refer the reader to [28] for the complete version of Algorithm CR-MPC in full detail.

For ease of reference, we restate the generic inequality-constrained CQP (6) here:

$$\begin{aligned} \mathop {\mathrm{minimize}}\limits _{x} \, f(x):=\frac{1}{2}x^THx+c^Tx \text{ s.t. } \, Ax \ge b. \end{aligned}$$

(58)

Algorithm CR-MPC is a feasible-start, descent CQP variant of S. Mehrotra’s predictor–corrector algorithm [31]; computational efficiency is enhanced on imbalanced problems by solving a mere approximate Newton-KKT system for search directions (“constraint reduction”). Specifically, the Newton-KKT system is approximated by including only a selected subset of constraints, which aims to estimate the active constraint set at a solution to (58). Also, an adaptive positive definite regularization W of the original Hessian H is substituted in that system.

Let Q denote the index set of the selected set of constraints, then the affine-scaling search direction is given by the solution of the approximate Newton-KKT system for (58)

$$\begin{aligned} \left[ \begin{array}{ccc} W &{} \quad -A_Q^T &{} \quad \mathbf{0}\\ A_Q &{} \quad \mathbf{0}&{} \quad -I\\ \mathbf{0}&{} \quad S_Q &{} \quad \Lambda _Q \end{array}\right] \left[ \begin{array}{c} \Delta x^{\mathrm{a}}\\ \Delta \lambda _Q^{\mathrm{a}}\\ \Delta s_Q^{\mathrm{a}} \end{array}\right] =\left[ \begin{array}{c} -\nabla f(x)+(A_Q)^T\lambda _Q \\ \mathbf{0}\\ -S_Q\lambda _Q \end{array}\right] \,, \end{aligned}$$

(59)

and the corrector direction is computed by solving

$$\begin{aligned} \left[ \begin{array}{ccc} W &{} \quad -A_Q^T &{} \quad \mathbf{0}\\ A_Q &{} \quad \mathbf{0}&{} \quad -I\\ \mathbf{0}&{} S_Q &{} \Lambda _Q \end{array}\right] \left[ \begin{array}{c} \Delta x^{\mathrm{c}}\\ \Delta \lambda _Q^{\mathrm{c}}\\ \Delta s_Q^{\mathrm{c}} \end{array}\right] =\left[ \begin{array}{c} \mathbf{0}\\ \mathbf{0}\\ \sigma \mu _{(Q)}\mathbf {1}-\Delta S_Q^{\mathrm{a}}\Delta \lambda _Q^{\mathrm{a}} \end{array}\right] \end{aligned}$$

(60)

where $\mu _{(Q)}:={\left\{ \begin{array}{ll} s_Q^T\lambda _Q/|Q|\,, &{} \quad \text {if } Q\ne \emptyset \\ 0\,, &{} \quad \text {otherwise} \end{array}\right. }$ and $\sigma =(1-\alpha ^{\mathrm{a}})^3$ with $\alpha ^{\mathrm{a}}$ the maximum feasible step for the affine-scaling direction. A pseudocode for an iteration in Algorithm CR-MPC is given in Pseudocode 5, which can serve as base_iteration() in Meta-Algorithm IS with minimal modifications that tailor the code for penalized relaxed problems of the form (P$_\varphi$).

B Revised primal simplex (RPS) algorithm

In this appendix, we give a brief description of the revised primal simplex method, which is considered in Sect. 6.3.1 in the comparison to the CR-MPC method discussed in Appendix A, as well as in Sect. 7 as a base iteration in the IS framework. We consider the RPS method that handles linear optimization problems in primal standard form

$$\begin{aligned} \mathop {\mathrm{minimize}}\limits _{x} \, c^Tx \text{ s.t. } \, Cx = d,\, x\ge \mathbf{0}. \end{aligned}$$

(61)

The RPS method considered here starts from an initial primal BFS for (61) and, at each iteration, it reduces the objective function value by moving to an adjacent primal BFS with lower value. Specifically, each RPS iteration forms a descent search direction by adding an entering column to the basis associated to the BFS, and takes a step towards the descent direction until one of the BFS entries becomes zero, the index of which is then identified as the leaving column and is removed from the basis. Pseudocode 6 presents the main steps of an RPS iteration, which can be used as iteration_RPS() in the wrapped RPS base iteration in Pseudocode 4 by incorporating the penalty parameter $\varphi$ into the input.