The confined primal integral: a measure to benchmark heuristic MINLP solvers against global MINLP solvers

Abstract

It is a challenging task to fairly compare local solvers and heuristics against each other and against global solvers. How does one weigh a faster termination time against a better quality of the found solution? In this paper, we introduce the confined primal integral, a new performance measure that rewards a balance of speed and solution quality. It emphasizes the early part of the solution process by using an exponential decay. Thereby, it avoids that the order of solvers can be inverted by choosing an arbitrarily large time limit. We provide a closed analytic formula to compute the confined primal integral a posteriori and an incremental update formula to compute it during the run of an algorithm. For the latter, we show that we can drop one of the main assumptions of the primal integral, namely the knowledge of a fixed reference solution to compare against. Furthermore, we prove that the confined primal integral is a transitive measure when comparing local solves with different final solution values. Finally, we present a computational experiment where we compare a local MINLP solver that uses certain classes of cutting planes against a solver that does not. Both versions show very different tendencies w.r.t. average running time and solution quality, and we use the confined primal integral to argue which of the two is the preferred setting.

Appendix

Lemma 2

Consider a series $C=\{c_1,\dots ,c_k\}$ of incumbent solutions and a new incumbent $c_{k+1}$ with $c_{k+1} < c_k$ being found at time $t_{k+1}$. If $c_{1} < 0$ (and hence all other $c_{i} < 0$) , then $P^{\text {cor}}_\alpha (t_{k+1},C,\{c_{k+1}\}) = \frac{c_k}{c_{k+1}}P^{\text {obs}}_\alpha (t_{k},C) + \alpha \cdot {\tilde{t}}_{k+1} \frac{c_{k+1}-c_k}{c_{k+1}}$. If $c_{k+1} > 0$ (and hence all other $c_{i} > 0$), then $P^{\text {cor}}_\alpha (t_{k+1},C,\{c_{k+1}\}) = P^{\text {obs}}_\alpha (t_{k},C) + D(c_{k}-c_{k+1})$ with a dynamic scaling factor $D=\alpha \sum _{i=2}^{k+1}\frac{{\tilde{t}}_i-{\tilde{t}}_{i-1}}{c_i}$.

Proof

Case 1, $c_{1} < 0$:

$$\begin{aligned}&P^{\text {cor}}_\alpha (t_{k+1},C,\{c_{k+1}\}) \\&= {\alpha }\sum _{i = 1}^{k+1} (\exp (\nicefrac {t_{i}}{\alpha })-\exp (\nicefrac {t_{i-1}}{\alpha }))p_{\tilde{x} _{\text {opt}}}(t _{i-1})\\&= {\alpha }\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k+1} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k+1}-c_{i-1}}{c_{k+1}}\Big )\\&= {\alpha }\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k+1}-c_{i-1}}{c_{k+1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k+1}-c_{k}}{c_{k+1}}\Big )\\&= {\alpha }\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k}-c_{i-1}+c_{k+1}-c_{k}}{c_{k+1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k+1}-c_{k}}{c_{k+1}}\Big )\\&= {\alpha }\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k}-c_{i-1}}{c_{k+1}}+ \sum _{i=1}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k+1}-c_{k}}{c_{k+1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k+1}-c_{k}}{c_{k+1}}\Big )\\&= {\alpha }\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k}-c_{i-1}}{c_{k+1}}+ {\tilde{t}}_{k+1}\frac{c_{k+1}-c_{k}}{c_{k+1}}\Big )\\&= {\alpha }\frac{c_{k}}{c_{k+1}}\Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k}-c_{i-1}}{c_{k}}\Big )+ {\alpha }\cdot {\tilde{t}}_{k+1}\frac{c_{k+1}-c_{k}}{c_{k+1}}\\&= \frac{c_{k}}{c_{k+1}}P^{\text {obs}}_\alpha (t_{k},C)+ {\alpha }\cdot {\tilde{t}}_{k+1}\frac{c_{k+1}-c_{k}}{c_{k+1}} \end{aligned}$$

Case 2, $c_{k+1} > 0$:

$$\begin{aligned} \begin{aligned}&P^{\text {cor}}_\alpha (t_{k+1},C,\{c_{k+1}\}) \\&= \alpha \Big ({\tilde{t}}_{1}+\sum _{i=2}^{k+1} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{i-1}-c_{k+1}}{c_{i-1}}\Big )\\&= \alpha \Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{i-1}-c_{k+1}}{c_{i-1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k}-c_{k+1}}{c_{k}}\Big )\\&= \alpha \Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{i-1}-c_{k}+c_{k}-c_{k+1}}{c_{i-1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k}-c_{k+1}}{c_{k}}\Big )\\&= \alpha \Big ({\tilde{t}}_{1}+\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{i-1}-c_{k}}{c_{i-1}}\\&+ (c_{k}-c_{k+1})\sum _{i=2}^{k} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{1}{c_{i-1}} + ({\tilde{t}}_{k+1}-{\tilde{t}}_{k})\frac{c_{k}-c_{k+1}}{c_{k}}\Big )\\&= P^{\text {obs}}_\alpha (t_{k},C) + \alpha (c_{k}-c_{k+1})\sum _{i=2}^{k+1} ({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{1}{c_{i-1}}\\ \end{aligned} \end{aligned}$$

$\square$

Note that in the above proof, all sums including a $c_{k}$ start with 2, since the case $c_0= \infty$ requires a special handling. According to Definition 2, the primal gap function is defined to be 1 in this case instead of being a difference of two objective function values divided by the larger one. Since ${\tilde{t}}_{0}=0$, the summand for $i=0$ becomes ${\tilde{t}}_{1}$ in both cases of the proof, instead of $({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{k+1}-c_{i-1}}{c_{k+1}}$ in Case 1 or $({\tilde{t}}_i-{\tilde{t}}_{i-1})\frac{c_{i-1}-c_{k+1}}{c_{i-1}}$ in Case 2.

Also note that the case that the incumbent solution switches the sign during optimization allows for an easy update, too. If the new incumbent at point $t_{k+1}$ is the first one with a negative sign, then $P^{\text {cor}}_\alpha (T,C,\{c_{k+1}\})=t_{k+1}$. If the sign switch happens at some point $t_i$ with $2<i<k+1$, then the first part (until point i) of both primal integral sums is identical, and the second part (all summands greater i) can be updated as in Case 2 of the proof.