An efficient alternating minimization method for fourth degree polynomial optimization

Abstract

In this paper, we consider a class of fourth degree polynomial problems, which are NP-hard. First, we are concerned with the bi-quadratic optimization problem (Bi-QOP) over compact sets, which is proven to be equivalent to a multi-linear optimization problem (MOP) when the objective function of Bi-QOP is concave. Then, we introduce an augmented Bi-QOP (which can also be regarded as a regularized Bi-QOP) for the purpose to guarantee the concavity of the underlying objective function. Theoretically, both the augmented Bi-QOP and the original problem share the same optimal solutions when the compact sets are specified as unit spheres. By exploiting the multi-block structure of the resulting MOP, we accordingly propose a proximal alternating minimization algorithm to get an approximate optimal value of the problem under consideration. Convergence of the proposed algorithm is established under mild conditions. Finally, some preliminary computational results on synthetic datasets are reported to show the efficiency of the proposed algorithm.

Appendices

Appendix

In this appendix, we present detailed proofs for Theorems 4 and 5.

A Proof of Theorem 4

Proof

Due to the compactness of both ${\mathbb {D}}_1$ and ${\mathbb {D}}_2$, it is not difficult to see that the sequence $\{(\mathbf{x}^{(k)},\mathbf{y}^{(k)},\mathbf{z}^{(k)},\mathbf{w}^{(k)})\}$ generated by Algorithm 1 is bounded, which further implies that such a sequence has at least one limit point $\mathbf{t}^*=(\mathbf{x}^*,\mathbf{y}^*,\mathbf{z}^*,\mathbf{w}^*)$. Without loss of generality, suppose that $\{\mathbf{t}^{(p_k)}\}$ is a subsequence of $\{\mathbf{t}^{(k)}\}$ with limit point $\mathbf{t}^*$, i.e.,

$$\begin{aligned} \lim _{k\rightarrow \infty }{} \mathbf{t}^{(p_k)}=\mathbf{t}^*. \end{aligned}$$

It follows from Item (ii) of Theorem 3 that

$$\begin{aligned} \Vert \mathbf{t}^{(p_k+1)}-\mathbf{t}^*\Vert \le \Vert \mathbf{t}^{(p_k+1)}-\mathbf{t}^{(p_k)}\Vert +\Vert \mathbf{t}^{(p_k)}-\mathbf{t}^*\Vert \rightarrow 0,~~ k\rightarrow \infty . \end{aligned}$$

To verify that $\mathbf{t}^*$ is a stationary point of (9), we just need to prove the following variational inequality holds:

$$\begin{aligned} \langle \nabla f_{\alpha }(\mathbf{t}^*), \mathbf{t}-\mathbf{t}^*\rangle \ge 0,\quad \forall \mathbf{t}\in {\mathbb {D}}_1\times {\mathbb {D}}_1\times {\mathbb {D}}_2\times {\mathbb {D}}_2. \end{aligned}$$

(14)

Since $\{\mathbf{t}^{(p_k)}\}$ is a subsequence of $\{\mathbf{t}^{(k)}\}$, it is clear from the $\mathbf{x}$-subproblem of Algorithm 1 that

$$\begin{aligned} f_{\alpha }(\mathbf{x},\mathbf{y}^{(p_k)},\mathbf{z}^{(p_k)},\mathbf{w}^{(p_k)})\ge f_{\alpha }(\mathbf{x}^{(p_k+1)},\mathbf{y}^{(p_k)},\mathbf{z}^{(p_k)},\mathbf{w}^{(p_k)}),~\forall ~\mathbf{x}\in {\mathbb {D}}_1. \end{aligned}$$

Consequently, letting $k\rightarrow \infty$ in the above inequality leads to

$$\begin{aligned} \langle \nabla _\mathbf{x}f_{\alpha }(\mathbf{t}^*),\mathbf{x}-\mathbf{x}^*\rangle&=f_{\alpha }(\mathbf{x}-\mathbf{x}^*,\mathbf{y}^*,\mathbf{z}^*,\mathbf{w}^*) \nonumber \\&=f_{\alpha }(\mathbf{x},\mathbf{y}^*,\mathbf{z}^*,\mathbf{w}^*)-f_{\alpha }(\mathbf{x}^*,\mathbf{y}^*,\mathbf{z}^*,\mathbf{w}^*) \nonumber \\&\ge 0. \end{aligned}$$

(15)

Similarly, it follows from the $\mathbf{y}$-, $\mathbf{z}$-, and $\mathbf{w}$-subproblems of Algorithm 1 that

$$\begin{aligned} \left\{ \begin{aligned}&\langle \nabla _\mathbf{y}f_{\alpha }(\mathbf{t}^*),\mathbf{y}-\mathbf{y}^*\rangle \ge 0, \quad \forall \mathbf{y}\in {\mathbb {D}}_1,\\&\langle \nabla _\mathbf{z}f_{\alpha }(\mathbf{t}^*),\mathbf{z}-\mathbf{z}^*\rangle \ge 0,\quad \forall \mathbf{z}\in {\mathbb {D}}_2,\\&\langle \nabla _\mathbf{w}f_{\alpha }(\mathbf{t}^*),\mathbf{w}-\mathbf{w}^*\rangle \ge 0, \quad \forall \mathbf{w}\in {\mathbb {D}}_2. \end{aligned} \right. \end{aligned}$$

(16)

By invoking the fact that

$$\begin{aligned} \nabla f_{\alpha }(\mathbf{t}^*)=\left( \nabla _\mathbf{x}f_{\alpha }(\mathbf{t}^*),\nabla _\mathbf{y}f_{\alpha }(\mathbf{t}^*),\nabla _\mathbf{z}f_{\alpha }(\mathbf{t}^*),\nabla _\mathbf{w}f_{\alpha }(\mathbf{t}^*)\right) , \end{aligned}$$

it then follows from (15) and (16) that variational inequality (14) holds, which means that $\mathbf{t}^*$ is a stationary point of the augmented MOP (9). $\square$

B Proof of Theorem 5

Proof

We first prove Item (i). By the concavity of ${{\varvec{f}}}^2_{\alpha }(\mathbf{x},\mathbf{z})$, it holds for $k\in {\mathbb {N}}$ that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)})-{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}) \le \left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \mathbf{x}^{(k+1)}-\mathbf{x}^{(k)}\right\rangle . \end{aligned}$$

(17)

Clearly, if $\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})=0$, then, it follows from Algorithm 2 that $\mathbf{x}^{(k+1)}=\mathbf{x}^{(k)}$ and (17) holds; Otherwise, i.e.,

$$\begin{aligned} \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\ne 0. \end{aligned}$$

It follows that

$$\begin{aligned}&\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \mathbf{x}^{(k+1)}-\mathbf{x}^{(k)}\right\rangle \\&=\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), -\frac{\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})}{\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\Vert }-\mathbf{x}^{(k)}\right\rangle \\&= -\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\Vert -\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \mathbf{x}^{(k)}\right\rangle , \end{aligned}$$

which, together with Cauchy-Schwarz inequality, implies that

$$\begin{aligned} \left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \mathbf{x}^{(k+1)}-\mathbf{x}^{(k)}\right\rangle \le 0. \end{aligned}$$

Consequently, it follows from (17) that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)})\le {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}). \end{aligned}$$

(18)

Similarly, we have

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})\le {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)}). \end{aligned}$$

(19)

Combining (19) with (18) immediately leads to

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})\le {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}). \end{aligned}$$

On the other hand, if there exists a k such that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})= {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \end{aligned}$$

then, it follows from (18) and (19) that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})={{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)})={{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}). \end{aligned}$$

From (17) and the fact that $\Vert \mathbf{x}^{(k+1)}\Vert =\Vert \mathbf{x}^{(k)}\Vert =1$, we know that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)})<{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})~\Leftrightarrow ~\mathbf{x}^{(k+1)}\ne \mathbf{x}^{(k)}. \end{aligned}$$

Similarly, we can obtain that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})<{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k)})~\Leftrightarrow ~\mathbf{z}^{(k+1)}\ne \mathbf{z}^{(k)}, \end{aligned}$$

which means that $\{{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\}$ is strictly decreasing.

Now, we prove Item (ii). If Algorithm 2 terminates in finite number of steps, without loss of generality, suppose it stops at the k-th iteration. From the discussion of Item (i), it holds that

$$\begin{aligned} {{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})={{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}) \end{aligned}$$

and

$$\begin{aligned} \mathbf{x}^{(k+1)}=\mathbf{x}^{(k)},~\mathbf{z}^{(k+1)}=\mathbf{z}^{(k)}. \end{aligned}$$

Then, it follows from the iterative schemes of Algorithm 2 that

$$\begin{aligned} \left\{ \begin{aligned}&\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})=\mathbf{0}, \\&\nabla _{\mathbf{z}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})=\mathbf{0}. \end{aligned} \right. \end{aligned}$$

Thus, for any $\mathbf{x}\in {\mathbb {R}}^n, \mathbf{z}\in {\mathbb {R}}^m$ with $\Vert \mathbf{x}\Vert =1$ and $\Vert \mathbf{z}\Vert =1$, it holds that

$$\begin{aligned} \left\langle \mathbf{x}-\mathbf{x}^{(k)}, \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\right\rangle +\left\langle \mathbf{z}-\mathbf{z}^{(k)}, \nabla _{\mathbf{z}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\right\rangle =0, \end{aligned}$$

which implies that $(\mathbf{x}^{(k)}, \mathbf{z}^{(k)})$ is a stationary point of Bi-QOP (2).

Hereafter, we prove Item (iii). Due to the monotonicity of $\{{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\}$ and compactness of ${\mathbb {D}}_1$ and ${\mathbb {D}}_2$, we know that $\{{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\}$ is a convergent sequence. It then follows from (17) that

$$\begin{aligned} \lim _{k\rightarrow \infty }\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k)},\mathbf{z}^{(k)}), \mathbf{x}^{(k+1)}-\mathbf{x}^{(k)}\right\rangle =0. \end{aligned}$$

(20)

On the other hand, the boundedness of the sequence $\{(\mathbf{x}^{(k)}, \mathbf{z}^{(k)})\}$ implies that it has at least one accumulation point. Without loss of generality, assume $({\hat{\mathbf{x}}}, {\hat{\mathbf{z}}})$ is an accumulation point of the subsequence $\{(\mathbf{x}^{(k_j)}, \mathbf{z}^{(k_j)})\}$. Then we have

$$\begin{aligned} \Vert {\hat{\mathbf{x}}}\Vert =\Vert {\hat{\mathbf{z}}}\Vert =1. \end{aligned}$$

Now, if $\lim _{j\rightarrow \infty }\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)})=\mathbf{0}$, it then follows from (20) and the continuity of ${{\varvec{f}}}^2_{\alpha }(\mathbf{x},\mathbf{z})$ that

$$\begin{aligned} \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}})=\mathbf{0}, \end{aligned}$$

which means that

$$\begin{aligned} \mathcal {A}{\hat{\mathbf{x}}}{\hat{\mathbf{z}}}{\hat{\mathbf{z}}}=\alpha {\hat{\mathbf{x}}}. \end{aligned}$$

(21)

If $\lim _{j\rightarrow \infty }\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)})\ne \mathbf{0}$, then by Step 3 of Algorithm 2, it follows from (17) and (20) that

$$\begin{aligned}&\lim _{k\rightarrow \infty }\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)}), \mathbf{x}^{(k_j+1)}-\mathbf{x}^{(k_j)}\right\rangle \\&=\lim _{k\rightarrow \infty }\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)}), -\frac{\nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)})}{\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)})\Vert } -\mathbf{x}^{(k_j)}\right\rangle \\&=\lim _{k\rightarrow \infty }\left( -\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)})\Vert -\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }(\mathbf{x}^{(k_j)},\mathbf{z}^{(k_j)}),{\hat{\mathbf{x}}}\right\rangle \right) \\&=-\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}})\Vert -\left\langle \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}}),{\hat{\mathbf{x}}}\right\rangle \\&=0, \end{aligned}$$

which, together with $\Vert {\hat{\mathbf{x}}}\Vert =1$, implies that

$$\begin{aligned} \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}})=-\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}})\Vert {\hat{\mathbf{x}}}. \end{aligned}$$

Consequently, we have

$$\begin{aligned} \mathcal {A}{\hat{\mathbf{x}}}{\hat{\mathbf{z}}}{\hat{\mathbf{z}}}=\frac{1}{2}\left( -\Vert \nabla _{\mathbf{x}}{{\varvec{f}}}^2_{\alpha }({\hat{\mathbf{x}}},{\hat{\mathbf{z}}})\Vert +2\alpha \right) {\hat{\mathbf{x}}}. \end{aligned}$$

(22)

Combining (21) with (22) yields the ${\hat{\mathbf{x}}}$-part KKT system of (2). Similarly, we can obtain the ${\hat{\mathbf{z}}}$-part KKT system of the problem. Hence, $({\hat{\mathbf{x}}}, {\hat{\mathbf{z}}})$ is a KKT point of Bi-QOP (2).

Finally, we prove Item (iv). If ${{\varvec{f}}}^2_{\alpha }(\mathbf{x},\mathbf{z})$ is strictly concave for any fixed $\mathbf{x}\in {\mathbb {D}}_1$ and $\mathbf{z}\in {\mathbb {D}}_2$, it then follows from (20) that

$$\begin{aligned} \lim _{k\rightarrow \infty } \left( \Vert \mathbf{x}^{(k+1)}\Vert ^2-\langle \mathbf{x}^{(k+1)},\mathbf{x}^{(k)}\rangle \right) =0, \end{aligned}$$

which implies that

$$\begin{aligned} \lim _{k\rightarrow \infty }\langle \mathbf{x}^{(k+1)},\mathbf{x}^{(k)}\rangle =\lim _{k\rightarrow \infty } \Vert \mathbf{x}^{(k+1)}\Vert ^2=1. \end{aligned}$$

So, we conclude that

$$\begin{aligned} \lim _{k\rightarrow \infty }\Vert \mathbf{x}^{(k+1)}-\mathbf{x}^{(k)}\Vert =0. \end{aligned}$$

(23)

On the other hand, by a similar proof, one can prove that $\lim _{k\rightarrow \infty }\Vert \mathbf{z}^{(k+1)}-\mathbf{z}^{(k)}\Vert =0$, which, together with (23), implies that

$$\begin{aligned} \lim _{k\rightarrow \infty }\Vert (\mathbf{x}^{(k+1)},\mathbf{z}^{(k+1)})-(\mathbf{x}^{(k)},\mathbf{z}^{(k)})\Vert =0. \end{aligned}$$

The proof is completed. $\square$