Robust mixed-norm constrained regression with application to face recognitions

Abstract

Most existing regression-based classification methods cope with pixelwise noise via $\ell _1$-norm or $\ell _2$-norm, but neglect the structural information between pixels. To the best of our knowledge, nuclear norm-based matrix regression approaches have achieved great success for addressing imagewise noise, but may result in unreasonable regression and incorrect classification, especially when test images are extremely corrupted by larger occlusions and severe illumination variations, since they apply the corrupted test images to reconstruction process directly, and the influence of noise will be unavoidable. To overcome this limitation, this paper presents a robust mixed-norm constrained regression model to deal with the structural noise corruption. To be more specific, nuclear norm of the error between corrupted test image and its corresponding recovered image is exploited as a regular term for characterizing the low rank noise structure, and Frobenius norm is utilized to depict the difference between the recovered image and restructured image on account of the less noise of recovered image. Then, we adopt the alternating direction method of multipliers to settle our proposed approaches efficiently. Furthermore, the theoretical convergence proof and detailed analysis of computational complexity are provided to assess our algorithms. Eventually, extensive experiments on five well-known face databases have manifested that the proposed methods outperform some state-of-the-art regression-based approaches for primarily addressing noise caused by occlusion and illumination changes.

Appendix A Proof of Theorem 3

Proof

In view of the fact that $({{\mathbf{x}}^{*}},{\widetilde{\mathbf{H}}}^{*} ,{\mathbf{T}}^{*},{{\mathbf{Z}}}^{*})$ is a saddle point of L, we have $L({{\mathbf{x}}^{*}},{\widetilde{\mathbf{H}}}^{*},{\mathbf{T}}^{*},{{\mathbf{Z}}}^{*}) \le L({{\mathbf{x}}_{k+1}},{\widetilde{\mathbf{H}}}_{k+1} ,{\mathbf{T}}_{k+1},{{\mathbf{Z}}}^{*})$, with ${{D}}({{\mathbf{x}}^{*}})-{\widetilde{\mathbf{H}}}^{*} - {\mathbf{T}}^{*} = 0$, and it can be rewritten as follows:

$$\begin{aligned} f^{*} - f_{k+1} \le \text {Tr}(({{\mathbf{Z}}}^{*})^{T}{\mathbf{R}}_{k+1}). \end{aligned}$$

(26)

For the sake of derivation, the augmented Lagrangian function of Eq. (5) can be reformulated as ${L_\mu }({\mathbf{x}},{\widetilde{\mathbf{H}}} ,{\mathbf{T}},{{\mathbf{Z}}})= {\Vert {\mathbf{T}} \Vert _F^2} + \lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}} \Vert _{*}} +\frac{\beta }{2} \left\| {\mathbf{x}} \right\| _2^{2} +\frac{\mu }{2} \Vert {{D}({\mathbf{x}})-{\widetilde{\mathbf{H}}}} - {\mathbf{T}} +\frac{1}{\mu } {{\mathbf{Z}}}\Vert _F^2-\frac{1 }{2\mu }\Vert {{\mathbf{Z}}}\Vert _F^2$.

${\mathbf{x}}_{k+1} = \arg \mathop {\min }\limits _{{\mathbf{x}}} {L_\mu }({\mathbf{x}},{\widetilde{\mathbf{H}}}_{k},{\mathbf{T}}_{k},{\mathbf{Z}}_{k})$, which is equivalent to

$$\begin{aligned} 0 &= \partial { {L_\mu }({\mathbf{x}}_{k+1},{\widetilde{\mathbf{H}}}_{k},{\mathbf{T}}_{k},{\mathbf{Z}}_{k})}\\&= \frac{\beta }{2}\partial {(\Vert {\mathbf{x}}_{k+1}\Vert _p^p)}+\mu {\mathbf{M}}^{T}( {\mathbf{M}}{\mathbf{x}}_{k+1} -\text {Vec} \left( {\widetilde{\mathbf{H}}}_{k}+ {\mathbf{T}}_{k}-\frac{1}{\mu }{{\mathbf{Z}}}_{k}\right) , \end{aligned}$$

by virtue of ${{\mathbf{Z}}}_{k+1} ={\mathbf{Z}}_{k} +\mu ({D}({\mathbf{x}}_{k+1})-{\widetilde{\mathbf{H}}}_{k+1} - {\mathbf{T}}_{k+1})$, we can recombine to get

$$\begin{aligned} 0&= \frac{\beta }{2}\partial {(\Vert {\mathbf{x}}_{k+1}\Vert _p^p)} + {\mathbf{M}}^{T}(\text {Vec} ({\mathbf{Z}}_{k+1}) +\mu \text {Vec}({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\,{\widetilde{\mathbf{H}}}_{k})+\mu \text {Vec}({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k})). \end{aligned}$$

This suggests that ${\mathbf{x}}_{k+1}$ optimizes $\frac{\beta }{2}\Vert {\mathbf{x}}\Vert _p^p + (\text {Vec} ({\mathbf{Z}}_{k+1}) +\mu \text {Vec}({\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}_{k})+ \mu \text {Vec}({\mathbf{T}}_{k+1} -{\mathbf{T}}_{k}))^{T}{\mathbf{M}}{\mathbf{x}}$. Similar to the above derivation, we can get the arguments that ${\widetilde{\mathbf{H}}}_{k+1}$ minimizes $\lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}} \Vert _{*}} -\text {Tr}(({\mathbf{Z}}_{k+1} + \mu ({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}))^{T}{\widetilde{\mathbf{H}}})$, and ${\mathbf{T}}_{k+1}$ minimizes $\Vert {\mathbf{T}}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}})$. Hence, we have

$$\begin{aligned} \begin{aligned}&(1) \quad \frac{\beta }{2}\Vert {\mathbf{x}}_{k+1}\Vert _p^p + (\text {Vec} ({\mathbf{Z}}_{k+1}) +\mu \text {Vec}({\widetilde{\mathbf{H}}}_{k+1} -{\widetilde{\mathbf{H}}}_{k})\\&\qquad \quad +\,\mu \text {Vec}({\mathbf{T}}_{k+1} -{\mathbf{T}}_{k}))^{T}{\mathbf{M}}{\mathbf{x}}_{k+1}\\&\qquad \le \beta \Vert {\mathbf{x}}^{*}\Vert _p^p + (\text {Vec} ({\mathbf{Z}}_{k+1}) +\mu \text {Vec}({\widetilde{\mathbf{H}}}_{k+1} -{\widetilde{\mathbf{H}}}_{k})\\&\qquad \quad +\,\mu \text {Vec}({\mathbf{T}}_{k+1} -{\mathbf{T}}_{k}))^{T}{\mathbf{M}}{\mathbf{x}}^{*}\\&(2) \quad \lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}}_{k+1} \Vert _{*}} -\text {Tr}(({\mathbf{Z}}_{k+1} \\&\qquad \quad +\, \mu ({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}))^{T}{\widetilde{\mathbf{H}}}_{k+1}) \\&\qquad \le \lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}}^{*} \Vert _{*}} -\text {Tr}(({\mathbf{Z}}_{k+1}\\&\qquad \quad +\, \mu ({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}))^{T}{\widetilde{\mathbf{H}}}^{*})\\&(3) \quad \Vert {\mathbf{T}}_{k+1}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}}_{k+1}) \\&\qquad \le \Vert {\mathbf{T}}^{*}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}}^{*}) \end{aligned} \end{aligned}$$

Adding these three inequalities, with ${{D}}({{\mathbf{x}}^{*}})-{\widetilde{\mathbf{H}}}^{*} - {\mathbf{T}}^{*} = 0$ and ${\mathbf{R}}_{k+1} = {D}({\mathbf{x}}_{k+1}) -{\widetilde{\mathbf{H}}}_{k+1}- {\mathbf{T}}_{k+1}$, and after regrouping, we obtain

$$\begin{aligned} \begin{aligned} f_{k+1} - f^{*}&\le - \text {Vec}^{T} ({\mathbf{Z}}_{k+1})\mathbf{{r}}_{k+1} \\&\quad -\, \mu \text {Vec}^{T} ({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})(\mathbf{{r}}_{k+1} + \text {Vec} ({\mathbf{T}}_{k+1}- {\mathbf{T}}^{*} ))\\&\quad -\,\mu \text {Vec}^{T}({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})(\mathbf{{r}}_{k+1}+ \text {Vec} (({\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}^{*})\\&\quad +\,({\mathbf{T}}_{k+1}- {\mathbf{T}}^{*} )) \end{aligned} \end{aligned}$$

(27)

Adding (26) and (27), and rearranging, we can obtain this inequality:

$$\begin{aligned} \begin{aligned}&\text {Vec}^{T} ({\mathbf{Z}}_{k+1}\\&\quad -\,{\mathbf{Z}}^{*})\mathbf{{r}}_{k+1} +\mu \text {Vec}^{T} ({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})\mathbf{{r}_{k+1}}\\&\quad +\,\mu \text {Vec}^{T} ({\mathbf{T}}_{k+1}\\&\quad -\, {\mathbf{T}}_{k})\text {Vec} ({\mathbf{T}}_{k+1} - {\mathbf{T}}^{*} )\\&\quad +\, \mu \text {Vec}^{T}({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\, {\widetilde{\mathbf{H}}}_{k})\text {Vec}({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\,{\widetilde{\mathbf{H}}}^{*})\\&\quad +\, \mu \text {Vec}^{T}({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})(\mathbf{{r}}_{k+1}\\&\quad +\, \text {Vec}({\mathbf{T}}_{k+1}- {\mathbf{T}}^{*} ))\\&\quad \le 0 \end{aligned} \end{aligned}$$

Substituting ${\mathbf{T}}_{k+1}-{\mathbf{T}}^{*}={\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}+{\mathbf{T}}_{k}-{\mathbf{T}}^{*}$ in the fourth term and utilizing $\text {Vec}^{T}(U)\text {Vec}(V)={\text {Tr}}(U^{T}V)$, where $\forall ~U, V \in {R}^{s \times t}$, the aforementioned inequality can be rewritten as

$$\begin{aligned} \begin{aligned}&-\text {Tr}(({\mathbf{Z}}_{k+1}-{\mathbf{Z}}^{*})^{T}{\mathbf{R}}_{k+1})-\mu \text {Tr} (({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})^{T}({\mathbf{T}}_{k+1}- {\mathbf{T}}^{*} ))\\&\quad -\, \mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})^{T}({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\,{\widetilde{\mathbf{H}}}^{*}))+\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})^{T}({\mathbf{T}}^{*}-{\mathbf{T}}_{k}) )\\& \ge \mu \text {Tr} (({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})^{T}{} \mathbf{{R}_{k+1}})+\mu \text {Tr} (({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\, {\widetilde{\mathbf{H}}}_{k})^{T}({\mathbf{R}}_{k+1}+({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})) \end{aligned} \end{aligned}$$

(28)

Subsequently, it can be proved that the two items on the right side in (28) are both greater than 0. To see this, ${\mathbf{T}}_{k+1}$ minimizes $\Vert {\mathbf{T}}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}})$, then ${\mathbf{T}}_{k}$ minimizes $\Vert {\mathbf{T}}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k})^{T}{\mathbf{T}})$, which are equivalent to

$$\begin{aligned} \begin{aligned}&\Vert {\mathbf{T}}_{k+1}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}}_{k+1})\\&\quad \le \Vert {\mathbf{T}}_{k}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k+1})^{T}{\mathbf{T}}_{k})\\&\quad \Vert {\mathbf{T}}_{k}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k})^{T}{\mathbf{T}}_{k})\\&\quad \le \Vert {\mathbf{T}}_{k+1}\Vert _F^2 - \text {Tr}(({\mathbf{Z}}_{k})^{T}{\mathbf{T}}_{k+1}) \end{aligned} \end{aligned}$$

Adding the two inequalities above, taking advantage of ${{\mathbf{Z}}}_{k+1} ={\mathbf{Z}}_{k} +\mu {\mathbf{R}}_{k+1}$ and reorganizing, we get $\mu \text {Tr} (({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})^{T}{} \mathbf{{R}_{k+1}}) \ge 0$. Similarly, using the argument that ${\widetilde{\mathbf{H}}}_{k+1}$ minimizes $\lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}} \Vert _{*}} -\text {Tr}(({\mathbf{Z}}_{k+1} + \mu ({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}))^{T}{\widetilde{\mathbf{H}}})$, we obtain $\mu \text {Tr} (({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})^{T}({\mathbf{R}}_{k+1}+({\mathbf{T}}_{k+1}-{\mathbf{T}}_{k})) \ge 0$.

In addition, since ${\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}_{k} ={\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}^{*}+{\widetilde{\mathbf{H}}}^{*}+{\widetilde{\mathbf{H}}}_{k}$, then

$$\begin{aligned} \begin{aligned}&\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}_{k})^{T}({\mathbf{T}}^{*}-{\mathbf{T}}_{k}) )\\&\quad =\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}^{*}-{\mathbf{T}}_{k}) )\\&\qquad +\,\mu \text {Tr}(({\widetilde{\mathbf{H}}}^{*}-{\widetilde{\mathbf{H}}}_{k})^{T}({\mathbf{T}}^{*}-{\mathbf{T}}_{k}))\\&\qquad \le \mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k}-{\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}_{k}\\&\qquad -\,{\mathbf{T}}^{*}))-\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}_{k+1}-{\mathbf{T}}^{*}) ) \end{aligned} \end{aligned}$$

With the previous step and multiplying through by 2, we can transform (28) into the following form

$$\begin{aligned} \begin{aligned}&-2\text {Tr}(({\mathbf{Z}}_{k+1}-{\mathbf{Z}}^{*})^{T}{\mathbf{R}}_{k+1}) \\&\quad -\,2\mu \text {Tr} (({\mathbf{T}}_{k+1}- {\mathbf{T}}_{k})^{T}({\mathbf{T}}_{k+1}\\&\quad -\, {\mathbf{T}}^{*} ))\\&\quad -\, 2\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1} - {\widetilde{\mathbf{H}}}_{k})^{T}({\widetilde{\mathbf{H}}}_{k+1}\\&\quad -\,{\widetilde{\mathbf{H}}}^{*}))+2\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k}\\&\quad -\,{\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}_{k}-{\mathbf{T}}^{*}))\\&\quad -\,2\mu \text {Tr}(({\widetilde{\mathbf{H}}}_{k+1}- {\widetilde{\mathbf{H}}}^{*})^{T} ({\mathbf{T}}_{k+1}-{\mathbf{T}}^{*}) )\\&\ge 0 \end{aligned} \end{aligned}$$

(29)

Using ${{\mathbf{Z}}}_{k+1} ={\mathbf{Z}}_{k} +\mu {\mathbf{R}}_{k+1}$ and perfect square expression, (29) can be rewritten as

$$\begin{aligned} \begin{aligned}&\left[ \frac{1}{\mu }\Vert {\mathbf{Z}}_{k}-{\mathbf{Z}}^{*}\Vert _F^2+\mu \Vert {\widetilde{\mathbf{H}}}_{k}\right. \\&\left. \quad +\,{\mathbf{T}}_{k}-{\widetilde{\mathbf{H}}}^{*}-{\mathbf{T}}^{*}\Vert _F^2\right] \\&\quad -\,\left[ \frac{1}{\mu }\Vert {\mathbf{Z}}_{k+1}-{\mathbf{Z}}^{*}\Vert _F^2\right. \\&\left. \quad +\,\mu \Vert {\widetilde{\mathbf{H}}}_{k+1}+{\mathbf{T}}_{k+1}-{\widetilde{\mathbf{H}}}^{*} -{\mathbf{T}}^{*}\Vert _F^2\right] \\&\qquad \ge \mu \Vert {\mathbf{R}}_{k+1}\Vert _F^2 +\mu \Vert {\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}_{k}\Vert _F^2 + \mu \Vert {\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}\Vert _F^2 \end{aligned} \end{aligned}$$

(30)

Let $V_{k} =\frac{1}{\mu }\Vert {\mathbf{Z}}_{k}-{\mathbf{Z}}^{*}\Vert _F^2+\mu \Vert {\widetilde{\mathbf{H}}}_{k}+{\mathbf{T}}_{k}-{\widetilde{\mathbf{H}}}^{*}-{\mathbf{T}}^{*}\Vert _F^2$ and the inequality (30) can be abbreviated as

$$\begin{aligned}&V_{k} -V_{k+1} \ge \mu \Vert {\mathbf{R}}_{k+1}\Vert _F^2 +\mu \Vert {\widetilde{\mathbf{H}}}_{k+1}\nonumber \\&\quad -\,{\widetilde{\mathbf{H}}}_{k}\Vert _F^2 + \mu \Vert {\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}\Vert _F^2 \end{aligned}$$

(31)

This indicates that $V_{k}$ decreases in each iteration since $\mu >0$, i.e., $V_{k+1} \le V_{k}\le V_{0}$. We iterate the inequality (31) and can acquire that

$$\begin{aligned}&\mu \sum \limits _{k=0}^{\infty } (\Vert {\mathbf{R}}_{k+1}\Vert _F^2+\Vert {\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}_{k}\Vert _F^2\\&\quad +\,\Vert {\mathbf{T}}_{k+1}-{\mathbf{T}}_{k}\Vert _F^2)\le V_{0}, \end{aligned}$$

which suggests that ${\mathbf{R}}_{k}\rightarrow 0$, ${\widetilde{\mathbf{H}}}_{k+1}-{\widetilde{\mathbf{H}}}_{k} \rightarrow 0$ and ${\mathbf{T}}_{k+1}-{\mathbf{T}}_{k} \rightarrow 0$ as $k \rightarrow \infty$ by the monotone bounded theorem. Thus, the right side in (26) and (27) both go to zero as $k \rightarrow \infty$. Further, we can get that $\lim \nolimits _{k \rightarrow \infty } f_{k} =f^{*}$.

That is, $\lim \nolimits _{k \rightarrow \infty }({\Vert {\mathbf{T}}_{k} \Vert _F^2} + \lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}}_{k} \Vert _{*}} +\beta /2\left\| {\mathbf{x}}_{k} \right\| _p^p) = {\Vert {\mathbf{T}}^{*} \Vert _F^2} + \lambda {\Vert {\mathbf{H}} - {\widetilde{\mathbf{H}}}^{*} \Vert _{*}} +\beta /2 \left\| {\mathbf{x}}^{*} \right\| _p^p$, which implies that $({\mathbf{x}}_{k},{\widetilde{\mathbf{H}}}_{k},{\mathbf{T}}_{k})$ is close to $({\mathbf{x}}^{*},{\widetilde{\mathbf{H}}}^{*}, {\mathbf{T}}^{*})$ as $k \rightarrow \infty$. By previous analysis and ${\mathbf{Z}}_{k}\rightarrow {{\mathbf{Z}}}^{*}$, we have $\lim \nolimits _{k \rightarrow \infty } {V_{k}} =\lim \nolimits _{k \rightarrow \infty }(\frac{1}{\mu }\Vert {\mathbf{Z}}_{k}-{\mathbf{Z}}^{*}\Vert _F^2+\mu \Vert {\widetilde{\mathbf{H}}}_{k}+{\mathbf{T}}_{k}-{\widetilde{\mathbf{H}}}^{*}-{\mathbf{T}}^{*}\Vert _F^2) = 0$. Thus, ${\widetilde{\mathbf{H}}}_{k}+{\mathbf{T}}_{k}-{\widetilde{\mathbf{H}}}^{*}-{\mathbf{T}}^{*} \rightarrow 0$. Evidently, $\text {Tr}(({\widetilde{\mathbf{H}}}_{k}-{\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}_{k}-{\mathbf{T}}^{*}) \ge 0$, then

$$\begin{aligned} \begin{aligned}&0 \le \Vert {\widetilde{\mathbf{H}}}_{k}-{\widetilde{\mathbf{H}}}^{*}\Vert _F^2\\&\quad +\,\Vert {\mathbf{T}}_{k}-{\mathbf{T}}^{*}\Vert _F^2\\&\quad \le \Vert {\widetilde{\mathbf{H}}}_{k}-{\widetilde{\mathbf{H}}}^{*}\Vert _F^2\\&\quad +\,\Vert {\mathbf{T}}_{k}-{\mathbf{T}}^{*}\Vert _F^2+2\text {Tr}(({\widetilde{\mathbf{H}}}_{k}\\&\quad -\,{\widetilde{\mathbf{H}}}^{*})^{T}({\mathbf{T}}_{k}-{\mathbf{T}}^{*}) \\&\quad = \Vert {\widetilde{\mathbf{H}}}_{k}+{\mathbf{T}}_{k}-{\widetilde{\mathbf{H}}}^{*}-{\mathbf{T}}^{*}\Vert _F^2 \rightarrow 0. \end{aligned} \end{aligned}$$

Therefore, ${\widetilde{\mathbf{H}}}_{k}\rightarrow {\widetilde{\mathbf{H}}}^{*}$ and ${\mathbf{T}}_{k}\rightarrow {\mathbf{T}}^{*}$ by squeeze rule. In virtue of the formula $\lim \nolimits _{k \rightarrow \infty } f_{k} =f^{*}$, we can derive ${\mathbf{x}}_{k}\rightarrow {\mathbf{x}}^{*}$. $\square$