Exploring nonnegative and low-rank correlation for noise-resistant spectral clustering

Abstract

Clustering has been extensively explored in pattern recognition and data mining in order to facilitate various applications. Due to the presence of data noise, traditional clustering approaches may become vulnerable and unreliable, thereby degrading clustering performance. In this paper, we propose a robust spectral clustering approach, termed Non-negative Low-rank Self-reconstruction (NLS), which simultaneously a) explores the nonnegative low-rank properties of data correlation as well as b) adaptively models the structural sparsity of data noise. Specifically, in order to discover the intrinsic correlation among data, we devise a self-reconstruction approach to jointly consider the nonnegativity and low-rank property of data correlation matrix. Meanwhile, we propose to model data noise via a structural norm, i.e., ℓ_p,2-norm, which not only naturally conforms to genuine patterns of data noise in real-world situations, but also provides more adaptivity and flexibility to different noise levels. Extensive experiments on various real-world datasets illustrate the advantage of the proposed robust spectral clustering approach compared to existing clustering methods.

Appendices

Appendix A: Proof of Lemma 1

Proof

Inspired by [34], we consider the following function

$$ f(a) = p a^{2} - 2 a^{p} + (2 - p), $$

(28)

where p ∈ (0, 2). We expect to show that when a > 0, f(a) ≥ 0. The first and second order derivatives of the function in (28) are $f^{\prime }(a ) = 2pa - 2p{a^{p - 1}}$ and $f^{\prime \prime }(a) = 2p - 2p(p - 1){a^{p - 2}}$, respectively. We can see that a = 1 is the only point that satisfies $f^{\prime }(a)=0$. Also, when 0 < a < 1, $f^{\prime }(a )<0$ and when a > 1, $f^{\prime }(a )>0$. This means that f(a) is monotonically decreasing when 0 < a < 1 and monotonically increasing when a > 1. Moreover, we have $f^{\prime \prime }(1)=2p(2-p)>0$. Therefore, for ∀a > 0, f(a) ≥ f(1) = 0.

Then, by substituting $a = \frac {\| \tilde {\varepsilon }_{i} \|}{\|\varepsilon _{i}\|}$ into (28), we obtain the conclusion

$$ \begin{array}{@{}rcl@{}} && p\frac{\| \tilde{\varepsilon}_{i} \|^{2}}{\| \varepsilon_{i} \|^{2}} - 2 \frac{\| \tilde{\varepsilon}_{i} \|^{p}}{\| \varepsilon_{i} \|^{p}} + (2 - p) \ge 0,\\ &\Leftrightarrow& p\| \tilde{\varepsilon}_{i} \|^{2} - 2\| \tilde{\varepsilon}_{i} \|^{p}\| \varepsilon_{i} \|^{2 - p} + (2 - p)\| \varepsilon_{i} \|^{2} \ge 0,\\ &\Leftrightarrow& p\| \tilde{\varepsilon}_{i} \|^{2}\| \varepsilon_{i} \|^{p - 2} - 2\| \tilde{\varepsilon}_{i} \|^{p} + (2 - p)\| \varepsilon_{i} \|^{p} \ge 0,\\ &\Leftrightarrow& 2\| \tilde{\varepsilon}_{i} \|^{p} - p\| \tilde{\varepsilon}_{i} \|^{2}\| \varepsilon_{i} \|^{p - 2} \le (2 - p)\| \varepsilon_{i} \|^{p},\\ &\Leftrightarrow& \| \tilde{\varepsilon}_{i} \|^{p} - \frac{p\| \tilde{\varepsilon}_{i} \|^{2}}{2\| \varepsilon_{i} \|^{2-p}} \le \| \varepsilon_{i} \|^{p} - \frac{p\| \varepsilon_{i} \|^{p}}{2\| \varepsilon_{i} \|^{2 - p}}. \end{array} $$

□

Appendix B: Proof of Theorem 1

Proof

Denote ${\mathcal{L}}(\mathcal {E}) = \frac {1}{2} \left \| \mathcal {E} - \left (X - XW + \frac {P}{\alpha }\right ) \right \|_{F}^{2}$ and $\lambda ^{\prime } = \lambda /\alpha$. Suppose $\tilde {\mathcal {E}}$ is the optimized solution of the alternative problem (18), then we obtain the following conclusion:

$$ \begin{array}{@{}rcl@{}} && \mathcal{L}(\tilde{\mathcal{E}}) + \lambda^{\prime} Tr(\tilde{\mathcal{E}}^{T} Z \tilde{\mathcal{E}}) \le \mathcal{L} (\mathcal{E}) + \lambda^{\prime} Tr(\mathcal{E}^{T} Z \mathcal{E})\\ &\Rightarrow& \mathcal{L} (\tilde{\mathcal{E}}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \frac{p\| \tilde{\varepsilon}_{i} \|^{2}}{2\| \varepsilon_{i} \|^{2 - p}} \le \mathcal{L} (\mathcal{E}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \frac{p\| \varepsilon_{i} \|^{2}}{2\| \varepsilon_{i} \|^{2 - p}} \\ &\Rightarrow& \mathcal{L} (\tilde{\mathcal{E}}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \| \tilde{\varepsilon}_{i} \|_{p}^{2} - \lambda^{\prime} \left( \sum\limits_{i = 1}^{n} \| \tilde{\varepsilon}_{i} \|_{p}^{2} - \sum\limits_{i = 1}^{n} \frac{p\| \tilde{\varepsilon}_{i} \|^{2}}{2\| \varepsilon_{i} \|^{2 - p}} \right)\\ &\le& \mathcal{L} (\mathcal{E}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \| \varepsilon_{i} \|_{p}^{2} - \lambda^{\prime} \left( \sum\limits_{i = 1}^{n} \| \varepsilon_{i} \|_{p}^{2} - \sum\limits_{i = 1}^{n} \frac{p\| \varepsilon_{i} \|^{2}}{2\| \varepsilon_{i} \|^{2 - p}}\right). \end{array} $$

Given the conclusion of Lemma 2, we finally arrive at

$$ \mathcal{L} (\tilde{\mathcal{E}}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \| \tilde{\varepsilon}_{i} \|_{p}^{2} \le \mathcal{L} (\mathcal{E}) + \lambda^{\prime} \sum\limits_{i = 1}^{n} \| \varepsilon_{i} \|_{p}^{2}. $$

Hence, the value of the objective function in (17) monotonically decreases in each iteration. □