Off-diagonal symmetric nonnegative matrix factorization

Abstract

Symmetric nonnegative matrix factorization (symNMF) is a variant of nonnegative matrix factorization (NMF) that allows handling symmetric input matrices and has been shown to be particularly well suited for clustering tasks. In this paper, we present a new model, dubbed off-diagonal symNMF (ODsymNMF), that does not take into account the diagonal entries of the input matrix in the objective function. ODsymNMF has three key advantages compared to symNMF. First, ODsymNMF is theoretically much more sound as there always exists an exact factorization of size at most n(n − 1)/2 where n is the dimension of the input matrix. Second, it makes more sense in practice as diagonal entries of the input matrix typically correspond to the similarity between an item and itself, not bringing much information. Third, it makes the optimization problem much easier to solve. In particular, it will allow us to design an algorithm based on coordinate descent that minimizes the component-wise ℓ₁ norm between the input matrix and its approximation. We prove that this norm is much better suited for binary input matrices often encountered in practice. We also derive a coordinate descent method for the component-wise ℓ₂ norm, and compare the two approaches with symNMF on synthetic and document datasets.

Appendix: The constrained weighted median problem

Algorithm 7 provides a pseudocode to compute the solution to the constrained weighted median problem:

$$ \min_{x \geq 0} {\sum}_{i} |a_{i} x - b_{i}|. $$

The algorithm works as follows:

• The set S of breakpoints $\frac {b_{i}}{a_{i}}$ is initialized for all i = 1,...,n such that a_i≠ 0 (because when a_i = 0, the contribution of the i th term in the objective function is a constant) and the vector a is then sorted and normalized according to the values in S,

• As the values a_i correspond to the slopes, the second step of the algorithm looks for the k th breakpoint for which we have ${\sum }_{i=1}^{k-1} a_{i} < {\sum }_{i=k}^{n} a_{i}$ and ${\sum }_{i=1}^{k} a_{i} \geq {\sum }_{i=k+1}^{n} a_{i}$. It corresponds to a global optimum since the slope on the left is negative, and on the right is nonnegative.