K-means—and the celebrated Lloyd’s algorithm—is more than the clustering method it was originally designed to be. It has indeed proven pivotal to help increase the speed of many machine learning, data analysis techniques such as indexing, nearest-neighbor search and prediction, data compression and, lately, inference with kernel machines. Here, we introduce an efficient extension of K-means, dubbed QuicK-means, that rests on the idea of expressing the matrix of the \(K\) cluster centroids as a product of sparse matrices, a feat made possible by recent results devoted to find approximations of matrices as a product of sparse factors. Using such a decomposition squashes the complexity of the matrix-vector product between the factorized \(K\times D\) centroid matrix \({\mathbf {U}}\) and any vector from \({\mathcal {O}}\left( KD\right)\) to \({\mathcal {O}}\left( A \log B~ +B\right)\), with \(A=\min \left( K,D\right)\) and \(B=\max \left( K,D\right)\), where \(D\) is the dimension of the data. This drastic computational saving has a direct impact in the assignment process of a point to a cluster. We propose to learn such a factorization during the Lloyd’s training procedure. We show that resorting to a factorization step at each iteration does not impair the convergence of the optimization scheme, and demonstrate the benefits of our approach experimentally.

K-means和著名的Lloyd算法比最初设计的聚类方法还多。事实证明，它确实至关重要，可以帮助提高许多机器学习，索引等数据分析技术，近邻搜索和预测，数据压缩以及最近对内核机器进行推理的速度。在这里，我们介绍了称为QuicK-means的K-means的有效扩展，它基于将\（K \）形心形矩阵表示为稀疏矩阵的产物的思想，这是最近专门研究的结果使得这一壮举成为可能。查找作为稀疏因子乘积的矩阵的近似值。使用这样的分解压扁了因子分解之间的矩阵矢量积的复杂性\（K \ times D \）重心矩阵\（{\ mathbf {U}} \）和从\（{\ mathcal {O}} \ left（KD \ right）\）到\（{\ mathcal { O}} \ left（A \ log B〜+ B \ right）\），其中\（A = \ min \ left（K，D \ right）\）和\（B = \ max \ left（K，D \ right）\），其中\（D \）是数据的维数。这种大量的计算节省直接影响了将点分配给群集的过程。我们建议在劳埃德（Lloyd）的培训过程中学习这种因式分解。我们证明了在每次迭代中采用分解步骤不会损害优化方案的收敛性，并通过实验证明了我们方法的好处。