Most of the existing deep subspace clustering methods leverage convolutional autoencoders to obtain feature representation for non-linear data points. These methods commonly adopt the structure of a few convolutional layers because stacking many convolutional layers may cause computationally inefficient and optimization difficulties. However, long-range dependencies can hardly be captured when convolutional operations are not repeated enough, thus affect the quality of feature extraction which the performance of deep subspace clustering method highly lies in. To deal with this issue, we propose a novel self-attention deep subspace clustering (SADSC) model, which learns more favorable data representations by introducing self-attention mechanisms into convolutional autoencoders. Specifically, SADSC leverages three convolutional layers and add the self-attention layers after the first and third ones in encoders, then decoders have symmetric structures. The self-attention layers maintain the variable input sizes and can be easily combined with different convolutional layers in autoencoder. Experimental results on the handwritten recognition, face and object clustering datasets demonstrate the advantages of SADSC over the state-of-the-art deep subspace clustering models.

大多数现有的深度子空间聚类方法都利用卷积自动编码器来获取非线性数据点的特征表示。这些方法通常采用一些卷积层的结构，因为堆叠许多卷积层可能会导致计算效率低下和优化困难。然而，当卷积运算不充分重复时，就很难捕获到远程依存关系，从而影响深度子空间聚类方法的性能所高度依赖的特征提取质量。针对此问题，我们提出了一种新颖的自我注意深度子空间聚类（SADSC）模型，该模型通过将自注意力机制引入卷积自动编码器中来学习更有利的数据表示形式。具体来说，SADSC利用三个卷积层，并在编码器中的第一层和第三层之后添加自注意层，然后，解码器具有对称结构。自我注意层保持可变的输入大小，并且可以轻松地与自动编码器中的不同卷积层组合。手写识别，面部和对象聚类数据集的实验结果证明了SADSC相对于最新的深度子空间聚类模型的优势。