We propose an unsupervised learning approach using a convolutional and fully connected autoencoder, which we call deep heterogeneous autoencoder, to learn discriminative features from segmentation masks and detection bounding boxes. To learn the mask shape information and its corresponding location in an input image, we extract coarse masks from a pretrained semantic segmentation network as well as their corresponding bounding boxes. We train the autoencoders jointly using task-dependent uncertainty weights to generate common latent features. The feature vector is then fed to the k-means clustering algorithm to separate the data points in the latent space. Finally, we incorporate additional penalties in the form of a constraints graph based on prior knowledge of the sequential data to increase clustering robustness. We evaluate the performance of our method using both synthetic and real world multi-object video datasets to demonstrate the applicability of our proposed model. Our results show that the proposed technique outperforms several state-of-the-art methods on challenging video sequences.

中文翻译：

---

用于序列数据子空间聚类的深度异构自编码器

我们提出了一种使用卷积和完全连接的自编码器（我们称之为深度异构自编码器）的无监督学习方法，以从分割掩码和检测边界框学习判别特征。为了学习掩码形状信息及其在输入图像中的相应位置，我们从预训练的语义分割网络及其相应的边界框中提取粗略掩码。我们使用依赖于任务的不确定性权重联合训练自动编码器以生成共同的潜在特征。然后将特征向量馈送到 k-means 聚类算法以分离潜在空间中的数据点。最后，我们基于序列数据的先验知识以约束图的形式加入额外的惩罚，以提高聚类的鲁棒性。我们使用合成和真实世界的多对象视频数据集来评估我们的方法的性能，以证明我们提出的模型的适用性。我们的结果表明，所提出的技术在具有挑战性的视频序列上优于几种最先进的方法。