In deep clustering frameworks, autoencoder (AE)- or variational AE-based clustering approaches are the most popular and competitive ones that encourage the model to obtain suitable representations and avoid the tendency for degenerate solutions simultaneously. However, for the clustering task, the decoder for reconstructing the original input is usually useless when the model is finished training. The encoder–decoder architecture limits the depth of the encoder so that the learning capacity is reduced severely. In this article, we propose a decoder-free variational deep embedding for unsupervised clustering (DFVC). It is well known that minimizing reconstruction error amounts to maximizing a lower bound on the mutual information (MI) between the input and its representation. That provides a theoretical guarantee for us to discard the bloated decoder. Inspired by contrastive self-supervised learning, we can directly calculate or estimate the MI of the continuous variables. Specifically, we investigate unsupervised representation learning by simultaneously considering the MI estimation of continuous representations and the MI computation of categorical representations. By introducing the data augmentation technique, we incorporate the original input, the augmented input, and their high-level representations into the MI estimation framework to learn more discriminative representations. Instead of matching to a simple standard normal distribution adversarially, we use end-to-end learning to constrain the latent space to be cluster-friendly by applying the Gaussian mixture distribution as the prior. Extensive experiments on challenging data sets show that our model achieves higher performance over a wide range of state-of-the-art clustering approaches.

中文翻译：

---

用于无监督聚类的无解码器变分深度嵌入


在深度聚类框架中，自动编码器（AE）或基于变分AE的聚类方法是最流行和最具竞争力的方法，它鼓励模型获得合适的表示并同时避免退化解决方案的趋势。然而，对于聚类任​​务，当模型完成训练时，用于重建原始输入的解码器通常是无用的。编码器-解码器架构限制了编码器的深度，从而严重降低了学习能力。在本文中，我们提出了一种用于无监督聚类（DFVC）的无解码器变分深度嵌入。众所周知，最小化重建误差相当于最大化输入与其表示之间的互信息（MI）的下界。这为我们抛弃臃肿的解码器提供了理论上的保证。受对比自监督学习的启发，我们可以直接计算或估计连续变量的 MI。具体来说，我们通过同时考虑连续表示的 MI 估计和分类表示的 MI 计算来研究无监督表示学习。通过引入数据增强技术，我们将原始输入、增强输入及其高级表示合并到 MI 估计框架中，以学习更多判别性表示。我们不是对抗性地匹配简单的标准正态分布，而是通过应用高斯混合分布作为先验，使用端到端学习来将潜在空间限制为集群友好。对具有挑战性的数据集进行的大量实验表明，我们的模型在各种最先进的聚类方法中实现了更高的性能。