This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a Dirichlet prior for a continuous latent variable that exhibits the characteristic of the categorical probabilities. To infer the parameters of DirVAE, we utilize the stochastic gradient method by approximating the Gamma distribution, which is a component of the Dirichlet distribution, with the inverse Gamma CDF approximation. Additionally, we reshape the component collapsing issue by investigating two problem sources, which are decoder weight collapsing and latent value collapsing, and we show that DirVAE has no component collapsing; while Gaussian VAE exhibits the decoder weight collapsing and Stick-Breaking VAE shows the latent value collapsing. The experimental results show that 1) DirVAE models the latent representation result with the best log-likelihood compared to the baselines; and 2) DirVAE produces more interpretable latent values with no collapsing issues which the baseline models suffer from. Also, we show that the learned latent representation from the DirVAE achieves the best classification accuracy in the semi-supervised and the supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models show better performances in most cases.

中文翻译：

---

Dirichlet 变分自编码器

本文提出了 Dirichlet Variational Autoencoder (DirVAE)，将 Dirichlet 先验用于展示分类概率特征的连续潜在变量。为了推断 DirVAE 的参数，我们利用随机梯度方法通过逆 Gamma CDF 逼近逼近 Gamma 分布（Dirichlet 分布的一个组成部分）。此外，我们通过调查解码器权重崩溃和潜在值崩溃这两个问题源来重塑组件崩溃问题，我们表明 DirVAE 没有组件崩溃；而 Gaussian VAE 显示解码器权重崩溃，而 Stick-Breaking VAE 显示潜在值崩溃。实验结果表明： 1) DirVAE 对潜在表征结果进行建模，与基线相比具有最佳对数似然；和 2) DirVAE 产生更多可解释的潜在值，没有基线模型所遭受的崩溃问题。此外，我们表明，与基线 VAE 相比，从 DirVAE 学习的潜在表示在 MNIST、OMNIGLOT 和 SVHN 上的半监督和监督分类任务中实现了最佳分类精度。最后，我们证明了 DirVAE 增强主题模型在大多数情况下表现出更好的性能。我们表明，与基线 VAE 相比，从 DirVAE 学习的潜在表示在 MNIST、OMNIGLOT 和 SVHN 上的半监督和监督分类任务中实现了最佳分类精度。最后，我们证明了 DirVAE 增强主题模型在大多数情况下表现出更好的性能。我们表明，与基线 VAE 相比，从 DirVAE 学习的潜在表示在 MNIST、OMNIGLOT 和 SVHN 上的半监督和监督分类任务中实现了最佳分类精度。最后，我们证明了 DirVAE 增强主题模型在大多数情况下表现出更好的性能。