Variational Auto-encoders (VAEs) are deep generative latent variable models consisting of two components: a generative model that captures a data distribution p(x) by transforming a distribution p(z) over latent space, and an inference model that infers likely latent codes for each data point (Kingma and Welling, 2013). Recent work shows that traditional training methods tend to yield solutions that violate modeling desiderata: (1) the learned generative model captures the observed data distribution but does so while ignoring the latent codes, resulting in codes that do not represent the data (e.g. van den Oord et al. (2017); Kim et al. (2018)); (2) the aggregate of the learned latent codes does not match the prior p(z). This mismatch means that the learned generative model will be unable to generate realistic data with samples from p(z)(e.g. Makhzani et al. (2015); Tomczak and Welling (2017)). In this paper, we demonstrate that both issues stem from the fact that the global optima of the VAE training objective often correspond to undesirable solutions. Our analysis builds on two observations: (1) the generative model is unidentifiable - there exist many generative models that explain the data equally well, each with different (and potentially unwanted) properties and (2) bias in the VAE objective - the VAE objective may prefer generative models that explain the data poorly but have posteriors that are easy to approximate. We present a novel inference method, LiBI, mitigating the problems identified in our analysis. On synthetic datasets, we show that LiBI can learn generative models that capture the data distribution and inference models that better satisfy modeling assumptions when traditional methods struggle to do so.

中文翻译：

---

表征和避免变分自编码器的有问题的全局优化

变分自动编码器 (VAE) 是由两个组件组成的深度生成潜在变量模型：通过在潜在空间上转换分布 p(z) 来捕获数据分布 p(x) 的生成模型，以及推断可能潜在潜在变量的推理模型每个数据点的代码（Kingma 和 Welling，2013 年）。最近的工作表明，传统的训练方法往往会产生违反建模需求的解决方案：（1）学习的生成模型捕获了观察到的数据分布，但这样做时忽略了潜在代码，导致代码不代表数据（例如 van den Oord 等人（2017 年）；Kim 等人（2018 年））；(2) 学习到的潜在代码的集合与先验 p(z) 不匹配。这种不匹配意味着学习的生成模型将无法使用来自 p(z) 的样本生成真实数据（例如 马赫扎尼等人。(2015); Tomczak 和 Welling (2017))。在本文中，我们证明了这两个问题都源于这样一个事实，即 VAE 训练目标的全局最优通常对应于不受欢迎的解决方案。我们的分析基于两个观察结果：(1) 生成模型是无法识别的——存在许多同样能很好地解释数据的生成模型，每个模型都有不同的（和可能不需要的）属性和 (2) VAE 目标中的偏差——VAE 目标可能更喜欢能够很好地解释数据但具有易于近似的后验的生成模型。我们提出了一种新的推理方法 LiBI，以减轻我们分析中发现的问题。在合成数据集上，