ABSTRACT

Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the $D$-dimensional mean-field factorisation, enabling to reduce the sampling complexity from $O(D^2)$ to $O(D)$. To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster.

摘要

可以有效地解决基于完全分解的平均场变分推理；但是，它忽略了潜在变量之间的依赖关系，导致性能下降。为了解决这个问题，提出了 copula 变分推理 (CVI) 方法，该方法使用成熟的 copula 来有效地捕获后验依赖关系，从而获得更好的近似值。但是，它存在计算问题，即对具有大量潜在变量的大型模型进行优化非常耗时。这主要是由于在 CVI 中形成嘈杂的蒙特卡洛梯度时的昂贵采样造成的。对于 CVI 加速，在本文中，我们提出了一种新颖的快速 CVI（简称 FCVI）。在 FCVI 中，我们通过平均场分解的期望推导出 CVI 目标的梯度。因此，我们可以从$D$维平均场分解，能够降低采样复杂度$○(D^2)$到$○(D)$. 为了评估 FCVI，我们将其与建模性能和运行时的基线方法进行比较。实验结果表明，FCVI 与 CVI 相当，但运行速度更快。