Journal of Experimental & Theoretical Artificial Intelligence ( IF 1.7 ) Pub Date : 2021-01-15 , DOI: 10.1080/0952813x.2021.1871970 Jinjin Chi 1, 2 , Jihong Ouyang 2 , Ang Zhang 2 , Xinhua Wang 2, 3 , Ximing Li 2
ABSTRACT
Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the -dimensional mean-field factorisation, enabling to reduce the sampling complexity from to . To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster.
中文翻译:
快速 copula 变分推理
摘要
可以有效地解决基于完全分解的平均场变分推理;但是,它忽略了潜在变量之间的依赖关系,导致性能下降。为了解决这个问题,提出了 copula 变分推理 (CVI) 方法,该方法使用成熟的 copula 来有效地捕获后验依赖关系,从而获得更好的近似值。但是,它存在计算问题,即对具有大量潜在变量的大型模型进行优化非常耗时。这主要是由于在 CVI 中形成嘈杂的蒙特卡洛梯度时的昂贵采样造成的。对于 CVI 加速,在本文中,我们提出了一种新颖的快速 CVI(简称 FCVI)。在 FCVI 中,我们通过平均场分解的期望推导出 CVI 目标的梯度。因此,我们可以从维平均场分解,能够降低采样复杂度到. 为了评估 FCVI,我们将其与建模性能和运行时的基线方法进行比较。实验结果表明,FCVI 与 CVI 相当,但运行速度更快。