Models with a large number of latent variables are often used to utilize the information in big or complex data, but can be difficult to estimate. Variational inference methods provide an attractive solution. These methods use an approximation to the posterior density, yet for large latent variable models existing choices can be inaccurate or slow to calibrate. Here, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. It combines a parsimonious approximation for the parameter posterior with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of optimization algorithms used for calibration. Implementation only requires exact or approximate generation from the conditional posterior of the latent variables, rather than computation of their density. In effect, our method provides a new way to employ Markov chain Monte Carlo (MCMC) within variational inference. We illustrate using two complex contemporary econometric examples. The first is a nonlinear multivariate state space model for U.S. macroeconomic variables. The second is a random coefficients tobit model applied to two million sales by 20,000 individuals in a consumer panel. In both cases, our approximating family is considerably more accurate than mean field or structured Gaussian approximations, and faster than MCMC. Last, we show how to implement data sub-sampling in variational inference for our approximation, further reducing computation time. MATLAB code implementing the method is provided.

具有大量潜在变量的模型通常用于利用大数据或复杂数据中的信息，但可能难以估计。变分推理方法提供了一个有吸引力的解决方案。这些方法使用后验密度的近似值，但对于大型潜在变量模型，现有选择可能不准确或校准缓慢。在这里，我们提出了一系列易于处理的变分近似，它们在这种情况下更准确、更快地进行校准。它结合了参数后验的简约近似与潜在变量的精确条件后验。我们推导出变分下界的重新参数化梯度的简化表达式，这是用于校准的优化算法的主要成分。实现只需要从潜在变量的条件后验中精确或近似生成，而不是计算它们的密度。实际上，我们的方法提供了一种在变分推理中使用马尔可夫链蒙特卡罗 (MCMC) 的新方法。我们使用两个复杂的当代计量经济学示例进行说明。第一个是美国宏观经济变量的非线性多元状态空间模型。第二个是随机系数 tobit 模型，适用于消费者面板中 20,000 个人的 200 万次销售额。在这两种情况下，我们的近似系列都比平均场或结构化高斯近似更准确，并且比 MCMC 更快。最后，我们展示了如何在变分推理中为我们的近似值实现数据子采样，进一步减少计算时间。