Restricted Boltzmann machines (RBMs) are a powerful class of generative models, but their training requires computing a gradient that, unlike supervised backpropagation on typical loss functions, is notoriously difficult even to approximate. Here, we show that properly combining standard gradient updates with an off-gradient direction, constructed from samples of the RBM ground state (mode), improves training dramatically over traditional gradient methods. This approach, which we call ‘mode-assisted training’, promotes faster training and stability, in addition to lower converged relative entropy (KL divergence). We demonstrate its efficacy on synthetic datasets where we can compute KL divergences exactly, as well as on a larger machine learning standard (MNIST). The proposed mode-assisted training can be applied in conjunction with any given gradient method, and is easily extended to more general energy-based neural network structures such as deep, convolutional and unrestricted Boltzmann machines.

受限玻尔兹曼机（RBM）是一类强大的生成模型，但是其训练需要计算一个梯度，与典型损失函数的有监督反向传播不同，该梯度甚至很难估计。在这里，我们表明，根据RBM基态（模式）的样本构建的，将标准梯度更新与偏离梯度方向正确组合起来，可以比传统的梯度方法显着提高训练效果。这种方法，我们称为“模式辅助训练”，除了可以降低收敛的相对熵（KL散度）之外，还可以促进更快的训练和稳定性。我们在可以精确计算KL散度的合成数据集以及较大的机器学习标准（MNIST）上展示了其功效。