当前位置: X-MOL 学术Neural Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Restricted Boltzmann Machines as Models of Interacting Variables
Neural Computation ( IF 2.9 ) Pub Date : 2021-09-16 , DOI: 10.1162/neco_a_01420
Nicola Bulso 1 , Yasser Roudi 2
Affiliation  

We study the type of distributions that restricted Boltzmann machines (RBMs) with different activation functions can express by investigating the effect of the activation function of the hidden nodes on the marginal distribution they impose on observed binary nodes. We report an exact expression for these marginals in the form of a model of interacting binary variables with the explicit form of the interactions depending on the hidden node activation function. We study the properties of these interactions in detail and evaluate how the accuracy with which the RBM approximates distributions over binary variables depends on the hidden node activation function and the number of hidden nodes. When the inferred RBM parameters are weak, an intuitive pattern is found for the expression of the interaction terms, which reduces substantially the differences across activation functions. We show that the weak parameter approximation is a good approximation for different RBMs trained on the MNIST data set. Interestingly, in these cases, the mapping reveals that the inferred models are essentially low order interaction models.



中文翻译:

作为相互作用变量模型的受限玻尔兹曼机

我们通过研究隐藏节点的激活函数对它们施加在观察到的二元节点上的边际分布的影响,研究了具有不同激活函数的限制玻尔兹曼机 (RBM) 可以表达的分布类型。我们以交互二元变量模型的形式报告了这些边缘的精确表达式,其中交互的显式形式取决于隐藏节点激活函数。我们详细研究了这些相互作用的特性,并评估了 RBM 近似二元变量分布的准确性如何取决于隐藏节点激活函数和隐藏节点的数量。当推断出的 RBM 参数较弱时,会发现交互项表达的直观模式,这大大减少了激活函数之间的差异。我们表明,对于在 MNIST 数据集上训练的不同 RBM,弱参数近似是一个很好的近似。有趣的是,在这些情况下,映射表明推断模型本质上是低阶交互模型。

更新日期:2021-09-17
down
wechat
bug