We provide gradient flow interpretations for the continuous-time continuous-state Hopfield neural network (HNN). The ordinary and stochastic differential equations associated with the HNN were introduced in the literature as analog optimizers and were reported to exhibit good performance in numerical experiments. In this work, we point out that the deterministic HNN can be transcribed into Amari's natural gradient descent, and thereby uncover the explicit relation between the underlying Riemannian metric and the activation functions. By exploiting an equivalence between the natural gradient descent and the mirror descent, we show how the choice of activation function governs the geometry of the HNN dynamics. For the stochastic HNN, we show that the so-called ``diffusion machine,'' while not a gradient flow itself, induces a gradient flow when lifted in the space of probability measures. We characterize this infinite-dimensional flow as the gradient descent of certain free energy with respect to a Wasserstein metric that depends on the geodesic distance on the ground manifold. Furthermore, we demonstrate how this gradient flow interpretation can be used for fast computation via recently developed proximal algorithms.

中文翻译：

---

Hopfield神经网络流程：几何观点。

我们为连续时间连续状态Hopfield神经网络（HNN）提供了梯度流解释。与HNN相关的普通和随机微分方程在文献中被引入为模拟优化器，并据报道在数值实验中表现出良好的性能。在这项工作中，我们指出，确定性HNN可以转录为Amari的自然梯度下降，从而揭示基本的黎曼度量和激活函数之间的显式关系。通过利用自然梯度下降和镜像下降之间的等价关系，我们展示了激活函数的选择如何控制HNN动力学的几何形状。对于随机HNN，我们证明了所谓的``扩散机''虽然本身不​​是梯度流，在概率测度空间中提升时会引起梯度流。我们将这种无限维流动表征为某些自由能相对于Wasserstein度量的梯度下降，该下降取决于地面流形上的测地距离。此外，我们演示了如何通过最近开发的近端算法将这种梯度流解释用于快速计算。