Analog crossbar architectures for accelerating neural network training and inference have made tremendous progress over the past several years. These architectures are ideal for dense layers with fewer than roughly a thousand neurons. However, for large sparse layers, crossbar architectures are highly inefficient. A new hardware architecture, dubbed the MN3 (Memristive Nanowire Neural Network), was recently described as an efficient architecture for simulating very wide, sparse neural network layers, on the order of millions of neurons per layer. The MN3 utilizes a high-density memristive nanowire mesh to efficiently connect large numbers of silicon neurons with modifiable weights. Here, in order to explore the MN3's ability to function as a deep neural network, we describe one algorithm for training deep MN3 models and benchmark simulations of the architecture on two deep learning tasks. We utilize a simple piecewise linear memristor model, since we seek to demonstrate that training is, in principle, possible for randomized nanowire architectures. In future work, we intend on utilizing more realistic memristor models, and we will adapt the presented algorithm appropriately. We show that the MN3 is capable of performing composition, gradient propagation, and weight updates, which together allow it to function as a deep neural network. We show that a simulated multilayer perceptron (MLP), built from MN3 networks, can obtain a 1.61% error rate on the popular MNIST dataset, comparable to equivalently sized software-based network. This work represents, to the authors' knowledge, the first randomized nanowire architecture capable of reproducing the backpropagation algorithm.

中文翻译：

---

忆阻纳米线网络中的深度学习

用于加速神经网络训练和推理的模拟交叉架构在过去几年取得了巨大进步。这些架构非常适用于少于大约一千个神经元的密集层。然而，对于大型稀疏层，交叉架构非常低效。一种新的硬件架构，被称为 MN3（忆阻纳米线神经网络），最近被描述为一种有效的架构，用于模拟非常宽、稀疏的神经网络层，每层有数百万个神经元。MN3 利用高密度忆阻纳米线网有效地连接大量具有可修改权重的硅神经元。在这里，为了探索 MN3 作为深度神经网络的能力，我们描述了一种用于训练深度 MN3 模型的算法以及在两个深度学习任务上对该架构进行基准模拟。我们利用简单的分段线性忆阻器模型，因为我们试图证明训练原则上可以用于随机纳米线架构。在未来的工作中，我们打算使用更现实的忆阻器模型，我们将适当地调整所提出的算法。我们展示了 MN3 能够执行组合、梯度传播和权重更新，这些共同使其能够充当深度神经网络。我们表明，从 MN3 网络构建的模拟多层感知器 (MLP) 可以在流行的 MNIST 数据集上获得 1.61% 的错误率，与同等大小的基于软件的网络相当。这项工作代表，据作者所知，