Abstract

The property of the natural parallelization of matrix-vector operations inherent in memristor crossbars creates opportunities for their effective use in neural network computing. Analog calculations are faster by orders of magnitude than calculations on a central processor and on graphics accelerators. Furthermore, the energy costs of mathematical operations are significantly lower. At the same time, the low level of accuracy is characteristic of analog computing. In relation to this, studying the dependence of the neural network’s quality on the accuracy of setting its weights is relevant. The paper considers two convolutional neural networks trained on the MNIST (handwritten digits) and CIFAR_10 (airplanes, boats, cars, etc.) data sets. The first convolutional neural network consists of two convolutional layers, one subsample layer and two fully connected layers. The second one consists of four convolutional layers, two subsample layers and two fully connected layers. Calculations in convolutional and fully connected layers are performed through matrix-vector operations that are implemented on memristor crossbars. Subsampling layers imply the operation of finding the maximum value from several values. This operation can be implemented at the analog level. The process of training a neural network runs separately from data analysis. As a rule, gradient optimization methods are used at the training stage. It is advisable to perform calculations using these methods on a CPU. When setting the weights, 3–4 precision bits are required to obtain an acceptable recognition quality in the case the network is trained on MNIST, and 6–8 precision bits are required if the network is trained on CIFAR_10.

中文翻译：

---

在忆阻器交叉开关上实现神经网络算法的问题

摘要

忆阻器交叉开关中固有的矩阵矢量运算的自然并行化特性为其在神经网络计算中的有效使用创造了机会。模拟计算要比中央处理器和图形加速器上的计算快几个数量级。此外，数学运算的能源成本大大降低。同时，低精度是模拟计算的特征。与此相关的是，研究神经网络质量对权重设置精度的依赖性。本文考虑了在MNIST（手写数字）和CIFAR_10（飞机，轮船，汽车等）数据集上训练的两个卷积神经网络。第一个卷积神经网络由两个卷积层组成，一个子样本层和两个完全连接的层。第二层包括四个卷积层，两个子样本层和两个完全连接的层。卷积层和完全连接层中的计算是通过在忆阻器交叉开关上实现的矩阵矢量运算执行的。二次采样层意味着从多个值中找到最大值的操作。此操作可以在模拟级别实现。训练神经网络的过程与数据分析是分开进行的。通常，在训练阶段使用梯度优化方法。建议在CPU上使用这些方法执行计算。设置权重时，如果网络在MNIST上进行训练，则需要3-4个精度位才能获得可接受的识别质量，