Deep neural networks (DNNs) have gained a strong momentum among various applications. The enormous matrix-multiplication exhibited in the above DNNs is computation and memory intensive. Resistive random-access memory crossbar (RRAM-crossbar) consisting of memristor cells can naturally carry out the matrix-vector multiplication. RRAM-crossbar-based accelerator, therefore, has two orders of magnitude of higher energy-efficiency than conventional accelerators. The imperfect fabrication process of RRAM-crossbars, however, causes various defects and process variations. These fabrication imperfections not only result in significant yield loss but also degrade the accuracy of DNNs executed on the RRAM-crossbars. In this article, we first propose an accelerator-friendly neural-network training method, by leveraging the inherent self-healing capability of the neural network, to prevent the large-weight synapses from being mapped to the imperfect memristors. Next, we propose a dynamic adjustment mechanism to extend the above method for DNNs, such as multilayer perceptrons (MLPs), wherein the imperfect-memristor induced errors can accumulate and magnify through multiple layers. Such off-device training method is a pure software solution, and it is unable to provide enough accuracy for convolutional neural networks (CNNs). Several works propose error-tolerable hardware design by allowing the retraining of CNNs on the RRAM-crossbar. Although this hardware-based on-device training method is effective, the frequent write operation on RRAM-crossbar hurt the endurance of RRAM-crossbars. Consequently, we propose a software and hardware co-design methodology to effectively preserve the classification accuracy of CNN with few on-device training iterations. The experimental results show that the proposed method can guarantee ≤1.1% loss of accuracy for resistance variations in MLP and CNN. Moreover, the proposed method can guarantee ≤1% loss of accuracy even when stuck-at-faults (SAFs) rate = 20%.

中文翻译：

---

ITT-RNA：基于 RRAM-Crossbar 的深度神经网络加速器的不完美可容忍训练

深度神经网络 (DNN) 在各种应用中获得了强劲的发展势头。上述 DNN 中展示的巨大矩阵乘法是计算和内存密集型的。由忆阻器单元组成的电阻式随机存取存储器交叉开关（RRAM-crossbar）可以自然地进行矩阵向量乘法。因此，基于 RRAM 交叉开关的加速器的能效比传统加速器高两个数量级。然而，RRAM-crossbar 的不完美制造工艺会导致各种缺陷和工艺变化。这些制造缺陷不仅会导致显着的良率损失，还会降低在 RRAM 交叉开关上执行的 DNN 的准确性。在本文中，我们首先提出了一种对加速器友好的神经网络训练方法，通过利用神经网络固有的自愈能力，防止大权重突触被映射到不完美的忆阻器。接下来，我们提出了一种动态调整机制来扩展 DNN 的上述方法，例如多层感知器 (MLP)，其中不完美忆阻器引起的误差可以通过多层累积和放大。这种脱机训练方法是纯软件解决方案，无法为卷积神经网络（CNN）提供足够的准确性。几项工作通过允许在 RRAM 交叉开关上重新训练 CNN 来提出容错硬件设计。虽然这种基于硬件的设备端训练方法是有效的，但对 RRAM-crossbar 的频繁写操作损害了 RRAM-crossbar 的耐久性。最后，我们提出了一种软件和硬件协同设计方法，可以有效地保持 CNN 的分类准确性，而设备上的训练迭代很少。实验结果表明，所提出的方法可以保证 MLP 和 CNN 中电阻变化的精度损失≤1.1%。此外，即使卡在故障 (SAF) 率 = 20% 时，所提出的方法也可以保证≤1% 的精度损失。