Recently, analog compute-in-memory (CIM) architectures based on emerging analog non-volatile memory (NVM) technologies have been explored for deep neural networks (DNN) to improve energy efficiency. Such architectures, however, leverage charge conservation, an operation with infinite resolution, and thus are susceptible to errors. The computations in DNN realized by analog NVM thus have high uncertainty due to the device stochasticity. Several reports have demonstrated the use of analog NVM for CIM in a limited scale. It is unclear whether the uncertainties in computations will prohibit large-scale DNNs. To explore this critical issue of scalability, this paper first presents a simulation framework to evaluate the feasibility of large-scale DNNs based on CIM architecture and analog NVM. Simulation results show that DNNs trained for high-precision digital computing engines are not resilient against the uncertainty of the analog NVM devices. To avoid such catastrophic failures, this paper introduces the analog floating-point representation for the DNN, and the Hessian-Aware Stochastic Gradient Descent (HA-SGD) training algorithm to enhance the inference accuracy of trained DNNs. As a result of such enhancements, DNNs such as Wide ResNets for the CIFAR-100 image recognition problem are demonstrated to have significant performance improvements in accuracy without adding cost to the inference hardware.

中文翻译：

---

模拟内存计算推理引擎的准确性和弹性

最近，基于新兴模拟非易失性存储器 (NVM) 技术的模拟内存计算 (CIM) 架构已被探索用于深度神经网络 (DNN)，以提高能源效率。然而，这种架构利用电荷守恒，这​​是一种具有无限分辨率的操作，因此容易出错。由于设备的随机性，由模拟 NVM 实现的 DNN 中的计算因此具有很高的不确定性。几份报告已经证明模拟 NVM 在有限范围内用于 CIM。目前尚不清楚计算中的不确定性是否会禁止大规模 DNN。为了探讨可扩展性这一关键问题，本文首先提出了一个模拟框架来评估基于 CIM 架构和模拟 NVM 的大规模 DNN 的可行性。仿真结果表明，为高精度数字计算引擎训练的 DNN 对模拟 NVM 设备的不确定性没有弹性。为了避免这种灾难性的失败，本文引入了 DNN 的模拟浮点表示和 Hessian-Aware 随机梯度下降 (HA-SGD) 训练算法，以提高训练后的 DNN 的推理精度。由于此类增强，DNN（例如用于 CIFAR-100 图像识别问题的 Wide ResNets）在准确性方面具有显着的性能改进，而不会增加推理硬件的成本。以及 Hessian-Aware 随机梯度下降 (HA-SGD) 训练算法，以提高训练 DNN 的推理精度。由于此类增强，DNN（例如用于 CIFAR-100 图像识别问题的 Wide ResNets）在准确性方面具有显着的性能改进，而不会增加推理硬件的成本。以及 Hessian-Aware 随机梯度下降 (HA-SGD) 训练算法，以提高训练 DNN 的推理精度。由于此类增强，DNN（例如用于 CIFAR-100 图像识别问题的 Wide ResNets）在准确性方面具有显着的性能改进，而不会增加推理硬件的成本。