Analog electronic and optical computing exhibit tremendous advantages over digital computing for accelerating deep learning when operations are executed at low precision. In this work, we derive a relationship between analog precision, which is limited by noise, and digital bit precision. We propose extending analog computing architectures to support varying levels of precision by repeating operations and averaging the result, decreasing the impact of noise. Such architectures enable programmable tradeoffs between precision and other desirable performance metrics such as energy efficiency or throughput. To utilize dynamic precision, we propose a method for learning the precision of each layer of a pre-trained model without retraining network weights. We evaluate this method on analog architectures subject to a variety of noise sources such as shot noise, thermal noise, and weight noise and find that employing dynamic precision reduces energy consumption by up to 89% for computer vision models such as Resnet50 and by 24% for natural language processing models such as BERT. In one example, we apply dynamic precision to a shot-noise limited homodyne optical neural network and simulate inference at an optical energy consumption of 2.7 aJ/MAC for Resnet50 and 1.6 aJ/MAC for BERT with <2% accuracy degradation.

中文翻译：

---

神经网络的动态精度模拟计算

当以低精度执行操作时，模拟电子和光学计算相对于数字计算具有巨大的优势，可加速深度学习。在这项工作中，我们得出了受噪声限制的模拟精度与数字位精度之间的关系。我们建议扩展模拟计算体系结构，以通过重复操作并取平均结果来支持变化的精度水平，从而减少噪声的影响。这样的架构使得能够在精度和其他期望的性能指标（例如能源效率或吞吐量）之间进行可编程的折衷。为了利用动态精度，我们提出了一种在不重新训练网络权重的情况下学习预训练模型各层精度的方法。我们在受多种噪声源（例如散粒噪声，热噪声和重量噪声）影响的模拟架构上评估了该方法，发现采用动态精度可将计算机视觉模型（如Resnet50）的能耗降低多达89％，并将能耗降低24％用于自然语言处理模型，例如BERT。在一个示例中，我们将动态精度应用于散粒噪声受限的零差光学神经网络，并以RESn50的2.7 aJ / MAC和BERT的1.6 aJ / MAC的光能消耗模拟推理，而精度降低了2％。