The training of Deep Neural Networks (DNNs) brings enormous memory requirements and computational complexity, which makes it a challenge to train DNN models on resource-constrained devices. Training DNNs with reduced-precision data representation is crucial to mitigate this problem. In this article, we conduct a thorough investigation on training DNNs with low-bit posit numbers, a Type-III universal number (Unum). Through a comprehensive analysis of quantization with various data formats, it is demonstrated that the posit format shows great potential to be employed in the training of DNNs. Moreover, a DNN training framework using 8-bit posit is proposed with a novel tensor-wise scaling scheme. The experiments show the same performance as the state-of-the-art (SOTA) across multiple datasets (MNIST, CIFAR-10, ImageNet, and Penn Treebank) and model architectures (LeNet-5, AlexNet, ResNet, MobileNet-V2, and LSTM). We further design an energy-efficient hardware prototype for our framework. Compared to the standard floating-point counterpart, our design achieves a reduction of 68, 51, and 75 percent in terms of area, power, and memory capacity, respectively.

中文翻译：

---

使用位置数字系统对深度神经网络训练的评估

深度神经网络 (DNN) 的训练带来了巨大的内存需求和计算复杂性，这使得在资源受限的设备上训练 DNN 模型成为一项挑战。用降低精度的数据表示训练 DNNs 对于缓解这个问题至关重要。在本文中，我们对使用低位定位数（Type-III 通用数 (Unum)）训练 DNN 进行了彻底调查。通过对各种数据格式的量化进行综合分析，证明 posit 格式在 DNN 的训练中显示出巨大的潜力。此外，还提出了一种使用 8 位 posit 的 DNN 训练框架，其中包含一种新颖的张量缩放方案。实验表明，在多个数据集（MNIST、CIFAR-10、ImageNet、和 Penn Treebank）和模型架构（LeNet-5、AlexNet、ResNet、MobileNet-V2 和 LSTM）。我们进一步为我们的框架设计了一个节能的硬件原型。与标准浮点对应物相比，我们的设计在面积、功耗和内存容量方面分别减少了 68%、51% 和 75%。