Conventional hardware-friendly quantization methods, such as fixed-point or integer, tend to perform poorly at very low word sizes as their shrinking dynamic ranges cannot adequately capture the wide data distributions commonly seen in sequence transduction models. We present AdaptivFloat, a floating-point inspired number representation format for deep learning that dynamically maximizes and optimally clips its available dynamic range, at a layer granularity, in order to create faithful encoding of neural network parameters. AdaptivFloat consistently produces higher inference accuracies compared to block floating-point, uniform, IEEE-like float or posit encodings at very low precision ($\leq$ 8-bit) across a diverse set of state-of-the-art neural network topologies. And notably, AdaptivFloat is seen surpassing baseline FP32 performance by up to +0.3 in BLEU score and -0.75 in word error rate at weight bit widths that are $\leq$ 8-bit. Experimental results on a deep neural network (DNN) hardware accelerator, exploiting AdaptivFloat logic in its computational datapath, demonstrate per-operation energy and area that is 0.9$\times$ and 1.14$\times$, respectively, that of equivalent bit width integer-based accelerator variants.

中文翻译：

---

AdaptivFloat：一种用于弹性深度学习推理的基于浮点的数据类型

传统的硬件友好量化方法，例如定点或整数，往往在非常低的字长下表现不佳，因为它们不断缩小的动态范围无法充分捕捉序列转换模型中常见的广泛数据分布。我们提出了 AdaptivFloat，这是一种用于深度学习的浮点启发的数字表示格式，它以层粒度动态最大化和最佳剪辑其可用动态范围，以便创建神经网络参数的忠实编码。与块浮点、统一、类似 IEEE 的浮点或位置编码相比，AdaptivFloat 始终以非常低的精度（$\leq$ 8 位）在各种最先进的神经网络拓扑中产生更高的推理精度. 值得注意的是，AdaptivFloat 的 BLEU 得分超过基准 FP32 性能高达 +0.3，在权重位宽为 $\leq$ 8 位时的字错误率超过 -0.75。在深度神经网络 (DNN) 硬件加速器上的实验结果，在其计算数据路径中利用 AdaptivFloat 逻辑，证明每次操作的能量和面积分别为 0.9$\times$ 和 1.14$\times$，相当于位宽整数基于加速器的变体。