Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is used in many DNN software frameworks. However, the performance gain is at the expense of a reduction in floating point (FP) numerical accuracy. In this article, we analyse the worst-case FP error and derive an estimation of the norm and conditioning of the algorithm. We show that the bound grows exponentially with the size of the convolution. Further, the error bound of the modified algorithm is slightly lower but still exponential. We propose several methods for reducing FP error. We propose a canonical evaluation ordering based on Huffman coding that reduces summation error. We study the selection of sampling “points” experimentally and find empirically good points for the most important sizes. We identify the main factors associated with good points. In addition, we explore other methods to reduce FP error, including mixed-precision convolution, and pairwise summation across DNN channels. Using our methods, we can significantly reduce FP error for a given block size, which allows larger block sizes to be used and reduced computation.

中文翻译：

---

深度神经网络 Winograd 卷积的误差分析和提高精度

流行的深度神经网络 (DNN) 将大部分执行时间用于计算卷积。Winograd 系列算法可以大大减少所需的算术运算次数，并在许多 DNN 软件框架中使用。然而，性能提升是以浮点 (FP) 数值精度降低为代价的。在本文中，我们分析了最坏情况下的 FP 误差，并推导出算法的范数和条件估计。我们证明了边界随着卷积的大小呈指数增长。此外，修改后的算法的误差范围略低，但仍然是指数级的。我们提出了几种减少FP错误的方法。我们提出了一种基于霍夫曼编码的规范评估排序，以减少求和误差。我们通过实验研究采样“点”的选择，并为最重要的大小找到经验上的好点。我们确定了与优点相关的主要因素。此外，我们探索了其他减少 FP 误差的方法，包括混合精度卷积和跨 DNN 通道的成对求和。使用我们的方法，我们可以显着减少给定块大小的 FP 误差，从而允许使用更大的块大小并减少计算量。