Neural networks employ massive interconnection of simple computing units called neurons to compute the problems that are highly nonlinear and could not be hard coded into a program. These neural networks are computation-intensive, and training them requires a lot of training data. Each training example requires heavy computations. We look at different ways in which we can reduce the heavy computation requirement and possibly make them work on mobile devices. In this paper, we survey various techniques that can be matched and combined in order to improve the training time of neural networks. Additionally, we also review some extra recommendations to make the process work for mobile devices as well. We finally survey deep compression technique that tries to solve the problem by network pruning, quantization, and encoding the network weights. Deep compression reduces the time required for training the network by first pruning the irrelevant connections, i.e., the pruning stage, which is then followed by quantizing the network weights via choosing centroids for each layer. Finally, at the third stage, it employs Huffman encoding algorithm to deal with the storage issue of the remaining weights.

中文翻译：

---

深度卷积神经网络在移动计算中的复杂性

神经网络利用称为神经元的简单计算单元的大量互连来计算高度非线性且无法硬编码到程序中的问题。这些神经网络是计算密集型的，对其进行训练需要大量的训练数据。每个训练示例都需要大量的计算。我们研究了可以减少繁重的计算需求并可能使它们在移动设备上运行的不同方式。在本文中，我们调查了各种可以匹配和组合的技术，以缩短神经网络的训练时间。此外，我们还审查了一些其他建议，以使该流程也适用于移动设备。我们最终研究了深度压缩技术，该技术试图通过网络修剪，量化和编码网络权重来解决问题。深度压缩通过首先修剪不相关的连接（即修剪阶段）来减少训练网络所需的时间，然后再通过为每个层选择质心来量化网络权重。最后，在第三阶段，它采用霍夫曼编码算法来处理剩余权重的存储问题。