Training of Deep Neural Network (DNN) at the edge faces the challenge of high energy consumption due to the requirements of a large number of memory accesses for gradient calculations. Therefore, it is necessary to minimize data fetches to perform training of a DNN model on the edge. In this paper, a novel technique has been proposed to reduce the memory access for the training of fully connected layers in transfer learning. By analyzing the memory access patterns in the backpropagation phase in fully connected layers, the memory access can be optimized. We introduce a new method to update the weights by introducing the delta term for every node of output and fully connected layer. Delta term aims to reduce memory access for the parameters which are required to access repeatedly during the training process of fully connected layers. The proposed technique shows 0.13x-13.93x energy savings for the training of fully connected layers for famous DNN architectures on multiple processor architectures. The proposed technique can be used to perform transfer learning on-chip to reduce energy consumption as well as memory access.

中文翻译：

---

片上传输学习的存储器访问优化

由于需要大量内存访问以进行梯度计算，因此在边缘训练深度神经网络（DNN）面临着高能耗的挑战。因此，有必要最小化数据获取以在边缘执行DNN模型的训练。在本文中，提出了一种新颖的技术来减少内存访问，以便在转移学习中训练完全连接的层。通过在完全连接的层中的反向传播阶段分析内存访问模式，可以优化内存访问。通过引入输出和完全连接层的每个节点的增量项，我们引入了一种更新权重的新方法。Delta术语旨在减少对在完全连接的层的训练过程中重复访问所需参数的内存访问。对于多处理器架构上著名的DNN架构的完全连接层的训练，所提出的技术显示出0.13x-13.93x的能源节省。所提出的技术可用于在芯片上执行传输学习，以减少能耗和内存访问。