In-memory computing reduces latency and energy consumption of Deep Neural Networks (DNNs) by reducing the number of off-chip memory accesses. However, crossbar-based in-memory computing may significantly increase the volume of on-chip communication since the weights and activations are on-chip. State-of-the-art interconnect methodologies for in-memory computing deploy a bus-based network or mesh-based Network-on-Chip (NoC). Our experiments show that up to 90% of the total inference latency of a DNN hardware is spent on on-chip communication when the bus-based network is used. To reduce the communication latency, we propose a methodology to generate an NoC architecture along with a scheduling technique customized for different DNNs. We prove mathematically that the generated NoC architecture and corresponding schedules achieve the minimum possible communication latency for a given DNN. Furthermore, we generalize the proposed solution for edge computing and cloud computing. Experimental evaluations on a wide range of DNNs show that the proposed NoC architecture enables 20%–80% reduction in communication latency with respect to state-of-the-art interconnect solutions.

中文翻译：

---

用于 DNN 内存加速的延迟优化的可重构 NoC

内存计算通过减少片外内存访问次数来减少深度神经网络 (DNN) 的延迟和能耗。然而，基于交叉开关的内存计算可能会显着增加片上通信的数量，因为权重和激活是在片上的。用于内存计算的最先进互连方法部署了基于总线的网络或基于网格的片上网络 (NoC)。我们的实验表明，当使用基于总线的网络时，DNN 硬件高达 90% 的总推理延迟花费在片上通信上。为了减少通信延迟，我们提出了一种生成 NoC 架构的方法以及针对不同 DNN 定制的调度技术。我们在数学上证明了生成的 NoC 架构和相应的调度实现了给定 DNN 的最小可能的通信延迟。此外，我们概括了所提出的边缘计算和云计算解决方案。对各种 DNN 的实验评估表明，相对于最先进的互连解决方案，所提出的 NoC 架构能够将通信延迟降低 20%–80%。