Hardware for energy-efficient AI has received significant attention over the last years with both start-ups and large corporations creating products that compete at different levels of performance and power consumption. The main objective of this hardware is to offer levels of efficiency and performance that cannot be obtained with general-purpose processors or graphics processing units. In parallel, innovative hardware techniques such as near- and sub-threshold voltage processing have been revisited, capitalizing on the low-power requirements of deploying AI at the network edge. In this paper, we evaluate recent developments in hardware for energy-efficient AI, focusing on inference in embedded systems at the network edge. We then explore a heterogeneous configuration that deploys a neural network that processes multiple independent inputs and deploys convolutional and LSTM (Long Short-Term Memory) layers. This heterogeneous configuration uses two devices with different performance/power characteristics connected with a feedback loop. It obtains energy reductions measured at 75% while simultaneously maintaining the level of inference accuracy.

过去几年，初创企业和大型公司都在开发具有不同性能和功耗水平竞争产品的产品，从而使能效AI的硬件受到了广泛的关注。该硬件的主要目的是提供通用处理器或图形处理单元无法达到的效率和性能水平。同时，利用在网络边缘部署AI的低功耗要求，重新审视了诸如近阈值和亚阈值电压处理之类的创新硬件技术。在本文中，我们评估了节能AI硬件的最新发展，重点是网络边缘嵌入式系统的推理。然后，我们探索了一种异构配置，该配置部署了一个神经网络，该神经网络处理多个独立的输入，并部署了卷积和LSTM（长期短期记忆）层。这种异构配置使用两个具有不同性能/功率特性的设备，并通过一个反馈环路进行连接。在保持推理精度水平的同时，它可以减少75％的能量。