The emerging field of artificial intelligence of things (AIoT, AI+IoT) is driven by the widespread use of intelligent infrastructures and the impressive success of deep learning (DL). With the deployment of DL on various intelligent infrastructures featuring rich sensors and weak DL computing capabilities, a diverse range of AIoT applications has become possible. However, DL models are notoriously resource-intensive. Existing research strives to realize near-/realtime inference of AIoT live data and low-cost training using AIoT datasets on resource-scare infrastructures. Accordingly, the accuracy and responsiveness of DL models are bounded by resource availability. To this end, the algorithm-system co-design that jointly optimizes the resource-friendly DL models and model-adaptive system scheduling improves the runtime resource availability and thus pushes the performance boundary set by the standalone level. Unlike previous surveys on resource-friendly DL models or hand-crafted DL compilers/frameworks with partially fine-tuned components, this survey aims to provide a broader optimization space for more free resource-performance tradeoffs. The cross-level optimization landscape involves various granularity, including the DL model, computation graph, operator, memory schedule, and hardware instructor in both on-device and distributed paradigms. Furthermore, due to the dynamic nature of AIoT context, which includes heterogeneous hardware, agnostic sensing data, varying user-specified performance demands, and resource constraints, this survey explores the context-aware inter-/intra-device controllers for automatic cross-level adaptation. Additionally, we identify some potential directions for resource-efficient AIoT systems. By consolidating problems and techniques scattered over diverse levels, we aim to help readers understand their connections and stimulate further discussions.

中文翻译：

---

通过跨级优化实现资源高效的 AIoT 系统：一项调查

人工智能物联网（AIoT、AI+IoT）这一新兴领域是由智能基础设施的广泛使用和深度学习（DL）的巨大成功推动的。随着深度学习在各种传感器丰富、深度学习计算能力较弱的智能基础设施上的部署，多样化的AIoT应用成为可能。然而，深度学习模型是出了名的资源密集型。现有研究致力于在资源稀缺的基础设施上使用 AIoT 数据集实现 AIoT 实时数据的近/实时推理和低成本训练。因此，深度学习模型的准确性和响应能力受到资源可用性的限制。为此，算法系统协同设计联合优化资源友好的深度学习模型和模型自适应系统调度，提高了运行时资源可用性，从而突破了独立级别设定的性能边界。与之前对资源友好型深度学习模型或带有部分微调组件的手工制作的深度学习编译器/框架的调查不同，本次调查旨在为更多免费的资源性能权衡提供更广泛的优化空间。跨级优化景观涉及各种粒度，包括设备上和分布式范式中的深度学习模型、计算图、运算符、内存调度和硬件指导器。此外，由于 AIoT 环境的动态特性，包括异构硬件、不可知的传感数据、不同的用户指定的性能需求和资源限制，本次调查探讨了用于自动跨级别的环境感知设备间/设备内控制器。适应。此外，我们还确定了资源高效型 AIoT 系统的一些潜在方向。通过整合分散在不同层面的问题和技术，我们的目标是帮助读者理解它们之间的联系并激发进一步的讨论。