Artificial intelligence and machine learning are experiencing widespread adoption in industry and academia. This has been driven by rapid advances in the applications and accuracy of AI through increasingly complex algorithms and models; this, in turn, has spurred research into specialized hardware AI accelerators. Given the rapid pace of advances, it is easy to forget that they are often developed and evaluated in a vacuum without considering the full application environment. This article emphasizes the need for a holistic, end-to-end analysis of artificial intelligence (AI) workloads and reveals the “AI tax.” We deploy and characterize Face Recognition in an edge data center. The application is an AI-centric edge video analytics application built using popular open source infrastructure and machine learning (ML) tools. Despite using state-of-the-art AI and ML algorithms, the application relies heavily on pre- and post-processing code. As AI-centric applications benefit from the acceleration promised by accelerators, we find they impose stresses on the hardware and software infrastructure: storage and network bandwidth become major bottlenecks with increasing AI acceleration. By specializing for AI applications, we show that a purpose-built edge data center can be designed for the stresses of accelerated AI at 15% lower TCO than one derived from homogeneous servers and infrastructure.

中文翻译：

---

人工智能税

人工智能和机器学习正在工业界和学术界得到广泛采用。这得益于人工智能应用程序和准确性的快速进步，算法和模型越来越复杂；这反过来又刺激了对专用硬件 AI 加速器的研究。鉴于进步的快速步伐，很容易忘记它们通常是在真空中开发和评估而没有考虑完整的应用环境。本文强调需要对人工智能 (AI) 工作负载进行全面的端到端分析，并揭示“人工智能税”。我们部署和表征人脸识别在边缘数据中心。该应用程序是使用流行的开源基础设施和机器学习 (ML) 工具构建的以 AI 为中心的边缘视频分析应用程序。尽管使用了最先进的 AI 和 ML 算法，但该应用程序在很大程度上依赖于预处理和后处理代码。随着以人工智能为中心的应用程序受益于加速器承诺的加速，我们发现它们对硬件和软件基础设施施加了压力：随着人工智能加速的增加，存储和网络带宽成为主要瓶颈。通过专注于 AI 应用程序，我们展示了一个专门构建的边缘数据中心可以设计用于加速 AI 的压力，而 TCO 比源自同质服务器和基础设施的 TCO 低 15%。