Modern convolutional neural networks (CNNs) in computer vision are trained on a large number of images from numerous categories to form rich discriminative feature extractors. Inference using such models on resource-constrained Internet-of-Things (IoT) platforms poses a challenge and an opportunity. Having limited computation, storage, and energy budgets, most IoT platforms are not capable of hosting such compute intensive models. However, typical IoT applications demand detection of a relatively small number of categories, albeit the specific categories of interest may change at runtime as the context evolves dynamically. In this letter, we take advantage of the opportunity to address the challenge. Specifically, we develop a novel transformation to the architecture of a given CNN, so that the majority of the inference workload is allocated to class-specific disjoint branches, which can be dynamically executed or skipped, based on the context, to fulfill the application requirements. Experiments demonstrate that our approach preserves the classification accuracy for the classes of interest, while proportionally decreasing the model complexity and inference workload.

中文翻译：

---

Octopus：物联网应用的上下文感知 CNN 推理

计算机视觉中的现代卷积神经网络 (CNN) 在来自众多类别的大量图像上进行训练，以形成丰富的判别特征提取器。在资源受限的物联网 (IoT) 平台上使用此类模型进行推理既是挑战也是机遇。由于计算、存储和能源预算有限，大多数物联网平台无法托管此类计算密集型模型。然而，典型的 IoT 应用程序需要检测相对较少的类别，尽管随着上下文的动态演变，感兴趣的特定类别可能会在运行时发生变化。在这封信中，我们利用这个机会来应对挑战。具体来说，我们对给定 CNN 的架构进行了新的转换，以便将大部分推理工作负载分配给特定于类的不相交分支，这些分支可以根据上下文动态执行或跳过，以满足应用程序要求。实验表明，我们的方法保留了感兴趣类别的分类准确性，同时按比例降低了模型复杂性和推理工作量。