Conditional computation for deep neural networks reduces overall computational load and improves model accuracy by running a subset of the network. In this work, we present a runtime dynamically throttleable neural network (DTNN) that can self-regulate its own performance target and computing resources by dynamically activating neurons in response to a single control signal, called utilization. We describe a generic formulation of throttleable neural networks (TNNs) by grouping and gating partial neural modules with various gating strategies. To directly optimize arbitrary application-level performance metrics and model complexity, a controller network is trained separately to predict a context-aware utilization via deep contextual bandits. Extensive experiments and comparisons on image classification and object detection tasks show that TNNs can be effectively throttled across a wide range of utilization settings, while having peak accuracy and lower cost that are comparable to corresponding vanilla architectures such as VGG, ResNet, ResNeXt, and DenseNet. We further demonstrate the effectiveness of the controller network on throttleable 3D convolutional networks (C3D) for video-based hand gesture recognition, which outperforms the vanilla C3D and all fixed utilization settings.

深度神经网络的条件计算通过运行网络的一个子集来减少整体计算负载并提高模型准确性。在这项工作中，我们提出了一个运行时动态可节流神经网络（DTNN），它可以通过响应单个控制信号动态激活神经元来自我调节自己的性能目标和计算资源，称为利用率. 我们通过使用各种门控策略对部分神经模块进行分组和门控来描述可节流神经网络 (TNN) 的通用公式。为了直接优化任意应用程序级性能指标和模型复杂性，控制器网络被单独训练以通过深度上下文强盗预测上下文感知利用率。对图像分类和目标检测任务的大量实验和比较表明，TNN 可以在广泛的使用设置中得到有效限制，同时具有与 VGG、ResNet、ResNeXt 和 DenseNet 等相应的普通架构相当的峰值精度和更低的成本. 我们进一步证明了控制器网络在基于视频的手势识别的可节流 3D 卷积网络 (C3D) 上的有效性，