In recent years, neural architecture search (NAS) has received intensive scientific and industrial interest due to its capability of finding a neural architecture with high accuracy for various artificial intelligence tasks such as image classification or object detection. In particular, gradient-based NAS approaches have become one of the more popular approaches thanks to their computational efficiency during the search. However, these methods often experience a mode collapse, where the quality of the found architectures is poor due to the algorithm resorting to choosing a single operation type for the entire network, or stagnating at a local minima for various datasets or search spaces. To address these defects, we present a differentiable variational inference-based NAS method for searching sparse convolutional neural networks. Our approach finds the optimal neural architecture by dropping out candidate operations in an over-parameterised supergraph using variational dropout with automatic relevance determination prior, which makes the algorithm gradually remove unnecessary operations and connections without risking mode collapse. The evaluation is conducted through searching two types of convolutional cells that shape the neural network for classifying different image datasets. Our method finds diverse network cells, while showing state-of-the-art accuracy with up to $3 \times$ fewer parameters.

中文翻译：

---

VINNAS：基于变分推理的神经网络架构搜索

近年来，神经体系结构搜索（NAS）由于能够为各种人工智能任务（例如图像分类或对象检测）找到高精度的神经体系结构，因此受到了科学界和工业界的广泛关注。尤其是，基于梯度的NAS方法由于其在搜索过程中的计算效率而已成为最受欢迎的方法之一。但是，这些方法经常会遇到模式崩溃的情况，在这种情况下，所发现的体系结构的质量很差，这是因为该算法只能为整个网络选择一种操作类型，或者对于各种数据集或搜索空间都只能停留在局部最小值。为了解决这些缺陷，我们提出了一种基于差分变分推理的NAS方法来搜索稀疏卷积神经网络。我们的方法通过使用带有自动相关性确定的先验变量落差来在过参数化的超图中落选候选操作，从而找到最佳的神经体系结构，这使算法逐渐删除了不必要的操作和连接，而不会冒模式崩溃的风险。通过搜索塑造神经网络以对不同图像数据集进行分类的两种类型的卷积细胞进行评估。我们的方法发现了各种各样的网络单元，同时显示了最先进的准确性，参数减少了多达$ 3乘以$。这使得算法逐渐消除了不必要的操作和连接，而不会冒模式崩溃的风险。通过搜索塑造神经网络以对不同图像数据集进行分类的两种类型的卷积细胞进行评估。我们的方法发现了各种各样的网络单元，同时显示了最先进的准确性，参数减少了多达$ 3乘以$。这使得算法逐渐消除了不必要的操作和连接，而不会冒模式崩溃的风险。通过搜索塑造神经网络以对不同图像数据集进行分类的两种类型的卷积细胞进行评估。我们的方法发现了各种各样的网络单元，同时显示了最先进的准确性，参数减少了多达$ 3乘以$。