当前位置: X-MOL 学术Neurocomputing › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Scalable NAS with factorizable architectural parameters
Neurocomputing ( IF 6 ) Pub Date : 2022-07-16 , DOI: 10.1016/j.neucom.2022.07.053
Lanfei Wang , Lingxi Xie , Kaili Zhao , Jun Guo , Qi Tian

Neural Architecture Search (NAS) replaces manually designed networks with automatically searched networks and has become a hot topic in machine learning and computer vision. One key factor of NAS, scaling up the search space by adding operators, could help bring about more possibilities for effective architectures. However, existing works require high search costs and are prone to make confused selections caused by competition among similar operators. This paper presents a scalable NAS that utilizes a factorized method, which improves existing art in the following aspects. (1) Our work explores a broader search space. We construct an ample search space through the cartesian product of activation operators and regular operators. (2) Our work experiences a limited computation burden even though searching in a more extensive search space. We factorize a search space into two subspaces and adopt different architectural parameters to control corresponding subspaces. (3) Our work avoids competition among similarly combined operators (e.g., ReLU & sep-conv-3x3 and ReLU6 & sep-conv-3x3 and PReLU & sep-conv-3x3). That is because our architectural parameters are optimized sequentially. Experimental results show that our approach achieves SOTA on CIFAR10 (2.38% test error) and ImageNet (23.8% test error). Furthermore, the excellent performance of 35.8% (AP) and 55.7% (AP50) on COCO further proves the superiority of our factorized method.



中文翻译:

具有可分解架构参数的可扩展 NAS

神经架构搜索(NAS)用自动搜索网络代替人工设计的网络,成为机器学习和计算机视觉领域的热门话题。NAS 的一个关键因素是通过添加运算符来扩大搜索空间,这有助于为有效架构带来更多可能性。然而,现有的作品需要较高的搜索成本,并且容易因类似运营商之间的竞争而导致混淆选择。本文提出了一种利用分解方法的可扩展NAS,它在以下方面改进了现有技术。(1) 我们的工作探索了更广阔的搜索空间。我们通过笛卡尔积构造了一个充足的搜索空间激活运算符和常规运算符。(2) 即使在更广泛的搜索空间中搜索,我们的工作也会遇到有限的计算负担。我们将一个搜索空间分解为两个子空间,并采用不同的架构参数来控制相应的子空间。(3) 我们的工作避免了相似组合算子之间的竞争(例如ReLU & sep-conv-3x3ReLU6 & sep-conv-3x3PReLU & sep-conv-3x3)。那是因为我们的架构参数是按顺序优化的。实验结果表明,我们的方法在 CIFAR10 (2.38%测试错误)和 ImageNet(23.8%测试错误)。此外,出色的性能35.8%美联社)和55.7%(美联社50) 在 COCO 上进一步证明了我们分解方法的优越性。

更新日期:2022-07-16
down
wechat
bug