Deep learning has promoted remarkable progress in various tasks while the effort devoted to these hand-crafting neural networks has motivated so-called neural architecture search (NAS) to discover them automatically. Recent aging evolution (AE) automatic search algorithm turns to discard the oldest model in population and finds image classifiers beyond manual design. However, it achieves a low speed of convergence. A nonaging evolution (NAE) algorithm tends to neglect the worst architecture in population to accelerate the search process whereas it obtains a lower performance compared with AE. To address this issue, in this letter, we propose to use an optimized evolution algorithm for recurrent NAS (EvoRNAS) by setting a probability $\epsilon $ to remove the worst or oldest model in population alternatively, which can balance the performance and time length. Besides, parameter sharing mechanism is introduced in our approach due to the heavy cost of evaluating the candidate models in both AE and NAE. Furthermore, we train the sharing parameters only once instead of many epochs like ENAS, which makes the evaluation of candidate models faster. On Penn Treebank, we first explore different $\epsilon $ in EvoRNAS and find the best value suited for the learning task, which is also better than AE and NAE. Second, the optimal cell found by EvoRNAS can achieve state-of-the-art performance within only 0.6 GPU hours, which is $20 \,\, \times $ and $40 \,\, \times $ faster than ENAS and DARTS. Moreover, the transferability of the learned architecture to WikiText-2 also shows strong performance compared with ENAS or DARTS.

中文翻译：

---

进化循环神经架构搜索

深度学习促进了各种任务的显着进步，而致力于这些手工制作的神经网络的努力促使所谓的神经架构搜索 (NAS) 自动发现它们。最近的老化进化 (AE) 自动搜索算法转向丢弃人口中最古老的模型，并找到超出手动设计的图像分类器。然而，它实现了低收敛速度。非老化进化 (NAE) 算法倾向于忽略种群中最差的架构以加速搜索过程，但与 AE 相比，它获得的性能较低。为了解决这个问题，在这封信中，我们建议通过设置概率为循环 NAS (EvoRNAS) 使用优化的进化算法 $\epsilon $ 交替删除种群中最差或最旧的模型，可以平衡性能和时间长度。此外，由于在 AE 和 NAE 中评估候选模型的成本很高，我们的方法中引入了参数共享机制。此外，我们只训练共享参数一次，而不是像 ENAS 这样的多次训练，这使得对候选模型的评估更快。在 Penn Treebank，我们首先探索不同的 $\epsilon $ 在 EvoRNAS 中找到适合学习任务的最佳值，这也优于 AE 和 NAE。其次，EvoRNAS 找到的最优单元可以在仅 0.6 个 GPU 小时内达到最先进的性能，这是 $20 \,\, \times $ 和 $40 \,\, \times $ 比 ENAS 和 DARTS 更快。此外，与 ENAS 或 DARTS 相比，学习到的架构到 WikiText-2 的可迁移性也显示出强大的性能。