Regarding the sequence learning of neural networks, there exists a problem of how to capture long-term dependencies and alleviate the gradient vanishing phenomenon. To manage this problem, we proposed a neural network with random connections via a scheme of a neural architecture search. First, a dense network was designed and trained to construct a search space, and then another network was generated by random sampling in the space, whose skip connections could transmit information directly over multiple periods and capture long-term dependencies more efficiently. Moreover, we devised a novel cell structure that required less memory and computational power than the structures of long short-term memories (LSTMs), and finally, we performed a special initialization scheme on the cell parameters, which could permit unhindered gradient propagation on the time axis at the beginning of training. In the experiments, we evaluated four sequential tasks: adding, copying, frequency discrimination, and image classification; we also adopted several state-of-the-art methods for comparison. The experimental results demonstrated that our proposed model achieved the best performance.

中文翻译：

---

神经架构搜索具有随机跳过连接的高效网络

关于神经网络的序列学习，存在如何捕获长期依赖性并减轻梯度消失现象的问题。为了解决这个问题，我们通过神经体系结构搜索的方案提出了一种具有随机连接的神经网络。首先，设计并训练了一个密集的网络来构建搜索空间，然后在该空间中通过随机采样生成另一个网络，该网络的跳过连接可以在多个周期内直接传输信息，并更有效地捕获长期依赖性。此外，我们设计了一种新颖的单元结构，该结构比长短期记忆（LSTM）的结构需要更少的内存和计算能力，最后，我们对单元参数执行了特殊的初始化方案，这样可以在训练开始时在时间轴上不受阻碍地传播梯度。在实验中，我们评估了四个顺序的任务：添加，复制，频率歧视和图像分类；我们还采用了几种最先进的方法进行比较。实验结果表明，我们提出的模型取得了最佳性能。