In this paper, we propose deep learning frameworks based on the randomized neural network. Inspired by the principles of Random Vector Functional Link (RVFL) network, we present a deep RVFL network (dRVFL) with stacked layers. The parameters of the hidden layers of the dRVFL are randomly generated within a suitable range and kept fixed while the output weights are computed using the closed-form solution as in a standard RVFL network. We also propose an ensemble deep network (edRVFL) that can be regarded as a marriage of ensemble learning with deep learning. Unlike traditional ensembling approaches that require training several models independently from scratch, edRVFL is obtained by training a single dRVFL network once. Both dRVFL and edRVFL frameworks are generic and can be used with any RVFL variant. To illustrate this, we integrate the deep learning RVFL networks with a recently proposed sparse pre-trained RVFL (SP-RVFL). Experiments on 46 tabular UCI classification datasets and 12 sparse datasets demonstrate that the proposed deep RVFL networks outperform state-of-the-art deep feed-forward neural networks (FNNs).

在本文中，我们提出了基于随机神经网络的深度学习框架。受随机向量功能链接（RVFL）网络原理的启发，我们提出了一个具有堆叠层的深层RVFL网络（dRVFL）。dRVFL的隐藏层的参数在合适的范围内随机生成，并保持固定，同时使用标准RVFL网络中的闭式解计算输出权重。我们还提出了集成深度网络（edRVFL），可以将其视为集成学习与深度学习的结合。不同于需要从头开始独立训练多个模型的传统集成方法，edRVFL是通过一次训练单个dRVFL网络而获得的。dRVFL和edRVFL框架都是通用的，可以与任何RVFL变体一起使用。为了说明这一点，我们将深度学习RVFL网络与最近提出的稀疏预训练RVFL（SP-RVFL）集成在一起。在46个表格UCI分类数据集和12个稀疏数据集上进行的实验表明，提出的深层RVFL网络优于最新的深层前馈神经网络（FNN）。