Abstract Extreme Learning Machine (ELM) is a powerful and favorable classifier used in various applications due to its fast speed and good generalization capability. However, when dealing with complex visual tasks, the shallow architecture of ELM makes it infeasible to have good performance when raw image data are directly fed in as input. Therefore, several works tried to make use of deep neural networks (DNNs) to extract features before ELM classification. On the other hand, when the depth of DNN is too deep, the ELM classifier may suffer from overfitting problem. To solve this issue, a novel deep and wide feature based Extreme Learning Machine (DW-ELM) has been proposed in this research work. We show that the overfitting problem can be largely remedied by employing a “widened” convolutional neural network (CNN) for feature extraction, in the sense that the number of feature maps for each convolutional layer is increased by factor of k compared to a reference model, i.e. deep residual networks (ResNets). While the wide design of residual networks has been shown to benefit image classification in terms of accuracy and efficiency, its application for feature extraction is not fully investigated. We provide an extensive experimental study in this work, showing that when combined with ELM that serves as a classifier, using wide ResNets (WRNs) for feature extraction can produce a performance leap on all benchmark datasets compared to a plain end-to-end trained network over a wide range of selections regardless of architecture choices and ELM designs, while normal ResNets as feature extractors do not provide a performance gain. The gap is even larger when fewer training iterations are employed. This indicates that a good feature extractor for ELM must be wide and deep. Experiments conducted on five benchmark datasets (CIFAR-100, CIFAR-10, STL-10, Flower-102 and Fashion-MNIST) have shown significant accuracy enhancement as well as training stability of the proposed DW-ELM. Ablation studies also demonstrate that the ELM classifier is an important component for DW-ELM which enables superior performance compared with SVM based image classification approaches.

中文翻译：

---

基于深度和宽特征的图像分类极限学习机

摘要 极限学习机（ELM）由于其速度快、泛化能力强而被广泛应用于各种应用中。然而，在处理复杂的视觉任务时，ELM 的浅层架构使得直接将原始图像数据作为输入输入时无法获得良好的性能。因此，一些作品试图在 ELM 分类之前利用深度神经网络 (DNN) 来提取特征。另一方面，当 DNN 的深度太深时，ELM 分类器可能会出现过拟合问题。为了解决这个问题，在这项研究工作中提出了一种新颖的基于深度和广度特征的极限学习机（DW-ELM）。我们表明，通过采用“加宽”卷积神经网络 (CNN) 进行特征提取，可以在很大程度上解决过拟合问题，从某种意义上说，与参考模型（即深度残差网络 (ResNet)）相比，每个卷积层的特征图数量增加了 k 倍。虽然残差网络的广泛设计已被证明在准确性和效率方面有利于图像分类，但尚未充分研究其在特征提取中的应用。我们在这项工作中提供了广泛的实验研究，表明当与作为分类器的 ELM 结合使用时，与普通的端到端训练相比，使用宽 ResNets (WRN) 进行特征提取可以在所有基准数据集上产生性能飞跃无论体系结构选择和 ELM 设计如何，网络都可以在广泛的选择范围内进行，而作为特征提取器的普通 ResNet 不会提供性能提升。当使用较少的训练迭代时，差距甚至更大。这表明一个好的 ELM 特征提取器必须是广而深的。在五个基准数据集（CIFAR-100、CIFAR-10、STL-10、Flower-102 和 Fashion-MNIST）上进行的实验表明，所提出的 DW-ELM 具有显着的准确性增强和训练稳定性。消融研究还表明，ELM 分类器是 DW-ELM 的一个重要组成部分，与基于 SVM 的图像分类方法相比，它具有更高的性能。