Artificial neural network training algorithms aim to optimize the network parameters regarding the pre-defined cost function. Gradient-based artificial neural network training algorithms support iterative learning and have gained immense popularity for training different artificial neural networks end-to-end. However, training through gradient methods is time-consuming. Another family of training algorithms is based on the Moore–Penrose inverse, which is much faster than many other gradient methods. Nevertheless, most of those algorithms are non-iterative and thus do not support mini-batch learning in nature. This work extends two non-iterative Moore–Penrose inverse-based training algorithms to enable online sequential learning: a single-hidden-layer autoencoder training algorithm and a sub-network-based classifier training algorithm. We further present an approach that uses the proposed autoencoder for self-supervised dimension reduction and then uses the proposed classifier for supervised classification. The experimental results show that the proposed approach achieves satisfactory classification accuracy on many benchmark datasets with extremely low time consumption (up to 50 times faster than the support vector machine on CIFAR 10 dataset).

人工神经网络训练算法旨在优化关于预定义成本函数的网络参数。基于梯度的人工神经网络训练算法支持迭代学习，并在端到端训练不同的人工神经网络方面广受欢迎。但是，通过梯度方法进行训练非常耗时。另一类训练算法基于 Moore-Penrose 逆，它比许多其他梯度方法快得多。尽管如此，这些算法中的大多数都是非迭代的，因此本质上不支持小批量学习。这项工作扩展了两种非迭代的基于 Moore-Penrose 逆的训练算法，以实现在线顺序学习：单隐藏层自动编码器训练算法和基于子网络的分类器训练算法。我们进一步提出了一种方法，该方法使用所提出的自动编码器进行自监督降维，然后使用所提出的分类器进行监督分类。实验结果表明，所提出的方法在许多基准数据集上以极低的时间消耗（比 CIFAR 10 数据集上的支持向量机快 50 倍）达到了令人满意的分类精度。