Three main requirements of a successful application of deep learning are the network architecture, a large enough training dataset, and a good optimization algorithm. In this paper we mainly focus on the optimization part. We propose a training algorithm for convolutional neural networks which makes use of both first and second order derivatives for training different layers. We utilize an approximate second order algorithm for the classification layer while we train the rest of the network with the conventional approach which is backpropagation with first order derivatives. We show that this approach helps us achieve a higher classification accuracy with a much smaller number of training iterations compared to training the whole network with gradient descent based algorithms. Moreover, although second order optimization is generally costlier, we show that the proposed approach is trained faster not only in terms of the number of iterations but also training duration. We also present the integration of CNNs with a probabilistic spatial model and apply this to the land cover classification problem in hyperspectral images. The results show that the algorithm allows us to achieve superior results with a simple network even with limited training data compared to existing approaches.

成功应用深度学习的三个主要要求是网络架构，足够大的训练数据集和良好的优化算法。在本文中，我们主要集中在优化部分。我们提出了一种卷积神经网络的训练算法，该算法利用一阶和二阶导数来训练不同的层。我们使用一个近似的二阶算法进行分类，同时使用常规方法训练网络的其余部分，该方法是使用一阶导数进行反向传播。我们证明，与使用基于梯度下降的算法训练整个网络相比，该方法可帮助我们以更少的训练迭代次数来实现更高的分类精度。而且，尽管二阶优化通常比较昂贵，我们表明，所提出的方法不仅在迭代次数方面而且在训练持续时间方面都得到了更快的训练。我们还提出了CNN与概率空间模型的集成，并将其应用于高光谱图像中的土地覆盖分类问题。结果表明，与现有方法相比，即使使用有限的训练数据，该算法也可以使我们通过简单的网络获得优异的结果。