This paper presents a novel optimisation method, termed Hessian-free Gradient Flow, for the optimisation of deep neural networks. The algorithm entails the design characteristics of the Truncated Newton, Conjugate Gradient and Gradient Flow method. It employs a finite difference approximation scheme to make the algorithm Hessian-free and makes use of Armijo conditions to determine the descent condition. The method is first tested on standard testing functions with a high optimisation model dimensionality. Performance on the testing functions has demonstrated the potential of the algorithm to be applied to large-scale optimisation problems. The algorithm is then tested on classification and regression tasks using real-world datasets. Comparable performance to conventional optimisers has been obtained in both cases.

本文提出了一种用于深度神经网络优化的新颖优化方法，称为无粗略的Hessian梯度流。该算法具有截断牛顿，共轭梯度和梯度流方法的设计特征。它采用有限差分逼近方案使算法不使用Hessian，并利用Armijo条件确定下降条件。该方法首先在具有高优化模型维度的标准测试功能上进行测试。测试功能的性能证明了该算法在大规模优化问题中的潜力。然后使用实际数据集在分类和回归任务上测试该算法。在两种情况下均获得了与传统优化器可比的性能。