当前位置: X-MOL 学术Cogn. Syst. Res. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A modified conjugate gradient-based Elman neural network
Cognitive Systems Research ( IF 3.9 ) Pub Date : 2021-03-03 , DOI: 10.1016/j.cogsys.2021.02.001
Long Li , Xuetao Xie , Tao Gao , Jian Wang

Elman recurrent network is a representative model with feedback mechanism. Although gradient descent method has been widely used to train Elman network, it frequently leads to slow convergence. According to optimization theory, conjugate gradient method is an alternative strategy in searching the descent direction during training. In this paper, an efficient conjugate gradient method has been presented to reach the optimal solution in two ways: (1) constructing a more effective conjugate coefficient, (2) determining adaptive learning rates in terms of the generalized Armijo search method. Experiments show that the performance of the new algorithm is superior to traditional algorithms, such as gradient descent method and conjugate gradient method. In particular, the new algorithm has better performance than the evolutionary algorithm. Finally, we prove the weak and strong convergence of the presented algorithm, i.e., the gradient norm of the error function with respect to the weight vectors converges to zero and the weight sequence approaches a fixed optimal point.



中文翻译:

改进的基于共轭梯度的Elman神经网络

Elman递归网络是具有反馈机制的代表性模型。尽管梯度下降法已被广泛用于训练Elman网络,但它经常导致收敛缓慢。根据优化理论,共轭梯度法是训练过程中搜索下降方向的一种替代策略。本文提出了一种有效的共轭梯度法,可以通过两种方式达到最优解:(1)构建更有效的共轭系数;(2)根据广义Armijo搜索方法确定自适应学习率。实验表明,新算法的性能优于传统的梯度下降法和共轭梯度法。特别地,新算法比进化算法具有更好的性能。最后,

更新日期:2021-03-21
down
wechat
bug