Long short-term memory (LSTM) has transformed both machine learning and neurocomputing fields. According to several online sources, this model has improved Google’s speech recognition, greatly improved machine translations on Google Translate, and the answers of Amazon’s Alexa. This neural system is also employed by Facebook, reaching over 4 billion LSTM-based translations per day as of 2017. Interestingly, recurrent neural networks had shown a rather discrete performance until LSTM showed up. One reason for the success of this recurrent network lies in its ability to handle the exploding/vanishing gradient problem, which stands as a difficult issue to be circumvented when training recurrent or very deep neural networks. In this paper, we present a comprehensive review that covers LSTM’s formulation and training, relevant applications reported in the literature and code resources implementing this model for a toy example.

中文翻译：

---

长短期记忆模型综述

长短期记忆 (LSTM) 已经改变了机器学习和神经计算领域。据多位在线消息人士透露，该模型改进了谷歌的语音识别能力，大大改进了谷歌翻译的机器翻译，以及亚马逊 Alexa 的回答。Facebook 也采用了这种神经系统，截至 2017 年，每天基于 LSTM 的翻译量达到超过 40 亿次。有趣的是，循环神经网络在 LSTM 出现之前表现出相当离散的性能。这种循环网络成功的一个原因在于它能够处理梯度爆炸/消失问题，这是在训练循环或非常深的神经网络时需要规避的一个难题。在本文中，我们对 LSTM 的制定和训练进行了全面的回顾，