Skip to main content
Log in

Motion trajectory prediction based on a CNN-LSTM sequential model

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Accurate monitoring the surrounding environment is an important research direction in the field of unmanned systems such as bio-robotics, and has attracted much research attention in recent years. The trajectories of surrounding vehicles should be predicted accurately in space and time to realize active defense and running safety of an unmanned system. However, there is uncertainty and uncontrollability in the process of trajectory prediction of surrounding obstacles. In this study, we propose a trajectory prediction method based on a sequential model, that fuses two neural networks of a convolutional neural network (CNN) and a long short-term memory network (LSTM). First, a box plot is used to detect and eliminate abnormal values of vehicle trajectories, and valid trajectory data are obtained. Second, the trajectories of surrounding vehicles are predicted by merging the characteristics of CNN space expansion and LSTM time expansion; the hyper-parameters of the model are optimized according to a grid search algorithm, which satisfies the double-precision prediction requirement in space and time. Finally, data from next generation simulation (NGSIM) and Creteil roundabout in France are taken as test cases; the correctness and rationality of the method are verified by prediction error indicators. Experimental results demonstrate that the proposed CNN-LSTM method is more accurate and features a shorter time cost, which meets the prediction requirements and provides an effective method for the safe operation of unmanned systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. He W, Zhang S. Control design for nonlinear flexible wings of a robotic aircraft. IEEE Trans Contr Syst Technol, 2017, 25: 351–357

    Article  Google Scholar 

  2. Cao Y, Ma L, Zhang Y. Application of fuzzy predictive control technology in automatic train operation. Cluster Comput, 2019, 22: 14135–14144

    Article  Google Scholar 

  3. Yu W K, Zhao C H. Online fault diagnosis for industrial processes with Bayesian network-based probabilistic ensemble learning strategy. IEEE Trans Automat Sci Eng, 2019, 16: 1922–1932

    Article  Google Scholar 

  4. Chai Z, Zhao C H. A fine-grained adversarial network method for cross-domain industrial fault diagnosis. IEEE Trans Automat Sci Eng, 2020. doi: https://doi.org/10.1109/TASE.2019.2957232

  5. Cao Y, Zhang Y Z, Wen T, et al. Research on dynamic nonlinear input prediction of fault diagnosis based on fractional differential operator equation in high-speed train control system. Chaos, 2019, 29: 013130

    Article  Google Scholar 

  6. Xie G, Peng X, Li X, et al. Remaining useful life prediction of lithium-ion battery based on an improved particle filter algorithm. Can J Chem Eng, 2019, 41: 23675

    Google Scholar 

  7. Xie G, Li X, Peng X, et al. Estimating the probability density function of remaining useful life for Wiener degradation process with uncertain parameters. Int J Control Autom Syst, 2019, 17: 2734–2745

    Article  Google Scholar 

  8. Xie G, Jin Y Z, Hei X H, et al. Adaptive identification of time-varying environmental parameters in train dynamics model. Acta Autom Sin, 2020. doi: https://doi.org/10.16383/j.aas.c190215

  9. Qiao S J, Jin K, Han N, et al. Trajectory prediction algorithm based on gaussian mixture model. J Softw, 2015, 26: 1048–1063

    MathSciNet  Google Scholar 

  10. Qiao S J, Shen D Y, Wang X T, et al. A self-adaptive parameter selection trajectory prediction approach via hidden Markov models. IEEE Trans Intell Transp Syst, 2015, 16: 284–296

    Article  Google Scholar 

  11. Wu P J, Yang W T, Yu C, et al. Trajectory prediction method for high precision servo control system (in Chinese). Electric Mach Control, 2014, 18: 1–5

    Google Scholar 

  12. Houenou A, Bonnifait P, Cherfaoui V, et al. Vehicle trajectory prediction based on motion model and maneuver recognition. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, 2013. 4363–4369

  13. Fei R, Li S S, Hei X H, et al. A motion simulation model for road network based crowdsourced map datum. J Intell Fuzzy Syst, 2020, 38: 391–407

    Article  Google Scholar 

  14. Xie G, Sun L L, Wen T, et al. Adaptive transition probability matrix-based parallel IMM algorithm. IEEE Trans Syst Man Cybern Syst, 2019. doi: https://doi.org/10.1109/TSMC.2019.2922305

  15. Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436

    Article  Google Scholar 

  16. Li D Y, Liu M, Zhao F, et al. Challenges and countermeasures of interaction in autonomous vehicles. Sci China Inf Sci, 2019, 62: 050201

    Article  Google Scholar 

  17. Deo N, Trivedi M M. Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018

  18. Park S H, Kim B D, Kang C M, et al. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), Changshu, 2018

  19. Altché F, Arnaud D L F. An LSTM network for highway trajectory prediction. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 353–359

  20. Zhang P, Yang T, Liu Y N, et al. QAR data feature extraction and prediction based on CNN-LSTM (in Chinese). Appl Res Comput, 2019, 36: 2958–2961

    Google Scholar 

  21. Kim B D, Kang C M, Lee S H, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 399–404

  22. Yang J, Xie G, Yang Y X, et al. Deep model integrated with data correlation analysis for multiple intermittent faults diagnosis. ISA Trans, 2019, 95: 306–319

    Article  Google Scholar 

  23. Yang J, Xie G, Yang Y X, et al. An improved deep network for intelligent diagnosis of machinery faults with similar features. IEEJ Trans Elec Electron Eng, 2019, 14: 1851–1864

    Article  Google Scholar 

  24. Cao Y, Sun Y K, Xie G, et al. Fault diagnosis of train plug door based on a hybrid criterion for IMFs selection and fractional wavelet package energy entropy. IEEE Trans Veh Technol, 2019, 68: 7544–7551

    Article  Google Scholar 

  25. Zhang S, Dong Y, Ouyang Y, et al. Adaptive neural control for robotic manipulators with output constraints and uncertainties. IEEE Trans Neural Netw Learn Syst, 2018, 29: 5554–5564

    Article  MathSciNet  Google Scholar 

  26. He W, Dong Y. Adaptive fuzzy neural network control for a constrained robot using impedance learning. IEEE Trans Neural Netw Learn Syst, 2018, 29: 1174–1186

    Article  Google Scholar 

  27. Xue Z B, Liu J C, Wu Z X, et al. Development and path planning of a novel unmanned surface vehicle system and its application to exploitation of Qarhan Salt Lake. Sci China Inf Sci, 2019, 62: 084202

    Article  Google Scholar 

  28. Thiemann C, Treiber M, Kesting A. Estimating acceleration and lane-changing dynamics based on NGSIM trajectory Data. Transport Res Record J Transport Res Board, 2008, 2088: 90–101

    Article  Google Scholar 

  29. Li P, Dargaville R, Cao Y, et al. Storage aided system property enhancing and hybrid robust smoothing for large-scale PV systems. IEEE Trans Smart Grid, 2017, 8: 2871–2879

    Article  Google Scholar 

  30. Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets. Neural Computat, 2006, 18: 1527–1554

    Article  MathSciNet  Google Scholar 

  31. Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proc IEEE, 1998, 86: 2278–2324

    Article  Google Scholar 

  32. Chan T A, Hermeking H, Lengauer C, et al. 14-3-3σ is required to prevent mitotic catastrophe after DNA damage. Nature, 1999, 401: 616–620

    Article  Google Scholar 

  33. Gers F A, Schmidhuber J, Cummins F. Learning to forget: continual prediction with LSTM. Neural Computat, 2000, 12: 2451–2471

    Article  Google Scholar 

  34. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computat, 1997, 9: 1735–1780

    Article  Google Scholar 

  35. Wang Y X, Liu J Q, Misic J, et al. Assessing optimizer impact on DNN model sensitivity to adversarial examples. IEEE Access, 2019, 7: 152766–152776

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key R&D Program of China (Grant No. 2018YFB1201500), National Natural Science Foundation of China (Grant Nos. 61873201, 61773313, U1734210), Key Research and Development Program of Shaanxi Province (Grant No. 2018GY-139), Natural Science Foundation of Shaanxi Provincial Department of Education (Grant No. 19JS051), CERNET Innovation Project (Grant No. NGII20161201), and Scientific and Technological Planning Project of Beilin District of Xi’an (Grant No. GX1819).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Guo Xie or Xinhong Hei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, G., Shangguan, A., Fei, R. et al. Motion trajectory prediction based on a CNN-LSTM sequential model. Sci. China Inf. Sci. 63, 212207 (2020). https://doi.org/10.1007/s11432-019-2761-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-019-2761-y

Keywords

Navigation