Abstract
Proper understanding and prediction of driving behavior of surrounding vehicles are one of the most significant requirements for automated driving especially when it comes to safety on a highway. In this paper, we propose a two-layer memory-attention hierarchical model (MAHM) for driving-behavior recognition and motion prediction. This model is based on the human driver’s thinking as well as on brain physiology, i.e., working memory and the selective-attention mechanism. The first layer is a hidden Markov model (HMM), which is used to achieve efficient recognition of driving behavior. The second layer is a memory-attention recurrent neural network (MARNN) for motion prediction, which derives the data from vehicles of interest as input according to driving behavior. Finally, the experimental analysis is performed on the real-data NGSIM US-101 and HighD datasets for highway-driving scenes. We report our results from three perspectives: accuracy of driving-behavior classification, error of predicted trajectories, and execution time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altché, F. and De La Fortelle, A. (2017). An LSTM network for highway trajectory prediction. IEEE 20th Int. Conf. Intelligent Transportation Systems. (ITSC). Yokohama, Kanagawa, Japan.
Ba, J., Hinton, G., Mnih, V., Leibo, J. and Ionescu, C. (2016). Using fast weights to attend to the recent past. arXiv: 1610.06258.
Brand, M., Oliver, N. and Pentland, A. (1997). Coupled hidden markov models for complex action recognition. Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition. San Juan, PR, USA.
Chaer, W. S., Bishop, R. H. and Ghosh, J. (1997). A mixture-of-experts framework for adaptive kalman filtering. IEEE Trans. Systems, Man, and Cybernetics, Part B. (Cybernetics) 27, 3, 452–464.
Chen, S., Zhang, S., Shang, J., Chen, B. and Zheng, N. (2017). Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans. Cognitive and Developmental Systems 11, 1, 13–25.
Choi, S., Kim, J. and Yeo, H. (2019). Attention-based recurrent neural network for urban vehicle trajectory prediction. Procedia Computer Science, 151, 327–334.
Colyar, J. and Halkias, J. (2007). US Highway 101 Dataset. Federal High-way Administration. (FHWA), Tech. Rep. FHWA-HRT-07-030.
Dagli, I., Brost, M. and Breuel, G. (2002). Action recognition and prediction for driver assistance systems using dynamic belief networks. Net. ObjectDays: Int. Conf. Object-Oriented and Internet-Based Technologies, Concepts, and Applications for a Networked World. Erfurt, Germany.
Deo, N. and Trivedi, M. M. (2018). Convolutional social pooling for vehicle trajectory prediction. Proc. IEEE Conf. Computer Vision and Pattern Recognition: Workshops. Salt Lake City, UT, USA.
Deo, Nachiket, Rangesh, A. and Trivedi, M. M. (2018). How would surround vehicles move? A unified framework for maneuver classification and motion prediction. IEEE Trans. Intelligent Vehicles 3, 2, 129–140.
Gindele, T., Brechtel, S. and Dillmann, R. (2010). A probabilistic model for estimating driver behaviors and vehicle trajectories in traffic environments. 13th Int. IEEE Conf. Intelligent Transportation Systems. Funchal, Madeira Island, Portugal.
Gindele, T., Brechtel, S. and Dillmann, R. (2013). Learning context sensitive behavior models from observations for predicting traffic situations. 16th Int. IEEE Conf. Intelligent Transportation Systems. (ITSC 2013). Den Haag, Netherlands.
Hoermann, S., Stumper, D. and Dietmayer, K. (2017). Probabilistic long-term prediction for autonomous vehicles. IEEE Intelligent Vehicles Symp. (IV). Los Angeles, CA, USA.
Hou, H., Jin, L., Niu, Q., Sun, Y. and Lu, M. (2011). Driver intention recognition method using continuous hidden markov model. Int. J. Computer Intelligent Systems 4, 3, 386–393.
Hu, Y., Zhan, W. and Tomizuka, M. (2018). Probabilistic prediction of vehicle semantic intention and motion. 2018 IEEE Intelligent Vehicles Symp. (IV). Changshu, Suzhou, China.
Jain, A., Koppula, H. S., Raghavan, B., Soh, S. and Saxena, A. (2015). Car that knows before you do: Anticipating maneuvers via learning temporal driving models. Proc. IEEE Int. Conf. Computer Vision. (ICCV). Santiago, Chile.
Kim, H., Kim, D., Kim, G., Cho, J. and Huh, K. (2020). Multi-head attention-based probabilistic vehicle trajectory prediction. 2020 IEEE Intelligent Vehicles Symp. (IV). Las Vegas, NV, USA.
Krajewski, R., Bock, J., Kloeker, L. and Eckstein, L. (2018). The HighD dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems. 21st Int. Conf. Intelligent Transportation Systems. (ITSC). Maui, HI, USA.
Kumar, P., Perrollaz, M., Lefevre, S. and Laugier, C. (2013). Learning-based approach for online lane change intention prediction. 2013 IEEE Intelligent Vehicles Symp. (IV). Gold Coast, Queensland, Australia.
Lenz, D., Diehl, F., Le, M. T. and Knoll, A. (2017). Deep neural networks for markovian interactive scene prediction in highway scenarios. IEEE Intelligent Vehicles Symp. (IV). Redondo Beach, CA, USA.
Li, D. and Gao, H. (2018). A hardware platform framework for an intelligent vehicle based on a driving brain. Engineering 4, 4, 464–470.
Liebner, M., Baumann, M., Klanner, F. and Stiller, C. (2012). Driver intent inference at urban intersections using the intelligent driver model. IEEE Intelligent Vehicles Symp. Alcala de Henares, Spain.
Mandalia, H. M. and Salvucci, D. D. (2005). Using support vector machines for lane change detection. Proc. Human Factors and Ergonomics Society Annual Meeting 49, 22, 1965–1969.
Mannering, F. L. and Bhat, C. R. (2014). Analytic methods in accident research: Methodological frontier and future directions. Analytic Methods in Accident Research, 1, 1–22.
Messaoud, K., Yahiaoui, I., Verroust-Blondet, A. and Nashashibi, F. (2019a). Non-local social pooling for vehicle trajectory prediction. IEEE Intelligent Vehicles Symp. (IV). Paris, France.
Messaoud, K., Yahiaoui, I., Verroust-Blondet, A. and Nashashibi, F. (2019b). Relational recurrent neural networks for vehicle trajectory prediction. IEEE Conf. Intelligent Transportation Systems. (ITSC). Auckland, New Zealand.
Meyer-Delius, D., Plagemann, C. and Burgard, W. (2009). Probabilistic situation recognition for vehicular traffic scenarios. IEEE Int. Conf. Robotics and Automation. Kobe, Hyogo, Japan.
Scheel, O., Nagaraja, N. S., Schwarz, L., Navab, N. and Tombari, F. (2019). Attention-based lane change prediction. Int. Conf. Robotics and Automation (ICRA). Montreal, QC, Canada.
Schlechtriemen, J., Wirthmueller, F., Wedel, A., Breuel, G. and Kuhnert, K-D. (2015). When will it change the lane? A probabilistic regression approach for rarely occurring events. IEEE Intelligent Vehicles Symp. (IV). Seoul, Korea.
Schreier, M., Willert, V. and Adamy, J. (2014). Bayesian, maneuver-based, longterm trajectory prediction and criticality assessment for driver assistance systems. 17th Int. IEEE Conf. Intelligent Transportation Systems. (ITSC). Qingdao, Shandong, China.
Schubert, R., Richter, E. and Wanielik, G. (2008). Comparison and evaluation of advanced motion models for vehicle tracking. 11th Int. Conf. Information Fusion. Cologne, North Rhine-Westphalia, Germany.
Streubel, T. and Hoffmann, K. H. (2014). Prediction of driver intended path at intersections. IEEE Intelligent Vehicles Symp. Proc., Ypsilanti, Michigan, USA.
Tran, D., Sheng, W., Liu, L. and Liu, M. (2015). A hidden markov model based driver intention prediction system. IEEE Int. Conf. Cyber Technology in Automation, Control, and Intelligent Systems. (CYBER). Shenyang, Liaoning, China.
Welch and Lloyd R. (2003). Hidden Markov models and the Baum-Welch algorithm. IEEE Information Theory Society Newsletter 53, 4, 1, 10–13.
Yang, D., Zhang, L. and Zhang, X. (2016). The review on the current highway motor vehicle crashes in China. Medical J. Communications 30, 5, 443–446.
Zhang, Y., Lin, Q., Wang, J., Verwer, S. and Dolan, J. M. (2018). Lane-change intention estimation for car-following control in autonomous driving. IEEE Trans. Intelligent Vehicles 3, 3, 276–286.
Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant No. 61701348. It was also supported by the Ministry of Science and Technology under Grant No. 2016YFB0100901 and No. 2018YFB 0105101. The authors would like to thank TUEV SUED for the kind support. We are also grateful for the efforts from our colleagues in Sino-German Center of Intelligent Systems.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yin, H., Wang, J., Lin, J. et al. A Memory-Attention Hierarchical Model for Driving-Behavior Recognition and Motion Prediction. Int.J Automot. Technol. 22, 895–908 (2021). https://doi.org/10.1007/s12239-021-0081-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12239-021-0081-8