Skip to main content
SpringerLink
Log in

Abnormal event detection via the analysis of multi-frame optical flow information

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Security surveillance of public scene is closely relevant to routine safety of individual. Under the stimulus of this concern, abnormal event detection is becoming one of the most important tasks in computer vision and video processing. In this paper, we propose a new algorithm to address the visual abnormal detection problem. Our algorithm decouples the problem into a feature descriptor extraction process, followed by an AutoEncoder based network called cascade deep AutoEncoder (CDA). The movement information is represented by a novel descriptor capturing the multi-frame optical flow information. And then, the feature descriptor of the normal samples is fed into the CDA network for training. Finally, the abnormal samples are distinguished by the reconstruction error of the CDA in the testing procedure. We validate the proposed method on several video surveillance datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. PETS. Performance evaluation of tracking and surveillance benchmark data. University of Reading, 2009

    Google Scholar 

  2. UMN. Unusual crowd activity dataset. University of Minnesota. 2006

    Google Scholar 

  3. UCSD. Anomaly Detection Dataset. University of California, San Diego, 2010

    Google Scholar 

  4. Wu S, Moore B E, Shah M. Chaotic invariants of lagrangian particle trajectories for anomaly detection in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2054–2060

    Google Scholar 

  5. Surana A, Nakhmani A, Tannenbaum A. Anomaly detection in videos: a dynamical systems approach. In: Proceedings of the 52nd IEEE Annual Conference on Decision and Control. 2013, 6489–6495

    Chapter  Google Scholar 

  6. Zhou S, Shen W, Zeng D, Zhang Z. Unusual event detection in crowded scenes by trajectory analysis. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2015, 1300–1304

    Google Scholar 

  7. Yang W, Gao Y, Cao L. Trasmil: a local anomaly detection framework based on trajectory segmentation and multi-instance learning. Computer Vision and Image Understanding, 2013, 117(10): 1273–1286

    Article  Google Scholar 

  8. Raghavendra R, Del Bue A, Cristani M, Murino V. Optimizing interaction force for global anomaly detection in crowded scenes. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2011, 136–143

    Google Scholar 

  9. Zhu X, Liu J, Wang J, Fang Y, Lu H. Anomaly detection in crowded scene via appearance and dynamics joint modeling. In: Proceedings of the 19th IEEE International Conference on Image Processing. 2012, 2705–2708

    Google Scholar 

  10. De la Calle Silos F, Diaz I G, de Maria E D. Mid-level feature set for specific event and anomaly detection in crowded scenes. In: Proceedings of the 20th IEEE International Conference on Image Processing. 2013, 4001–4005

    Google Scholar 

  11. Wang J, Xu Z. Spatio-temporal texture modelling for real-time crowd anomaly detection. Computer Vision and Image Understanding, 2016, 114: 177–187

    Article  Google Scholar 

  12. Wang T, Snoussi H. Detection of abnormal visual events via global optical flow orientation histogram. IEEE Transactions on Information Forensics and Security, 2014, 9(6): 988–998

    Article  Google Scholar 

  13. Wang T, Qiao M, Zhu A, Niu Y, Li C, Snoussi H. Abnormal event detection via covariance matrix for optical flow based feature. Multimedia Tools and Applications, 2018, 77(13): 17375–17395

    Article  Google Scholar 

  14. Zhang Y, Liu X, Chang M C, Ge W, Chen T. Spatio-temporal phrases for activity recognition. In: Proceedings of European Conference on Computer Vision. 2012, 707–721

    Google Scholar 

  15. Wang T, Chen Y, Zhang M, Chen J, Snoussi H. Internal transfer learning for improving performance in human action recognition for small datasets. IEEE Access, 2017, 5: 17627–17633

    Article  Google Scholar 

  16. Yuan Y, Fang J, Wang Q. Online anomaly detection in crowd scenes via structure analysis. IEEE Transactions on Cybernetics, 2015, 45(3): 562–575

    Article  Google Scholar 

  17. Xiong G, Cheng J, Wu X, Chen Y L, Ou Y, Xu Y. An energy model approach to people counting for abnormal crowd behavior detection. Neurocomputing, 2012, 83: 121–135

    Article  Google Scholar 

  18. Chiappino S, Morerio P, Marcenaro L, Regazzoni C S. A bio-inspired knowledge representation method for anomaly detection in cognitive video surveillance systems. In: Proceedings of the 16th International Conference on Information Fusion. 2013, 242–249

    Google Scholar 

  19. Mahadevan V, Li W, Bhalodia V, Vasconcelos N. Anomaly detection in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 1975–1981

    Google Scholar 

  20. Li W, Mahadevan V, Vasconcelos N. Anomaly detection and localization in crowded scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1): 18–32

    Article  Google Scholar 

  21. Hu Y, Zhang Y, Davis L. Unsupervised abnormal crowd activity detection using semiparametric scan statistic. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2013, 767–774

    Google Scholar 

  22. Cong Y, Yuan J, Liu J. Sparse reconstruction cost for abnormal event detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 3449–3456

    Google Scholar 

  23. Cong Y, Yuan J, Liu J. Abnormal event detection in crowded scenes using sparse representation. Pattern Recognition, 2013, 46(7): 1851–1864

    Article  Google Scholar 

  24. Zhao B, Li F F, Xing E P. Online detection of unusual events in videos via dynamic sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 3313–3320

    Google Scholar 

  25. Cui X, Liu Q, Gao M, Metaxas D N. Abnormal detection using interaction energy potentials. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 3161–3167

    Google Scholar 

  26. Li N, Zhang Z. Abnormal crowd behavior detection using topological methods. In: Proceedings of the 12th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. 2011, 13–18

    Google Scholar 

  27. Chen D Y, Huang P C. Visual-based human crowds behavior analysis based on graph modeling and matching. IEEE Sensors Journal, 2013, 13(6): 2129–2138

    Article  Google Scholar 

  28. Wang L, Dong M. Detection of abnormal human behavior using a matrix approximation-based approach. In: Proceedings of the 13th International Conference on Machine Learning and Applications. 2014, 324–329

    Google Scholar 

  29. Wang L, Dong M. Real-time detection of abnormal crowd behavior using a matrix approximation-based approach. In: Proceedings of the 19th IEEE International Conference on Image Processing. 2012, 2701–2704

    Google Scholar 

  30. Lu C, Shi J, Jia J. Abnormal event detection at 150 FPS in matlab. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 2720–2727

    Google Scholar 

  31. Rao A S, Gubbi J, Rajasegarar S, Marusic S. Detection of anomalous crowd behaviour using hyperspherical clustering. In: Proceedings of the IEEE International Conference on Digital Image Computing: Techniques and Applications. 2014, 1–8

    Google Scholar 

  32. Ren W, Li G, Sun B, Huang K. Unsupervised kernel learning for abnormal events detection. Visual Computer International Journal of Computer Graphics, 2015, 31(3): 245–255

    Google Scholar 

  33. Horn B K, Schunck B G. Determining optical flow. Artificial Intelligence, 1981, 17(1): 185–203

    Article  Google Scholar 

  34. Sun D, Roth S, Black M J. Secrets of optical flow estimation and their principles. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010, 2432–2439

    Google Scholar 

  35. Wang T, Chen Y, Qiao M, Snoussi H. A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 2018, 94(9): 3465–3471

    Article  Google Scholar 

  36. Mehran R, Oyama A, Shah M. Abnormal crowd behavior detection using social force model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2009, 935–942

    Google Scholar 

  37. Shi Y, Gao Y, Wang R. Real-time abnormal event detection in complicated scenes. In: Proceedings of the 20th International Conference on Pattern Recognition. 2010, 3653–3656

    Google Scholar 

  38. Adam A, Rivlin E, Shimshoni I, Reinitz D. Robust real-time unusual event detection using multiple fixed-location monitors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(3), 555–560

    Article  Google Scholar 

  39. Chaker R, Al Aghbari Z, Junejo I N. Social network model for crowd anomaly detection and localization. Pattern Recognition, 2017, 61: 266–281

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Key R&D Program of China (2016YFE0204200), the National Natural Science Foundation of China (Grant Nos. 61503017, U1435220), the Fundamental Research Funds for the Central Universities (YWF-14-RSC-102), the Aeronautical Science Foundation of China (2016ZC51022), the ANR AutoFerm project, the Platform CAPSEC funded by Region Champagne-Ardenne and FEDER.

Author information

Authors and Affiliations

  1. School of Automation Science and Electrical Engineering, Beihang University, Beijing, 100191, China

    Tian Wang & Meina Qiao

  2. School of Computer Science and Technology, Nanjing University of Technology, Nanjing, 210094, China

    Aichun Zhu

  3. School of Instrumentation Science and Opto-electronics Engineering, Beihang University, Beijing, 100191, China

    Guangcun Shan

  4. Institute Charles Delaunay-LM2S-UMR STMR 6281 CNRS, University of Technology of Troyes, Troyes, 10010, France

    Hichem Snoussi

Authors
  1. Tian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meina Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aichun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangcun Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hichem Snoussi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangcun Shan.

Additional information

Tian Wang received the PhD degree from University of Technology of Troyes, France in 2014. He is an assistant professor at the School of Automation of Science and Electrical Engineering, Beihang University, China. His research interests include computer vision and pattern recognition.

Meina Qiao is a Master in School of Automation Science and Electrical Engineering in Beihang University, China. She is involved in abnormal events detection, action recognition and video surveillance. Her academic interests are computer vision, pattern recognition, and machine learning.

Aichun Zhu received the PhD degree from the University of Technology of Troyes, France. He is an assistant professor in the School of Computer Science and Technology, Nanjing University of Technology, China. His academic interests span computer vision and machine learning.

Guangcun Shan received the PhD degree from City University of Hong Kong, China in 2013. He has been a full professor at Beihang University, China after being selected into the National 1000-talent Youth Program of China in 2016. His research interests include the machine learning algorithm, the model design, and fabrication of

Hichem Snoussi received his PhD degrees from the University of Paris-Sud, France in 2003. Since 2010, he has been a full professor at the University of Technology of Troyes, France. His research interests include signal processing, computer vision, and machine learning.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, T., Qiao, M., Zhu, A. et al. Abnormal event detection via the analysis of multi-frame optical flow information. Front. Comput. Sci. 14, 304–313 (2020). https://doi.org/10.1007/s11704-018-7407-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-018-7407-3

Keywords

Search

Navigation