Skip to main content
SpringerLink
Log in

Crowd flow estimation from calibrated cameras

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Many crowd analysis methods rely on optical flow techniques to estimate the main moving directions. In this work, we propose a crowd flow filtering approach for calibrated cameras that can be coupled to any generic optical flow method. It projects the input optical flow to the world coordinate system, performs a local motion analysis exploring a Social Forces Model and then projects the filtered flow back onto the image plane. The method was tested on publicly available datasets involving crowded scenarios used in conjunction with different optical flow methods, and results indicate that the proposed filtering method provides coherent crowd flows when coupled to the tested methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://lmb.informatik.uni-freiburg.de/index.php.

  2. http://cs.brown.edu/~dqsun/research/index.html.

  3. http://www.cvg.rdg.ac.uk/PETS2009.

References

  1. Abdallah, A.C.B., Gouiffès, M., Lacassagne, L.: A modular system for global and local abnormal event detection and categorization in videos. Mach. Vis. Appl. 27(4), 463–481 (2016). https://doi.org/10.1007/s00138-016-0752-z

    Article  Google Scholar 

  2. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: Proceedings of the 10th European Conference on Computer Vision: Part II, ECCV’08, pp. 1–14. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-88688-4_1

  3. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)

    Article  Google Scholar 

  4. Bouguet, J.Y.: Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm. Intel Corp. 5(1–10), 4 (2001)

    Google Scholar 

  5. Bradski, G.: Opencv: examples of use and new applications in stereo, recognition and tracking. In: Proc. of International Conference on Vision Interface (2002)

  6. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500–513 (2011)

    Article  Google Scholar 

  7. Chen, D.Y., Huang, P.C.: Motion-based unusual event detection in human crowds. J. Vis. Commun. Image Represent. 22(2), 178–186 (2011)

    Article  Google Scholar 

  8. Cheriyadat, A.M., Radke, R.J.: Detecting dominant motions in dense crowds. IEEE J. Sel. Top. Signal Process. 2(4), 568–581 (2008)

    Article  Google Scholar 

  9. de Almeida, I.R., Jung, C.R.: Change detection in human crowds. In: 2013 26th SIBGRAPI-Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 63–69. IEEE (2013)

  10. de Almeida, I.R., Cassol, V.J., Badler, N.I., Musse, S.R., Jung, C.R.: Detection of global and local motion changes in human crowds. IEEE Trans. Circuits Syst. Video Technol. 27(3), 603–612 (2017)

    Article  Google Scholar 

  11. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., Brox, T.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)

  12. Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. In: Image Analysis, pp. 363–370 (2003)

  13. Fortun, D., Bouthemy, P., Kervrann, C.: Optical flow modeling and computation: a survey. Comput. Vis. Image Underst. 134, 1–21 (2015). https://doi.org/10.1016/j.cviu.2015.02.008.

    Article  MATH  Google Scholar 

  14. Hall, E.T.: The Hidden Dimension, vol. 609. Doubleday, Garden City, NY (1966)

    Google Scholar 

  15. He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013)

    Article  Google Scholar 

  16. Helbing, D., Molnár, P.: Self-organization phenomena in pedestrian crowds. In: Self-organization of Complex Structures: From Individual to Collective Dynamics. Citeseer, Princeton (1997)

  17. Helbing, D., Farkas, I., Vicsek, T.: Simulating dynamical features of escape panic. Nature 407(6803), 487–490 (2000)

    Article  Google Scholar 

  18. Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)

    Article  Google Scholar 

  19. Hui, T.W., Tang, X., Change Loy, C.: Liteflownet: a lightweight convolutional neural network for optical flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8981–8989 (2018)

  20. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470 (2017)

  21. Jung, C.R.: Efficient background subtraction and shadow removal for monochromatic video sequences. IEEE Trans. Multimed 11(3), 571–577 (2009)

    Article  Google Scholar 

  22. Kajo, I., Malik, A.S., Kamel, N.: An evaluation of optical flow algorithms for crowd analytics in surveillance system. In: 2016 6th International Conference on Intelligent and Advanced Systems (ICIAS), pp. 1–6 (2016). https://doi.org/10.1109/ICIAS.2016.7824064

  23. Kajo, I., Kamel, N., Malik, A.S.: An adaptive block-based matching algorithm for crowd motion sequences. Multimed. Tools Appl. (2017). https://doi.org/10.1007/s11042-016-4327-9

    Article  Google Scholar 

  24. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1446–1453. IEEE (2009)

  25. Kusakunniran, W., Li, H., Zhang, J.: A direct method to self-calibrate a surveillance camera by observing a walking pedestrian. In: 2009 Digital Image Computing: Techniques and Applications, pp. 250–255. IEEE (2009)

  26. Lim, M.K., Kok, V.J., Loy, C.C., Chan, C.S.: Crowd saliency detection via global similarity structure. In: 2014 22nd International Conference on Pattern Recognition, pp. 3957–3962 (2014)

  27. Liu, C., Freeman, W.T., Adelson, E.H., Weiss, Y.: Human-assisted motion annotation. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

  28. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 674–679. Morgan Kaufmann Publishers, Burlington (1981)

  29. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009)

  30. Mousavi, H., Mohammadi, S., Perina, A., Chellali, R., Murino, V.: Analyzing tracklets for the detection of abnormal crowd behavior. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 148–155. IEEE (2015)

  31. Moussaïd, M., Helbing, D., Theraulaz, G.: How simple rules determine pedestrian behavior and crowd disasters. Proc. Nat. Acad. Sci. 108(17), 6884–6888 (2011)

    Article  Google Scholar 

  32. Renno, J., Orwell, J., Jones, G.A.: Learning surveillance tracking models for the self-calibrated ground plane. In: BMVC, pp. 1–10 (2002)

  33. Rodriguez, M., Laptev, I., Sivic, J., Audibert, J.Y.: Density-aware person detection and tracking in crowds. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2423–2430. IEEE (2011)

  34. Shi, J., et al.: Good features to track. In: Proceedings of1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994. CVPR’94, pp. 593–600. IEEE (1994)

  35. Solmaz, B., Moore, B.E., Shah, M.: Identifying behaviors in crowd scenes using stability analysis for dynamical systems. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 2064–2070 (2012)

    Article  Google Scholar 

  36. Sun, D., Roth, S., Black, M.J.: Secrets of optical flow estimation and their principles. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2432–2439. IEEE (2010)

  37. Sun, D., Roth, S., Black, M.J.: A quantitative analysis of current practices in optical flow estimation and the principles behind them. Int. J. Comput. Vis. 106(2), 115–137 (2014)

    Article  Google Scholar 

  38. Tu, Z., Xie, W., Zhang, D., Poppe, R., Veltkamp, R.C., Li, B., Yuan, J.: A survey of variational and CNN-based optical flow techniques. Signal Process. Image Commun. 72, 9–24 (2019)

    Article  Google Scholar 

  39. Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: European Conference on Computer Vision. Springer, pp. 589–600 (2006)

  40. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: 2013 IEEE International Conference on Computer Vision, pp. 1385–1392 (2013). https://doi.org/10.1109/ICCV.2013.175

  41. Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13–18 June 2010, pp. 2054–2060 (2010)

  42. Zhang, X., He, H., Cao, S., Liu, H.: Flow field texture representation-based motion segmentation for crowd counting. Mach. Vis. Appl. 26(7), 871–883 (2015). https://doi.org/10.1007/s00138-015-0703-0

    Article  Google Scholar 

  43. Zhao, W., Zhang, Z., Huang, K.: Gestalt laws based tracklets analysis for human crowd understanding. Pattern Recognit. (2017). https://doi.org/10.1016/j.patcog.2017.06.020

Download references

Author information

Authors and Affiliations

  1. Federal University of Rio Grande do Sul, Porto Alegre, Brazil

    Igor Almeida & Claudio Jung

Authors
  1. Igor Almeida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudio Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Igor Almeida.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almeida, I., Jung, C. Crowd flow estimation from calibrated cameras. Machine Vision and Applications 32, 7 (2021). https://doi.org/10.1007/s00138-020-01132-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01132-y

Keywords

Search

Navigation