Skip to main content
SpringerLink
Log in

Graph-based topic models for trajectory clustering in crowd videos

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Probabilistic topic modelings, such as latent Dirichlet allocation (LDA) and correlated topic models (CTM), have recently emerged as powerful statistical tools for processing video content. They share an important property, i.e., using a common set of topics to model all data. However, such property can be too restrictive for modeling complex visual data such as crowd scenes where multiple fields of heterogeneous data jointly provide rich information about objects and events. This paper proposes graph-based extensions of LDA and CTM, referred to as GLDA and GCTM, to learn and analyze motion patterns by trajectory clustering in a highly cluttered and crowded environment. Unlike previous works that relied on a scene prior, we apply a spatio-temporal graph to uncover the spatial and temporal coherence between the trajectories of crowd motion during the learning process. The presented models advance the conventional approaches by integrating a manifold-based clustering as initialization and iterative statistical inference as optimization. The output of GLDA and GCTM are mid-level features that represent the motion patterns used later to generate trajectory clusters. Experiments on three different datasets show the effectiveness of the approaches in trajectory clustering and crowd motion modeling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Both LDA and CTM were implemented following the approach in [24].

  2. We used the publicly available code from the authors’ website [38].

References

  1. Alghamdi, M., Gotoh, Y.: Graph-based correlated topic model for motion patterns analysis in crowded scenes from tracklets. In: The British Machine Vision Conference (BMVC) (2018)

  2. Alghamdi, M., Gotoh, Y.: Graph-based correlated topic model for trajectory clustering in crowded videos. In: The IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1029–1037 (2018)

  3. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: ECCV, pp. 1–14 (2008)

  4. Ali, Y., Zafar, B., Simsim, M.: Estimation of density levels in the holy mosque from a network of cameras. In: Traffic and Granular Flow, pp. 27–34 (2016)

  5. Blei, D., Lafferty, J.: A correlated topic model of science. Ann. Appl. Stat. 1, 17–35 (2007)

    Article  MathSciNet  Google Scholar 

  6. Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  7. Borg, I., Groenen, P.: Modern Multidimensional Scaling: Theory and Applications. Springer, Berlin (2005)

    MATH  Google Scholar 

  8. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) Computer Vision—ECCV, pp. 25–36. Springer, Berlin (2004)

    Google Scholar 

  9. Burceanu, E., Leordeanu, M.: Learning a robust society of tracking parts using co-occurrence constraints. In: Leal-Taixé, L., Roth, S. (eds.) Computer Vision—ECCV Workshops, pp. 162–178. Springer International Publishing, Cham (2019)

    Google Scholar 

  10. Chen, M., Wang, Q., Li, X.: Patch-based topic model for group detection. Sci. China Inf. Sci. 60(11), 113101–113107 (2017)

    Article  Google Scholar 

  11. Gariel, M., Srivastava, A.N., Feron, E.: Trajectory clustering and an application to airspace monitoring. IEEE Trans. Intell. Transp. Syst. 12(4), 1511–1524 (2011)

    Article  Google Scholar 

  12. Kratz, L., Nishino, K.: Tracking pedestrians using local spatio-temporal motion patterns in extremely crowded scenes. IEEE Trans. PAMI 34(5), 987–1002 (2012)

    Article  Google Scholar 

  13. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: The IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 2169–2178 (2006)

  14. Leal-Taixé, L., Milan, A., Schindler, K., Cremers, D., Reid, I.D., Roth, S.: Tracking the trackers: an analysis of the state of the art in multiple object tracking. arXiv:1704.02781 (2017)

  15. Lee, S., Baker, J., Song, J., Wetherbe, J.: An empirical comparison of four text mining methods. In: Proceedings of the Annual Hawaii International Conference on System Sciences, pp. 1–10 (2010)

  16. Lin, W., Mi, Y., Wang, W., Wu, J., Wang, J., Mei, T.: A diffusion and clustering-based approach for finding coherent motions and understanding crowd scenes. IEEE Trans. Image Process. 25(4), 1674–1687 (2016)

    Article  MathSciNet  Google Scholar 

  17. Lu, W., Wei, X., Xing, W., Liu, W.: Trajectory-based motion pattern analysis of crowds. Neurocomputing 247, 213–223 (2017)

    Article  Google Scholar 

  18. Luo, W., Xing, J., Zhang, X., Zhao, X., Kim, T.: Multiple object tracking: a literature review. In: CoRR (2014)

  19. Mao, Y., Zhong, H., Qi, H., Ping, P., Li, X.: An adaptive trajectory clustering method based on grid and density in mobile pattern analysis. In: Sensors (2017)

  20. Moberts, B., Vilanova, A., van Wijk, J.: Evaluation of fiber clustering methods for diffusion tensor imaging. In: IEEE Visualization, pp. 65–72 (2005)

  21. Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: The IEEE 12th International Conference on Computer Vision, pp. 261–268 (2009)

  22. Rabiee, H., Mousavi, H., Nabi, M., Ravanbakhsh, M.: Detection and localization of crowd behavior using a novel tracklet-based model. Int. J. Mach. Learn. Cybernet. 9(12), 1999–2010 (2018)

    Article  Google Scholar 

  23. Raghavendra, R., Del Bue, A., Cristani, M., Murino, V.: Abnormal crowd behavior detection by social force optimization. In: Salah, A.A., Lepri, B. (eds) Human Behavior Understanding, pp. 134–145 (2011)

  24. Rodriguez, M., Ali, S., Kanade, T.: Tracking in unstructured crowded scenes. In: ICCV, pp. 1389–1396 (2009)

  25. Rodriguez, M., Sivic, J., Laptev, I., Audibert, J.Y.: Data-driven crowd analysis in videos. In: The International Conference on Computer Vision (ICCV), pp. 1235–1242 (2011)

  26. Salti, S., Cavallaro, A., Di Stefano, L.: Adaptive appearance modeling for video tracking: survey and evaluation. IEEE Trans. Image Process. 21(10), 4334–4348 (2012)

    Article  MathSciNet  Google Scholar 

  27. Serre, T., Wolf, L., Poggio, T.: Object recognition with features inspired by visual cortex. In: CVPR, pp. 994–1000 (2005)

  28. Sharma, R., Guha, T.: A trajectory clustering approach to crowd flow segmentation in videos. In: ICIP, pp. 1200–1204 (2016)

  29. Silva, V.D., Tenenbaum, J.B.: Global versus local methods in nonlinear dimensionality reduction. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 721–728. MIT Press, Cambridge (2003)

    Google Scholar 

  30. Smeulders, A.W.M., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M.: Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1442–1468 (2014)

    Article  Google Scholar 

  31. Tomasi, C., Kanade, T.: Detection and tracking of point features. In: Technical Reports. Carnegie Mellon University (1991)

  32. van der Maaten, L., Postma, E., van den Herik, H.: Dimensionality reduction: a comparative review. J. Mach. Learn. Res. 10, 66–71 (2009)

    Google Scholar 

  33. Vedaldi, A., Fulkerson, B.: Vlfeat: an open and portable library of computer vision algorithms. In: International Conference on Multimedia, pp. 1469–1472 (2010)

  34. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: CVPR, pp. 3360–3367 (2010)

  35. Xu, H., Zhou, Y., Lin, W., Zha, H.: Unsupervised trajectory clustering via adaptive multi-kernel-based shrinkage. In: The IEEE International Conference on Computer Vision (ICCV) pp. 4328–4336 (2015)

  36. Yuan, G., Xia, S., Zhang, L., Zhou, Y., Ji, C.: An efficient trajectory-clustering algorithm based on an index tree. Trans. Inst. Meas. Control 34, 850–861 (2012)

    Article  Google Scholar 

  37. Zhou, B., Tang, X., Zhang, H., Wang, X.: Measuring crowd collectiveness. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1586–1599 (2014)

    Article  Google Scholar 

  38. Zhou, B., Wang, X., Tang, X.: Random field topic model for semantic region analysis in crowded scenes from tracklets. In: CVPR, pp. 3441–3448 (2011)

  39. Zou, J., Ye, Q., Cui, Y., Doermann, D., Jiao, J.: A belief based correlated topic model for trajectory clustering in crowded video scenes. In: ICPR, pp. 2543–2548 (2014)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Umm Al-Qura University, Mecca, Saudi Arabia

    Manal Al Ghamdi

  2. Department of Computer Science, University of Sheffield, Sheffield, UK

    Yoshihiko Gotoh

Authors
  1. Manal Al Ghamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoshihiko Gotoh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manal Al Ghamdi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al Ghamdi, M., Gotoh, Y. Graph-based topic models for trajectory clustering in crowd videos. Machine Vision and Applications 31, 39 (2020). https://doi.org/10.1007/s00138-020-01092-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01092-3

Keywords

Search

Navigation