Skip to main content
SpringerLink
Log in

Multiple player tracking in basketball court videos

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

To build a smart basketball court, one basic task is to track players with the aid of the basketball court monitoring. This task can be regarded as a special case of the multiple object tracking (MOT) problem. But different from it, the task puts request to the existing MOT methods with both good accuracy and operational efficiency (toward real time) in the new basketball scenario. To deal with this task, we make the following attempts: (1) Considering the differences between pedestrians and basketball players and the lack of corresponding dataset for basketball players tracking, we construct a new MOT dataset under the basketball court monitoring scene, to better evaluate MOT methods in our task and to help promote future research in the related task, (2) evaluating the performance of the several candidate MOT methods on the new dataset, and (3) proposing the issues to be further addressed in this specific scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: Human trajectory prediction in crowded spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961–971 (2016)

  2. Andriyenko, A., Schindler, K.: Multi-target tracking by continuous energy minimization. In: CVPR 2011, pp. 1265–1272. IEEE (2011)

  3. Ba, S., Alameda-Pineda, X., Xompero, A., Horaud, R.: An on-line variational Bayesian model for multi-person tracking from cluttered scenes. Comput. Vis. Image Underst. 153, 64–76 (2016)

    Article  Google Scholar 

  4. Bae, S.H., Yoon, K.J.: Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 595–610 (2017)

    Article  Google Scholar 

  5. Ban, Y., Ba, S., Alameda-Pineda, X., Horaud, R.: Tracking multiple persons based on a variational Bayesian model. In: European Conference on Computer Vision, pp. 52–67. Springer (2016)

  6. Berclaz, J., Fleuret, F., Turetken, E., Fua, P.: Multiple object tracking using k-shortest paths optimization. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1806–1819 (2011)

    Article  Google Scholar 

  7. Bergmann, P., Meinhardt, T., Leal-Taixe, L.: Tracking Without Bells and Whistles (2019). arXiv preprint arXiv:1903.05625

  8. Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the clear mot metrics. J. Image Video Process. 2008, 1 (2008)

    Article  Google Scholar 

  9. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and real-time tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)

  10. Chen, B., Wang, D., Li, P., Wang, S., Lu, H.: Real-time ’actor-critic’ tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 318–334 (2018)

  11. Choi, W.: Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3029–3037 (2015)

  12. Choi, W., Savarese, S.: Multiple target tracking in world coordinate with single, minimally calibrated camera. In: European Conference on Computer Vision, pp. 553–567. Springer (2010)

  13. Choi, W., Savarese, S.: A unified framework for multi-target tracking and collective activity recognition. In: European Conference on Computer Vision, pp. 215–230. Springer (2012)

  14. Han, J., Farin, D., de With, P.: A mixed-reality system for broadcasting sports video to mobile devices. IEEE Multimed. 18(2), 72–84 (2010)

    Article  Google Scholar 

  15. Han, J., Farin, D., de With, P.H.: Broadcast court-net sports video analysis using fast 3-d camera modeling. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1628–1638 (2008)

    Article  Google Scholar 

  16. Henschel, R., Leal-Taixe, L., Cremers, D., Rosenhahn, B.: Fusion of head and full-body detectors for multi-object tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1428–1437 (2018)

  17. Jiang, H., Fels, S., Little, J.J.: A linear programming approach for multiple object tracking. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

  18. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  19. Keuper, M., Tang, S., Andres, B., Brox, T., Schiele, B.: Motion segmentation & multiple object tracking by correlation co-clustering. IEEE Trans. Pattern Anal. Mach. Intell. 42, 140–153 (2018)

    Article  Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  21. Kuhn, H.W.: The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)

    Article  MathSciNet  Google Scholar 

  22. Leal-Taixé, L., Canton-Ferrer, C., Schindler, K.: Learning by tracking: Siamese CNN for robust target association. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2016)

  23. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K.: Motchallenge 2015: Towards a Benchmark for Multi-target tracking (2015). arXiv preprint arXiv:1504.01942

  24. Leal-Taixé, L., Pons-Moll, G., Rosenhahn, B.: Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 120–127. IEEE (2011)

  25. Leal-Taixe, L., Pons-Moll, G., Rosenhahn, B.: Branch-and-price global optimization for multi-view multi-target tracking. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1987–1994. IEEE (2012)

  26. Li, T., Chang, H., Wang, M., Ni, B., Hong, R., Yan, S.: Crowded scene analysis: a survey. IEEE Trans. Circuits Syst. Video Technol. 25(3), 367–386 (2014)

    Article  Google Scholar 

  27. Li, B., Yan, J., Wu, W., Zhu, Z., Hu, X.: High performance visual tracking with Siamese region proposal network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8971–8980 (2018)

  28. Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., Kim, T.K.: Multiple Object Tracking: A Literature Review (2014). arXiv preprint arXiv:1409.7618

  29. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K.: Mot16: A Benchmark for Multi-object Tracking (2016). arXiv preprint arXiv:1603.00831

  30. Milan, A., Leal-Taixé, L., Schindler, K., Reid, I.: Joint tracking and segmentation of multiple targets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5397–5406 (2015)

  31. Pellegrini, S., Ess, A., Schindler, K., Van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 261–268. IEEE (2009)

  32. Pirsiavash, H., Ramanan, D., Fowlkes, C.C.: Globally-optimal greedy algorithms for tracking a variable number of objects. In: CVPR 2011, pp. 1201–1208. IEEE (2011)

  33. Redmon, J., Farhadi, A.: Yolov3: An Incremental Improvement (2018). arXiv preprint arXiv:1804.02767

  34. Ren, L., Lu, J., Wang, Z., Tian, Q., Zhou, J.: Collaborative deep reinforcement learning for multi-object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 586–602 (2018)

  35. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

  36. Robicquet, A., Sadeghian, A., Alahi, A., Savarese, S.: Learning social etiquette: human trajectory understanding in crowded scenes. In: European Conference on Computer Vision, pp. 549–565. Springer (2016)

  37. Schenk, M.J., Reed, D.D.: Experimental evaluation of matching via a commercially available basketball video game. J. Appl. Behav. Anal. 53, 209–221 (2019)

    Google Scholar 

  38. Scovanner, P., Tappen, M.F.: Learning pedestrian dynamics from the real world. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 381–388. IEEE (2009)

  39. Son, J., Baek, M., Cho, M., Han, B.: Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5620–5629 (2017)

  40. Tang, S., Andres, B., Andriluka, M., Schiele, B.: Multi-person tracking by multicut and deep matching. In: European Conference on Computer Vision, pp. 100–111. Springer (2016)

  41. Wang, Z., Zheng, L., Liu, Y., Wang, S.: Towards Real-Time Multi-object Tracking (2019). arXiv preprint arXiv:1909.12605

  42. Wojke, N., Bewley, A., Paulus, D.: Simple online and real-time tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649. IEEE (2017)

  43. Wu, Z., Kunz, T.H., Betke, M.: Efficient track linking methods for track graphs using network-flow and set-cover techniques. In: CVPR 2011, pp. 1185–1192. IEEE (2011)

  44. Xiang, Y., Alahi, A., Savarese, S.: Learning to track: online multi-object tracking by decision making. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4705–4713 (2015)

  45. Xu, Y., Ban, Y., Alameda-Pineda, X., Horaud, R.: Deepmot: A Differentiable Framework for Training Multiple Object Trackers (2019). arXiv preprint arXiv:1906.06618

  46. Yamaguchi, K., Berg, A.C., Ortiz, L.E., Berg, T.L.: Who are you with and where are you going? In: CVPR 2011, pp. 1345–1352. IEEE (2011)

  47. Yang, F., Choi, W., Lin, Y.: Exploit all the layers: fast and accurate CNN object detector with scale dependent pooling and cascaded rejection classifiers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2129–2137 (2016)

  48. Zhang, L., Li, Y., Nevatia, R.: Global data association for multi-object tracking using network flows. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)

  49. Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., Yang, M.H.: Online multi-object tracking with dual matching attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 366–382 (2018)

Download references

Author information

Authors and Affiliations

  1. Department of Public Physical and Art Education, Zhejiang University, Hangzhou, 310058, China

    Xubo Fu

  2. Institute of Automation, Chinese Academy of Sciences (CASIA), Beijing, 100190, China

    Kun Zhang, Changgang Wang & Chao Fan

Authors
  1. Xubo Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changgang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chao Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kun Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, X., Zhang, K., Wang, C. et al. Multiple player tracking in basketball court videos. J Real-Time Image Proc 17, 1811–1828 (2020). https://doi.org/10.1007/s11554-020-00968-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00968-x

Keywords

Search

Navigation