Abstract
Under the influence of deep learning, many trackers have emerged recently. Among them, Siamese network reaches a pleasant balance between accuracy and speed, but its tracking performance still lags behind other trackers. In this paper, we have proposed a Hierarchical Correlation Siamese Network (HC-Siam) for object tracking. The tracker uses convolutional features of each layer to compare the correlation and identifies the position of the tracking object depending on the greatest correlation. Meanwhile, we have designed a Correlation Attention Module (CA-Module). For various objects, this module can assign different weights to the hierarchical correlation and help the network choose the distinct correlation from the hierarchical correlation. Besides, objects’ size and scale constantly varied during tracking, we claimed to use the separate scale factor in the wide and high directions to decrease the deformation of bounding boxes and increase the accuracy of our tracker. On the OTB dataset, the accuracy of HC-Siam is 6.5% higher than the baseline, and the speed of our tracker can reach 85 fps. On the VOT dataset, HC-Siam also has better performance in speed and accuracy.
Similar content being viewed by others
References
Liu L, Xing J, Ai H, Ruan X (2012) Hand posture recognition using finger geometric feature. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE, pp 565–568
Lee K -H, Hwang J-N (2015) On-road pedestrian tracking across multiple driving recorders. IEEE Trans Multimed 17(9):1429–1438
Tang S, Andriluka M, Andres B, Schiele B (2017) Multiple people tracking by lifted multicut and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3539–3548
Yilmaz A, Javed O, Shah M (2006) Object tracking: A survey. ACM Computing Survey 38(4):13
Jia X, Lu H, Yang M-H (2012) Visual tracking via adaptive structural local sparse appearance model. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 1822–1829
Danelljan M, Bhat G, Shahbaz Khan F, Felsberg M (2017) ECO: Efficient convolution operators for tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6638–6646
Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 2544–2550
Kiani Galoogahi H, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 1135–1143
Lukezic A, Vojir T, Cehovin Zajc L, Matas J, Kristan M (2017) Discriminative correlation filter with channel and spatial reliability. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6309–6318
Zhang M, Xing J, Gao J, Hu W (2015) Robust visual tracking using joint scale-spatial correlation filters. In: 2015 IEEE international conference on image processing (ICIP), IEEE, pp 1468–1472
Zhang M, Xing J, Gao J, Shi X, Wang Q, Hu W (2015) Joint scale-spatial correlation tracking with adaptive rotation estimation. In: Proceedings of the IEEE international conference on computer vision workshops, pp 32–40
Nam H, Han B (2016) Learning multi-domain convolutional neural networks for visual tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4293–4302
Sun C, Wang D, Lu H, Yang M-H (2018) Learning spatial-aware regressions for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8962–8970
Wang L, Ouyang W, Wang X, Lu H (2015) Visual tracking with fully convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 3119–3127
Henriques J F, Caseiro R, Martins P, Batista J (2014) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583–596
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Li F-F (2015) Imagenet large scale visual recognition challenge. International journal of computer vision 115(3):211–252
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Held D, Thrun S, Savarese S (2016) Learning to track at 100 fps with deep regression networks. In: European conference on computer vision. Springer, pp 749–765
Bertinetto L, Valmadre J, Henriques JF, Vedaldi A, Torr PH (2016) Fully-convolutional siamese networks for object tracking. In: European conference on computer vision. Springer, pp 850–865
Wang X, Li C, Luo B, Tang J (2018) Sint++: robust visual tracking via adversarial positive instance generation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4864–4873
Yang T, Chan AB (2017) Recurrent filter learning for visual tracking. In: Proceedings of the IEEE international conference on computer vision, pp 2010–2019
He A, Luo C, Tian X, Zeng W (2018) A twofold siamese network for real-time object tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4834–4843
Zhang Y, Wang L, Qi J, Wang D, Feng M, Lu H (2018) Structured siamese network for real-time visual tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 351–366
Wang Q, Gao J, Xing J, Zhang M, Hu W (2017) Discriminant correlation filters network for visual tracking. arXiv
Huang C, Lucey S, Ramanan D (2017) Learning policies for adaptive tracking with deep feature cascades. In: Proceedings of the IEEE international conference on computer vision, pp 105–114
Guo Q, Feng W, Zhou C, Huang R, Wan L, Wang S (2017) Learning dynamic siamese network for visual object tracking. In: Proceedings of the IEEE international conference on computer vision, pp 1763–1771
Yang T, Chan AB (2018) Learning dynamic memory networks for object tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 152—167
Dong X, Shen J (2018) Triplet loss in siamese network for object tracking. In: Proceedings of the European conference on computer vision (ECCV), pp 459–474
Wu Y, Lim J, Yang M-H (2013) Online object tracking: A benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2411–2418
Kristan M, Matas J, Leonardis A, Felsberg M, Cehovin L, Fernandez G, Vojir T, Hager G, Nebehay G, Pflugfelder R (2015) The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE international conference on computer vision workshops, pp 1–23
Qi Y, Qin L, Zhang S, Huang Q, Yao H (2019) Robust visual tracking via scale-and-state-awareness. Neurocomputing 329(2019):75–85
Wang Q, Gao J, Xing J, Zhang M (2017) Hu WDJapa Discriminant correlation filters network for visual tracking. arXiv
Danelljan M, Robinson A, Khan FS, Felsberg M (2016) Beyond correlation filters: Learning continuous convolution operators for visual tracking. In: European conference on computer vision. Springer, pp 472–488
Lee H, Kim D (2018) Salient region-based online object tracking. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1170–1177
Chi Z, Li H, Lu H, Yang M -H (2017) Dual deep network for visual tracking. IEEE Trans. Image Process 26(4):2005–2015
Nam H, Baek M, Han B (2016) Modeling and propagating cnns in a tree structure for visual tracking. arXiv preprint arXiv:1608.07242
Bertinetto L, Valmadre J, Golodetz S, Miksik O (2016) Torr PH staple: Complementary learners for real-time tracking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1401–1409
Zhu G, Porikli F, Li H (2016) Beyond local search: Tracking objects everywhere with instance-specific proposals. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 943–951
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Meng, Y., Deng, Z., Zhao, K. et al. Hierarchical correlation siamese network for real-time object tracking. Appl Intell 51, 3202–3211 (2021). https://doi.org/10.1007/s10489-020-01992-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01992-x