Abstract
Detecting pedestrians, especially under heavy occlusion, is a challenging computer vision problem with numerous real-world applications. This paper introduces a novel approach, termed as PSC-Net, for occluded pedestrian detection. The proposed PSC-Net contains a dedicated module that is designed to explicitly capture both inter and intra-part co-occurrence information of different pedestrian body parts through a graph convolutional network (GCN). Both inter and intra-part co-occurrence information contribute towards improving the feature representation for handling varying level of occlusions, ranging from partial to severe occlusions. Our PSC-Net exploits the topological structure of pedestrian and does not require part-based annotations or additional visible bounding-box (VBB) information to learn part spatial co-occurrence. Comprehensive experiments are performed on three challenging datasets: CityPersons, Caltech, and CrowdHuman datasets. Particularly, in terms of log-average miss rates and with the same backbone and input scale as those of the state-of-the-art MGAN, the proposed PSC-Net achieves absolute gains of 4.0% and 3.4% over MGAN on the heavy occlusion subsets of CityPersons and Caltech test sets, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhang S, Benenson R, Schiele B. Citypersons: a diverse dataset for pedestrian detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017
Zhang Z J, Pang Y W. CGNet: cross-guidance network for semantic segmentation. Sci China Inf Sci, 2020, 63: 120104
Sun H Q, Pang Y W. GlanceNets-efficient convolutional neural networks with adaptive hard example mining. Sci China Inf Sci, 2018, 61: 109101
Ma S, Pang Y W, Pan J, et al. Preserving details in semantics-aware context for scene parsing. Sci China Inf Sci, 2020, 63: 120106
Liu W, Liao S, Hu W, et al. Learning efficient single-stage pedestrian detectors by asymptotic localization fitting. In: Proceedings of European Conference on Computer Vision, 2018
Noh J, Lee S, Kim B, et al. Improving occlusion and hard negative handling for single-stage pedestrian detectors. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018
Brazil G, Liu X. Pedestrian detection with autoregressive network phases. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019
Liu S, Huang D, Wang Y, et al. Adaptive NMS: refining pedestrian detection in a crowd. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019
Zhou C, Yuan J. Bi-box regression for pedestrian detection and occlusion estimation. In: Proceedings of European Conference on Computer Vision, 2018
Cao J, Pang Y, Han J, et al. Taking a look at small-scale pedestrians and occluded pedestrians. IEEE Trans Image Process, 2020, 29: 3143–3152
Cao J, Pang Y, Zhao S, et al. High-level semantic networks for multi-scale object detection. IEEE Trans Circuits Syst Video Technol, 2019. doi: https://doi.org/10.1109/TCSVT.2019.2950526
Zhou C, Yang M, Yuan J, et al. Discriminative feature transformation for occluded pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2019
Pang Y, Xie J, Khan M, et al. Mask-guided attention network for occluded pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2019
Zhang S, Yang J, Schiele B, et al. Occluded pedestrian detection through guided attention in CNNs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018
Zhang S, Wen L, Bian X, et al. Occlusion-aware R-CNN: detecting pedestrians in a crowd. In: Proceedings of European Conference on Computer Vision, 2018
Brazil G, Xi Y, Liu X. Illuminating pedestrians via simultaneous detection and segmentation. In: Proceedings of IEEE International Conference on Computer Vision, 2017
Wang X, Xiao T, Jiang Y, et al. Repulsion loss: detecting pedestrians in a crowd. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018
Mao J, Xiao T, Jiang Y, et al. What can help pedestrian detection? In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017
Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of Conference and Workshop on Neural Information Processing Systems, 2015
Tian Y, Luo P, Wang X, et al. Deep learning strong parts for pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2015
Zhou C, Yuan J. Multi-label learning of part detectors for heavily occluded pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2017
Ouyang W, Wang X. Joint deep learning for pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2013
Mathias M, Benenson R, Timofte R, et al. Handling occlusions with Franken-classifiers. In: Proceedings of IEEE International Conference on Computer Vision, 2013
Ouyang W, Zeng X, Wang X. Modeling mutual visibility relationship in pedestrian detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2013
Mikolajczyk K, Schmid C, Zisserman A. Human detection based on a probabilistic assembly of robust part detectors. In: Proceedings of European Conference on Computer Vision, 2004
Mohan A, Papageorgiou C, Poggio T. Example-based object detection in images by components. IEEE Trans Pattern Anal Machine Intell, 2001, 23: 349–361
Zhou C, Yuan J. Learning to integrate occlusion-specific detectors for heavily occluded pedestrian detection. In: Proceedings of Asian Conference on Computer Vision, 2016
Biederman I, Mezzanotte R J, Rabinowitz J C. Scene perception: detecting and judging objects undergoing relational violations. Cognitive Psychol, 1982, 14: 143–177
Bar M, Ullman S. Spatial context in recognition. Perception, 1996, 25: 343–352
Galleguillos C, Rabinovich A, Belongie S. Object categorization using co-occurrence, location and appearance. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2008
Cai Z, Fan Q, Feris R, et al. A unified multi-scale deep convolutional neural network for fast object detection. In: Proceedings of European Conference on Computer Vision, 2016
Kipf T, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of International Conference on Learning Representations, 2017
Li Q, Han Z, Wu X. Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018
Dollar P, Wojek C, Schiele B, et al. Pedestrian detection: an evaluation of the state of the art. IEEE Trans Pattern Anal Mach Intell, 2012, 34: 743–761
Shao S, Zhao Z, Li B. CrowdHuman: a benchmark for detecting human in a crowd. 2018. ArXiv: 1805.00123
Kingma D, Ba J. Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations, 2014
Karen S, Andrew Z. Very deep convolutional networks for large-scale image recognition. 2014. ArXiv: 1409.1556
Song T, Sun L, Xie D, et al. Small-scale pedestrian detection based on topological line localization and temporal feature aggregation. In: Proceedings of European Conference on Computer Vision, 2018
Liu W, Liao S, Ren W, et al. High-level semantic feature detection: a new perspective for pedestrian detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019
Cai Z, Vasconcelos N. Cascade R-CNN: high quality object detection and instance segmentation. 2019. ArXiv: 1906.09756
Cai Z, Saberian M, Vasconcelos N. Learning complexity-aware cascades for deep pedestrian detection. In: Proceedings of IEEE International Conference on Computer Vision, 2015
Cao J, Pang Y, Li X. Learning multilayer channel features for pedestrian detection. IEEE Trans Image Process, 2017, 26: 3210–3220
Li J, Liang X, Shen S M, et al. Scale-aware fast R-CNN for pedestrian detection. IEEE Trans Multimedia, 2017. doi: https://doi.org/10.1109/TMM.2017.2759508
Zhang L, Lin L, Liang X, et al. Is faster R-CNN doing well for pedestrian detection? In: Proceedings of European Conference on Computer Vision, 2016
Lin C, Lu J, Wang G, et al. Graininess-aware deep feature learning for pedestrian detection. In: Proceedings of European Conference on Computer Vision, 2018
Acknowledgements
This work was supported by National Natural Science Foundation of China (Grant No. 61632018) and National Key R&D Program of China (Grant Nos. 2018AAA0102800, 2018AAA0102802).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xie, J., Pang, Y., Cholakkal, H. et al. PSC-Net: learning part spatial co-occurrence for occluded pedestrian detection. Sci. China Inf. Sci. 64, 120103 (2021). https://doi.org/10.1007/s11432-020-2969-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-020-2969-8