Elsevier

Pattern Recognition Letters

Volume 138, October 2020, Pages 282-289
Pattern Recognition Letters

Person re-identification for smart cities: State-of-the-art and the path ahead

https://doi.org/10.1016/j.patrec.2020.07.030Get rights and content

Highlights

  • Provides a methodology based taxonomy for the state-of-the-art approaches in PRId.

  • Highlights the importance of PRId as a key component of smart-city surveillance.

  • Briefly summarizes various widely used datasets that are available for PRId and their challenges.

  • Discusses usage of multi-modal information to reduce problems of appearance based methods.

  • Summarizes the results of different PRId methods and discussed several applications towards smart cities.

Abstract

One of the indispensable pillars of a smart city is its surveillance infrastructure, and it requires smart techniques to analyze the videos acquired from the surveillance cameras. Person re-identification (PRId) is one of the fundamental tasks in automated visual surveillance, and it has been an area of extensive research spanning the past decade. PRId aims at finding a person who has previously been seen or identified using some unique descriptor of the person. This survey comprises a broad spectrum of PRId methods spanning from traditional to deep-learning, being analyzed and compared. This survey also discusses various PRId frameworks based on machine learning and deep learning. This study emphasizes the challenges in building PRId systems for the benefits of smart cities and presents a critical overview of recent progress and the state-of-the-art approaches to solving some significant challenges of existing PRId systems.

Introduction

With the use of cutting-edge technology, smart cities are no longer distant dreams, rather a possible reality. The use of smarter technologies is the answer to problems like traffic congestion, crime, vandalism, etc. that the urban communities are facing. Video Analytics is one such smart technology that, by and large, improves security as well as operational productivity. The surveillance infrastructure predominantly uses networks of video cameras to capture the scenes round the clock. These cameras produce thousands of hours of video data every day. Should there be a requirement to find a clue for an untoward incident in such humongous data, it is humanly impossible.

There is a significant effort underway among the researchers to handle this problem by developing a fully automated surveillance and monitoring systems. Person Re-Identification (PRId) involves the task of assigning the same identifier to all instances of a particular individual captured in a series of images or videos. As human beings, our brain performs PRId all the time effortlessly. Our brain and eyes trained to detect, localize, recognize, and later re-identify people and objects in real-life scenarios. Humans can obtain such a descriptor based on the person’s attire, built and complexion, height, face, shape [50], position, hairstyle [23], gait [32], skeleton features, surface features, and carried objects, which are suitable for re-identification. However, machine-based detection and recognition of an individual across multiple cameras with overlapping or non-overlapping views is a challenging problem, given the scenarios like occlusions, changes in pose, scale, and illumination variation that significantly modify the perceived appearance of a person across cameras. Despite the many advances in this field, PRId is still an open problem. This survey aims to address all the aspects of PRId, including issues, challenges, available datasets, evaluation metrics, and its applications.

Section snippets

The genesis of person re-identification

Formally, PRId can be modeled as a matching/recognition task. A query image/video of a person whose identity is to be established (typically known as ‘probe’) is matched against a large image/video database of different persons whose identity is established aprior (typically known as ‘gallery’). Thus, the problem of re-identifying an individual represented by its descriptor P can be formulated as:P=argminGiD(Gi,Q),GiG where G={G1,,GN} is a gallery of N template descriptors, Q is the query or

Contextual methods

These methods mainly depend on the external contextual information to establish the correspondence or extracting features for re-identification.

Non-contextual methods

These methods primarily focus on the study of visual descriptors of the person for establishing the correspondence without using any external information. These PRId methods again divided into active and passive depending on the learning techniques.

Databases

In this segment, we briefly summarize the various PRId datasets that are available and its challenges. PRId task has several problems such as low-resolution images (RES), viewpoint variations (VPV), occlusions (OCC), illumination variations (IV), background clutter (BC), and detection errors (DE). Table 3, provides the statistical summary of each dataset. According to the number of images of each probe person, PRID datasets characterized into single-shot (SS), multi-shot (MS), or the

Performance analysis

In this section, we summarize the results of the overall evaluation and discuss several aspects of these results in details. In Table 2, we summarize the performance of some PRId methods on each category of the dataset measured by rank-1 (R1), rank-5 (R5), rank-10 (R10), rank-15 (R15), rank-20 (R20) performance, normalized area under the curve (nAUC), and a mean Average Precision (mAP) estimate. However, on most datasets, the performance is still far from the point where we would consider

Person re-identification in smart cities

The smart city vision provides several intelligent services in various fields, such as more sustainable environment and a better quality of life for residents. As a key component of smart city security, intelligent surveillance system highlights the importance of PRId. It has many demonstrations in various application domains.

  • Biometric recognition has long been perceived by the consumer public as a very specialized mechanism used to protect the smart cities and nations [5]. However, it is

Conclusions and future scope

In this article, our goal is to systematically characterize different PRId methods and to investigate the course of research that has conducted throughout the last decade. We have reviewed several scholastic articles in this survey and also presented the usefulness and challenges available in several benchmark datasets captured in different spectrum.

In contrast to pioneer works where feature engineering was taken up as the utmost challenge, over the last few years, the role of deep learning has

Declaration of Competing Interest

We declare that the work presented here does not bear any conflict of interest.

Acknowledgment

This research is partially supported by a project titled "Deep learning applications for computer vision task" funded by National Institute of Technology Overseas Alumni Association (NITROAA) with support of Lenovo P920 workstation and NVIDIA Corporation with support of NVIDIA Titan V GPU.

References (62)

  • X. Xu et al.

    Dynamic hybrid graph matching for unsupervised video-based person re-identification

    Int. J. Artif. Intell. Tools

    (2020)
  • E. Ahmed et al.

    An improved deep learning architecture for person re-identification

    Proceedings of the CVPR

    (2015)
  • A. Albiol et al.

    Who is who at different cameras: people re-identification using depth cameras

    IET Comput. Vis.

    (2012)
  • S. Bak et al.

    One-shot metric learning for person re-identification

    Proceedings of the CVPR

    (2017)
  • D. Baltieri et al.

    Multi-view people surveillance using 3D information

    Proceedings of the ICCV

    (2011)
  • S. Barra et al.

    Cloud-based biometrics (biometrics as a service) for smart cities, nations, and beyond

    IEEE Cloud Comput.

    (2018)
  • E. Basaran, M. Gokmen, M.E. Kamasak, An efficient framework for visible-infrared cross modality person...
  • A. Bialkowski et al.

    Person re-identification using group information

    International Conference on Digital Image Computing Techniques and Applications

    (2013)
  • X. Chang et al.

    Multi-level factorisation net for person re-identification

    Proceedings of the CVPR

    (2018)
  • L. Chen et al.

    Real-time multiple people tracking with deeply learned candidate selection and person re-identification

    Proceedings of the ICME

    (2018)
  • X. Chen et al.

    Self-enhanced R-CNNs for human detection with semi-supervised assumptions

    IEEE Access

    (2020)
  • S. Cosar et al.

    Volume-based human re-identification with RGB-D cameras.

    Proceedings of the VISIGRAPP

    (2017)
  • N. Dalal et al.

    Histograms of oriented gradients for human detection

    Proceedings of the CVPR

    (2005)
  • M. Dikmen et al.

    Pedestrian recognition with a learned metric

    Proceedings of the ACCV

    (2010)
  • D. Gray et al.

    Viewpoint invariant pedestrian recognition with an ensemble of localized features

    Proceedings of the ECCV

    (2008)
  • Y. Huang et al.

    Person re-identification based on hierarchical bipartite graph matching

    2016 IEEE International Conference on Image Processing (ICIP)

    (2016)
  • Z. Imani et al.

    Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification

    Natl. Acad. Sci. Lett.

    (2019)
  • M.M. Kalayeh et al.

    Human semantic parsing for person re-identification

    Proceedings of the CVPR

    (2018)
  • K. Kansal et al.

    SDL: Spectrum-disentangled representation learning for visible-infrared person re-identification

    IEEE Trans. Circuits Syst. Video Technol.

    (2020)
  • V.V. Kniaz et al.

    ThermalGAN: Multimodal color-to-thermal image translation for person re-identification in multispectral dataset

    Proceedings of the ECCV

    (2018)
  • C.-H. Kuo, V.D. Shet, S. Khamis, Machine-Learnt Person Re-Dentification, 2016. Patent No. US 9,396.412 B2, Filed Jun...
  • Cited by (7)

    • Deep learning-based person re-identification methods: A survey and outlook of recent works

      2022, Image and Vision Computing
      Citation Excerpt :

      Zhou et al. [82] provided a review to summarize the developments in domain generalization for computer vision over the past decade. Behera et al. [83] reviewed traditional and deep learning person Re-ID methods in both contextual and non-contextual dimensions. Wu et al. [84] proposed new taxonomies for the two components of feature extraction and metric learning on person Re-ID.

    • Futuristic person re-identification over internet of biometrics things (IoBT): Technical potential versus practical reality

      2021, Pattern Recognition Letters
      Citation Excerpt :

      A pre-processing of surveillance data for making the data super-resolution [38] and for retrieval of fine-grained image [1] can be helpful in subsequent processing. Biometric authentication is a unique mechanism that assesses an individual’s biological traits, such as the face, fingerprints, ears, iris, lips, periocular, facial expressions, and behavioral traits such as keystroke dynamics, gestures, and gait are used to identify or re-identify a particular person under different circumstances [5,9,27] through appropriate feature representation [47]. As these traits are unique to a particular individual, hence these are acquired from various biometric devices and IoT-enabled sensors, and are then compared with the existing database to attain a person match.

    View all citing articles on Scopus
    View full text