Person re-identification for smart cities: State-of-the-art and the path ahead
Graphical abstract
Introduction
With the use of cutting-edge technology, smart cities are no longer distant dreams, rather a possible reality. The use of smarter technologies is the answer to problems like traffic congestion, crime, vandalism, etc. that the urban communities are facing. Video Analytics is one such smart technology that, by and large, improves security as well as operational productivity. The surveillance infrastructure predominantly uses networks of video cameras to capture the scenes round the clock. These cameras produce thousands of hours of video data every day. Should there be a requirement to find a clue for an untoward incident in such humongous data, it is humanly impossible.
There is a significant effort underway among the researchers to handle this problem by developing a fully automated surveillance and monitoring systems. Person Re-Identification (PRId) involves the task of assigning the same identifier to all instances of a particular individual captured in a series of images or videos. As human beings, our brain performs PRId all the time effortlessly. Our brain and eyes trained to detect, localize, recognize, and later re-identify people and objects in real-life scenarios. Humans can obtain such a descriptor based on the person’s attire, built and complexion, height, face, shape [50], position, hairstyle [23], gait [32], skeleton features, surface features, and carried objects, which are suitable for re-identification. However, machine-based detection and recognition of an individual across multiple cameras with overlapping or non-overlapping views is a challenging problem, given the scenarios like occlusions, changes in pose, scale, and illumination variation that significantly modify the perceived appearance of a person across cameras. Despite the many advances in this field, PRId is still an open problem. This survey aims to address all the aspects of PRId, including issues, challenges, available datasets, evaluation metrics, and its applications.
Section snippets
The genesis of person re-identification
Formally, PRId can be modeled as a matching/recognition task. A query image/video of a person whose identity is to be established (typically known as ‘probe’) is matched against a large image/video database of different persons whose identity is established aprior (typically known as ‘gallery’). Thus, the problem of re-identifying an individual represented by its descriptor P can be formulated as: where is a gallery of N template descriptors, Q is the query or
Contextual methods
These methods mainly depend on the external contextual information to establish the correspondence or extracting features for re-identification.
Non-contextual methods
These methods primarily focus on the study of visual descriptors of the person for establishing the correspondence without using any external information. These PRId methods again divided into active and passive depending on the learning techniques.
Databases
In this segment, we briefly summarize the various PRId datasets that are available and its challenges. PRId task has several problems such as low-resolution images (RES), viewpoint variations (VPV), occlusions (OCC), illumination variations (IV), background clutter (BC), and detection errors (DE). Table 3, provides the statistical summary of each dataset. According to the number of images of each probe person, PRID datasets characterized into single-shot (SS), multi-shot (MS), or the
Performance analysis
In this section, we summarize the results of the overall evaluation and discuss several aspects of these results in details. In Table 2, we summarize the performance of some PRId methods on each category of the dataset measured by rank-1 (R1), rank-5 (R5), rank-10 (R10), rank-15 (R15), rank-20 (R20) performance, normalized area under the curve (nAUC), and a mean Average Precision (mAP) estimate. However, on most datasets, the performance is still far from the point where we would consider
Person re-identification in smart cities
The smart city vision provides several intelligent services in various fields, such as more sustainable environment and a better quality of life for residents. As a key component of smart city security, intelligent surveillance system highlights the importance of PRId. It has many demonstrations in various application domains.
- •
Biometric recognition has long been perceived by the consumer public as a very specialized mechanism used to protect the smart cities and nations [5]. However, it is
Conclusions and future scope
In this article, our goal is to systematically characterize different PRId methods and to investigate the course of research that has conducted throughout the last decade. We have reviewed several scholastic articles in this survey and also presented the usefulness and challenges available in several benchmark datasets captured in different spectrum.
In contrast to pioneer works where feature engineering was taken up as the utmost challenge, over the last few years, the role of deep learning has
Declaration of Competing Interest
We declare that the work presented here does not bear any conflict of interest.
Acknowledgment
This research is partially supported by a project titled "Deep learning applications for computer vision task" funded by National Institute of Technology Overseas Alumni Association (NITROAA) with support of Lenovo P920 workstation and NVIDIA Corporation with support of NVIDIA Titan V GPU.
References (62)
- et al.
Part-based spatio-temporal model for multi-person re-identification
Pattern Recognit. Lett.
(2012) - et al.
A survey of approaches and trends in person re-identification
Image Vis. Comput.
(2014) - et al.
Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views
Comput. Vis. Image Underst.
(2008) - et al.
Online RGB-D person re-identification based on metric model update
CAAI Trans. Intell. Technol.
(2017) - et al.
Person re-identification in crowd
Pattern Recognit. Lett.
(2012) - et al.
An unsupervised meta-graph clustering based prototype-specific feature quantification for human re-identification in video surveillance
Eng. Sci. Technol.
(2017) - et al.
People re-identification using skeleton standard posture and color descriptors from RGB-D data
Pattern Recognit.
(2019) - et al.
Fast person re-identification based on dissimilarity representations
Pattern Recognit. Lett.
(2012) - et al.
Person re-identification for estimating bus passenger flow
Proceedings of the MIPR
(2019) - et al.
Multi-task learning with low rank attribute embedding for multi-camera person re-identification
IEEE Trans. Pattern Anal. Mach. Intell.
(2018)
Dynamic hybrid graph matching for unsupervised video-based person re-identification
Int. J. Artif. Intell. Tools
An improved deep learning architecture for person re-identification
Proceedings of the CVPR
Who is who at different cameras: people re-identification using depth cameras
IET Comput. Vis.
One-shot metric learning for person re-identification
Proceedings of the CVPR
Multi-view people surveillance using 3D information
Proceedings of the ICCV
Cloud-based biometrics (biometrics as a service) for smart cities, nations, and beyond
IEEE Cloud Comput.
Person re-identification using group information
International Conference on Digital Image Computing Techniques and Applications
Multi-level factorisation net for person re-identification
Proceedings of the CVPR
Real-time multiple people tracking with deeply learned candidate selection and person re-identification
Proceedings of the ICME
Self-enhanced R-CNNs for human detection with semi-supervised assumptions
IEEE Access
Volume-based human re-identification with RGB-D cameras.
Proceedings of the VISIGRAPP
Histograms of oriented gradients for human detection
Proceedings of the CVPR
Pedestrian recognition with a learned metric
Proceedings of the ACCV
Viewpoint invariant pedestrian recognition with an ensemble of localized features
Proceedings of the ECCV
Person re-identification based on hierarchical bipartite graph matching
2016 IEEE International Conference on Image Processing (ICIP)
Local binary pattern, local derivative pattern and skeleton features for RGB-D person re-identification
Natl. Acad. Sci. Lett.
Human semantic parsing for person re-identification
Proceedings of the CVPR
SDL: Spectrum-disentangled representation learning for visible-infrared person re-identification
IEEE Trans. Circuits Syst. Video Technol.
ThermalGAN: Multimodal color-to-thermal image translation for person re-identification in multispectral dataset
Proceedings of the ECCV
Cited by (7)
Person search over security video surveillance systems using deep learning methods: A review
2024, Image and Vision ComputingStudent behavior recognition for interaction detection in the classroom environment
2023, Image and Vision ComputingDeep learning-based person re-identification methods: A survey and outlook of recent works
2022, Image and Vision ComputingCitation Excerpt :Zhou et al. [82] provided a review to summarize the developments in domain generalization for computer vision over the past decade. Behera et al. [83] reviewed traditional and deep learning person Re-ID methods in both contextual and non-contextual dimensions. Wu et al. [84] proposed new taxonomies for the two components of feature extraction and metric learning on person Re-ID.
Futuristic person re-identification over internet of biometrics things (IoBT): Technical potential versus practical reality
2021, Pattern Recognition LettersCitation Excerpt :A pre-processing of surveillance data for making the data super-resolution [38] and for retrieval of fine-grained image [1] can be helpful in subsequent processing. Biometric authentication is a unique mechanism that assesses an individual’s biological traits, such as the face, fingerprints, ears, iris, lips, periocular, facial expressions, and behavioral traits such as keystroke dynamics, gestures, and gait are used to identify or re-identify a particular person under different circumstances [5,9,27] through appropriate feature representation [47]. As these traits are unique to a particular individual, hence these are acquired from various biometric devices and IoT-enabled sensors, and are then compared with the existing database to attain a person match.
Feature diversity learning with sample dropout for unsupervised domain adaptive person re-identification
2024, Multimedia Tools and ApplicationsDREAMT: Diversity Enlarged Mutual Teaching for Unsupervised Domain Adaptive Person Re-Identification
2023, IEEE Transactions on Multimedia