Elsevier

Computer Science Review

Volume 38, November 2020, 100289
Computer Science Review

Survey
Deep Learning Methods for Multi-Species Animal Re-identification and Tracking – a Survey

https://doi.org/10.1016/j.cosrev.2020.100289Get rights and content

Abstract

Technology has an important part to play in wildlife and ecosystem conservation, and can vastly reduce time and effort spent in the associated tasks. Deep learning methods for computer vision in particular show good performance on a variety of tasks; animal detection and classification using deep learning networks are widely used to assist ecological studies. A related challenge is tracking animal movement over multiple cameras. For effective animal movement tracking, it is necessary to distinguish between individuals of the same species to correctly identify an individual moving between two cameras. Such problems could potentially be solved through animal re-identification methods. In this paper, the applicability of existing animal re-identification techniques for fully automated individual animal tracking in a cross-camera setup is explored. Recent developments in animal re-identification in the context of open-set recognition of individuals, and the extension of these systems to multiple species is examined. Some of the best performing human re-identification and object tracking systems are also reviewed in view of extending ideas within them to individual animal tracking. The survey concludes by presenting common trends in re-identification methods, lists a few challenges in the domain and recommends possible solutions.

Introduction

Use of technological solutions for conservation of wildlife is on the rise. Recent advances in development of low powered fast computation devices, parallel processing, advanced and efficient learning algorithms among others make this feasible. Several methods are now available for ecologists and academia to ease their research and help build tools for protection of endangered species. The focus of this paper is the use of technology to automatically monitor and track animals in their natural habitat.

Animals straying into human settlements, primarily in search for food cause conflicts resulting in injury to humans, animals, or both. A fully automated monitoring system that detects animal transgressions and alerts concerned authorities can help reduce causalities. Computer Vision is one choice of technology that can potentially solve most of the associated problems. The system referred to henceforth is in context of a network of cameras running image processing software.

It is not sufficient to simply detect if a wild animal is close by. It would also be highly beneficial to provide the closest estimate of current animal location. The problem thus faced here is that of tracking an individual animal (or a group of individual animals) in a cross-camera setup in real or near-real time. It is not necessary to remember the identity of the individual(s) for an extended duration of time, but the identity would only need to be retained until the animal has moved completely out of range of the system (this could be in the order of minutes or hours).

There are multiple references to re-identification, feature extraction and closed/open-set recognition throughout this paper. A brief introduction to each of these terms is presented below.

Re-Identification refers to ranking a list of known individuals (the gallery set), in context of a probe or query image. Within the ranked or ordered set of gallery images, the top few ranks (generally referred to as top-k) would have high probability of containing the individual in the probe image. If such a system is built for a controlled environment, such as a laboratory or an animal reserve, the problem is categorized as a closed-set recognition problem; all the individuals in the gallery are known before hand. In most cases, it is likely that the individual being probed would not appear in the gallery set. This is referred to as an open-set recognition problem.

Feature extraction maps an image to a vector, typically of much smaller dimension than the original image. The resultant vector consists of the significant features of the original image. Such feature vectors are easily compared using well known metrics, such as cosine similarity, euclidean distance or even softmax classification. Classification convolutional neural networks (CNNs), such as Alexnet [1], Inception [2] and ResNet [3] which have been pre-trained over a large set of images, such as the ImageNet database are often used to extract features. Recently, training feature extraction networks using a Siamese Architecture, inspired by [4], has gained prominence. In the Siamese Architecture, two or more neural networks are trained together with shared weights to find the similarity (or dissimilarity) of pairs of images. The networks learn to discriminate between two images by auto-learning the most distinguishing features contained in them.

The challenge being addressed here is a variation of the re-identification problem. For the purposes of fully automated monitoring and tracking, each probe needs to generate a single result — either the identity of the probe individual or a new identity (in case of open-set recognition). In re-identification terms, instead of the top-k matches, tracking systems use only the top-1 match always. It goes without saying that re-identification applies only for individuals of the same species. However, general applications of animal tracking would require identifying multiple kinds of animals, and it must be capable of identifying each of them separately.

In summary, this study focuses on the top-1, open-set re-identification of multiple species of animals in a cross-camera setup with identities assigned on a non-permanent basis (short-term tracking).

The challenges for animal re-identification bear strong resemblance to those of person re-identification or vehicle re-identification. There could be large variations in the images in terms of illumination, angle of capture, pose of the individual, lateral translations, minor or complete occlusions, differing weather conditions etc. While re-identification of humans also has to deal with changes in clothing and accessories (which form nearly the whole image area of the human) this particular challenge does not exist in animal or vehicle re-identification.

However, shape or form of the animal varies significantly when compared to that of humans. Since human movements involve smaller changes to pose due to a primarily rigid style of movement, animals are considerably more fluid and undergo larger differences in form. Angle of capture also vastly affects re-identification of animals, since some identifying traits may not appear in certain angles for example, tigers which are identified by their stripe patterns, require side capture vs. frontal capture.

The contributions of this paper are:

  • 1.

    Present a survey of recent studies in animal re- identification which use Deep Learning methods for feature extraction

  • 2.

    Review such works in the context of multi-species re-identification

  • 3.

    Provide an overview of such methods in the context of open-set recognition and their applicability for the same

  • 4.

    Explore application of ideas used in state-of-the-art object tracking and human re-identification methods to animal re-identification

  • 5.

    List common trends in animal re-identification, the challenges faced and possible solutions to such problems

Section snippets

Related work

Accurate tracking of animals in their natural habitats allow ecologists to obtain their approximate locations in a non-invasive manner. Besides monitoring of individual animals, tracking data can offer useful insights about the ecological status of the location under study. It also forms a basis for crucial decisions. For example, Hays et al. [8] compile several successful case studies where tracking information of endangered animals has helped form policies for their conservation. With wide

Object tracking

Since the problem described in this paper directly relates to tracking, some state-of-the-art object tracking methods are explored in this section, along with their advantages and shortcomings. The papers selected for review are some of the top performers from the Multi Object Tracking (MOT) challenge [11], which is a widely used benchmark. Apart from the overall score (MOTA), two other key metrics have been taken into consideration when selecting the papers for review — the number of id

Deep learning methods for animal re-identification

Building on the conclusions of Schneider et al. in [9], the publications studied in this paper are only those that make use of deep learning methods for feature extraction. The list of publications surveyed here had various motivations for their research. Fig. 1 illustrates a chart that categorizes them into one of four types, namely (a) animal monitoring for individual welfare/protection, (b) animal monitoring for behavioural studies, (c) conducting animal census for wildlife conservation, and

Person re-identification

While there have been significant strides in animal re-identification, person re-identification is an older challenge that has been extensively explored. There have been vast improvements in methods of person re-identification post application of deep learning. It is therefore possible that systems and methods designed for person re-id could prove highly effective for animal re-id as well.

In this section, a few of the state-of-the art published methods for person re-identification are reviewed

Discussion

This paper presents a study of several animal re-identification methods that use deep learning for feature extraction. A few state-of-the-art object tracking and person re-identification methods have also been investigated to understand and extend ideas within them to the problem of animal tracking. A few of the challenges in building an animal tracking system are discussed in this section. Further, common traits used to solve them are listed.

Conclusion

This paper looks at the problem of re-identification in a new light — namely as a potential tool for improving tracking performance in animals. Several recent object tracking methods are reviewed, and the importance of feature extraction in such systems is highlighted. Further, several animal re-identification networks are investigated which could potentially be used for feature extraction during animal tracking. These publications are extensively examined in view of their effectiveness,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (47)

  • SchneiderS. et al.

    Past, present and future approaches using computer vision for animal re-identification from camera trap data

    Methods Ecol. Evol.

    (2019)
  • WhiteheadH.

    Computer assisted individual identification of sperm whale flukes

    Rep. Int. Whal. Comm.

    (1990)
  • MilanA. et al.

    MOT16: A benchmark for multi-object tracking

    (2016)
  • HermansA. et al.

    In defense of the triplet loss for person re-identification

    (2017)
  • PhyoC.N. et al.

    A hybrid rolling skew histogram-neural network approach to dairy cow identification system

  • BergaminiL. et al.

    Multi-views embedding for cattle re-identification

  • CheemaG.S. et al.

    Automatic detection and recognition of individuals in patterned species

  • KonovalovD.A. et al.

    Individual minke whale recognition using deep learning convolutional neural networks

    J. Geosci. Environ. Prot.

    (2018)
  • KörschensM. et al.

    Towards automatic identification of elephants in the wild

    (2018)
  • HeQ. et al.

    Distinguishing individual red pandas from their faces

  • ShuklaA. et al.

    A hybrid approach to tiger re-identification

  • LiuC. et al.

    Part-pose guided Amur tiger re-identification

  • BrustC. et al.

    Towards automated visual monitoring of individual gorillas in the wild

  • Cited by (28)

    • Assessing optimal frequency for image acquisition in computer vision systems developed to monitor feeding behavior of group-housed Holstein heifers

      2023, Journal of Dairy Science
      Citation Excerpt :

      However, the identification algorithm trained in this study was able to predict the individual identity of group-housed heifers with high accuracy, sensitivity, specificity, and positive and negative predicted values. The ability to predict the identity of each heifer in a group is essential to generate the behavior of interest on a specific animal, thereby moving from group behavior to individual behavior recognition (Prashanth and Sudarshan, 2020). Therefore, the development of a computer vision system such as this has the potential to be successfully applied in broader scenarios, such as animal tracking, disease detection, animal welfare, and other behavior inventories (i.e., lying, drinking, and ruminating time).

    • Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning

      2021, Computers and Electronics in Agriculture
      Citation Excerpt :

      In the above studies, it can be seen that multi-pig/cattle part segmentation mainly adopts deep learning technologies. The role of identification is to determine the identity of each individual in the group, and this identity can locate the recognised behaviour on a specific animal, thereby realising the transformation from group behaviour recognition to individual behaviour recognition (Prashanth & Sudarshan, 2020). Table 2 illustrates the overview of identification of pigs and cattle based on computer vision and deep learning.

    View all citing articles on Scopus
    View full text