Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning

Highlights

• Computer vision methods for livestock behaviour recognition were evaluated.

• Journey from computer vision to deep learning was elaborated.

• This paper involves with segmentation, identification and behaviour recognition.

• Research trend of livestock behaviour recognition was illustrated from four aspects.

• This review could provide researchers and producers with technical references.

Abstract

The increasing demand for sustainable livestock products also demands new considerations in animal breeding. Breeding programs are now seeking to integrate animal behavioural phenotypes, as these relate to the productivity, health and welfare of the animals and thereby can influence yield and economic benefits in the industry. Traditional manual observation of pig behaviour is time-consuming, laborious, subjective, and difficult to achieve in continuous and large-scale operations. It is not surprising that computer vision technology with the advantages of being objective, non-invasive and continuous has been widely researched for its use in the recognition of livestock behaviours over recent years. Nevertheless, in studies of livestock behaviour recognition, computer vision technology faces some challenges, e.g., complex scenes, variable illumination, occlusion, touching and overlapping between livestock, which has limited the fast translation of technology to industry. On the other hand, deep learning technology has proven to solve these difficulties to a certain extent and is being adopted to recognise livestock behaviours. This paper mainly evaluates the recent developments in computer vision methods for recognition of these behaviours in pigs and cattle. The focus on these species is made possible by the number of studies exist quantifying behaviours that are of importance for their health, welfare and productivity such as aggression, drinking, feeding, lameness, mounting, posture, tail-biting and nursing. This review paper especially analyses the development of image segmentation, identification and behaviour recognition using tradition computer vision and more recent deep learning methods, and evaluates the evolution of key research in the field. We elaborate the research trend of livestock behaviour recognition from four aspects, i.e., development of robust livestock identification algorithms, recognition of livestock behaviours for different growth stages, further quantification of the results of behaviour recognition, and building evaluation system of growth status, health and welfare.

Introduction

With the increasing demand for sustainable animal products, livestock breeding and careful animal management have become an important means of improving production efficiency of the livestock industry (Norton et al., 2019). Livestock behaviours reflect the health, welfare and growth status of the pigs, thereby affecting the yield and economic benefits (Larsen et al., 2021). Individual animal behaviours are related to the amount of water and feed consumed, and are important for understanding animal productivity (Botreau et al., 2007). The social behaviour of animals can give important insights into their welfare status, for example aggression between pigs can cause skin trauma, infection and even fatal injuries (Turner et al., 2006). Excessive mounting behaviours can cause a high risk of poor welfare, arising from skin lesions, lameness and stress, and economic losses from reduced performance (Teixeira and Boyle, 2014). Tail-biting is considered to be a welfare-reducing problem with economic consequences for pig production (Larsen et al., 2019). Playing behaviour of pigs towards enrichment objects can reduce the occurrence of tail-biting, mounting and aggression and consequently improve animal welfare (Lahrmann et al., 2018). Nursing behavior, as one of behaviours of sows during lactation, is critical for early survival and growth of their piglets prior to weaning (Muns et al., 2013), which has a great impact on the economic benefit of pig farms (Vila and Tummaruk, 2016). Furthermore, animal body-part movement can be used for disease detection, for example lameness has become a frequent and serious problem for herd productivity and animal welfare in the dairy industry (Bruijnis et al., 2012). Clinical lameness has a significant impact on milk yield (Ouared et al., 2015) and reproductive performance (Morris et al., 2011). Cattle pose estimation is a key step analysing cattle behaviors and evaluating cattle health, thereby greatly significant for intelligent breeding of cattle (Yazdanbakhsh et al., 2017). Therefore, the monitoring and recognition of livestock behaviours are of great significance to the development of precision livestock farming.

As the traditional method of manual observation of livestock behaviours is time-consuming, laborious, subjective, and difficult to achieve continuous and large-scale operations, in recent years computer vision technology that has the advantages of objective, non-invasive and continuous has been widely applied for recognition of livestock behaviours, e.g., aggression, drinking, feeding, lameness, mounting, posture, tail-biting, nursing, playing and other behaviours. Computer vision is a simulation of biological vision using computers and related equipment, and is an important part of the field of artificial intelligence. Its main task is to obtain the information of the corresponding scene by processing the collected images or videos. A traditional computer vision system is mainly aimed at extracting features from images, and it also includes a series of other subtasks, e.g., edge detection, corner detection, image segmentation and pattern recognition. Traditional feature extraction algorithms include Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Binary Robust Independent Element Feature (BRIEF) (García et al., 2020). According to the type and quality of input images, different algorithms have different degrees of success. Ultimately, the accuracy of the entire system depends on the method of extracting features. The main challenge of this method is to tell the system which features to look for in the image. As the algorithm runs according to the designer's definition and the extracted features are artificially designed, poor algorithm performance can be improved by fine-tuning in the implementation. However, such changes need to be done manually and hard-coded for specific applications, which pose a big obstacle to the realisation of high-quality computer vision. In the existing studies of livestock behaviour recognition, computer vision technology also faces other challenges, e.g., complex scenes, variable illumination, occlusion, touching and overlapping between livestock (Tu et al., 2020).

Deep learning technology that can solve the above difficulties to a certain extent has been gradually used to recognise livestock behaviours. For instances, Yang et al. (2018a) used a fully convolutional network (FCN) to segment images of lactating sows with different scenes, variable illumination, etc. Furthermore, Tian et al. (2019) used the modified Counting Convolutional Neural Network (CNN) model based on the architecture ResNeXt to count the number of pigs under the conditions of partial occlusion, overlapping and different perspectives. Therefore, development from computer vision to deep learning is necessary in the field of livestock behaviour recognition. Currently, deep learning systems have made significant progress in dealing with some related subtasks. The biggest difference in deep learning is that it no longer uses carefully programmed algorithms to search for specific features, but instead trains the neural network in the deep learning system (Yang & Xiao, 2020). As the computing power provided by deep learning systems increases, the computer will be able to recognise and react to everything it sees, which has made significant progress. In recent years, the development of deep learning has not only break-through many difficult visual problems to improve the level of image cognition, but also accelerated the progress of related technologies in the field of computer vision. With the continuous improvement of deep learning models and computing power, autonomous systems can continue to develop steadily and truly realise the interpretation and response to what they perceive.

Through the investigation of a large number of computer vision-based and deep learning-based pig and cattle behaviour recognition literature, this paper mainly evaluates the methods for recognition of these behaviours of pigs and cattle. Since image segmentation and identification are the basis of livestock behaviour recognition, this paper analyses the development process of image segmentation, identification and behaviour recognition from computer vision to deep learning, and provides researchers and producers with technical references. Furthermore, this paper elaborates the research trend of livestock behaviour recognition from four aspects, i.e., development of robust livestock identification algorithms, recognition of livestock behaviours for different growth stages, further quantification of the results of behaviour recognition, and building evaluation system of growth status, health and welfare.

This paper is organised as follows: Section 2 reviews the studies of image segmentation of pigs and cattle from body entirety to body part. Section 3 reviews the studies of identification of pigs and cattle from body part to body entirety. Section 4 reviews the studies of behaviour recognition of pigs and cattle from computer vision to deep learning. Section 5 proposes the research trend of livestock behaviour recognition.