Plant species recognition based on global–local maximum margin discriminant projection

Abstract

Plant species recognition using leaves is an important and challenging research topic, because the plant leaves are various and irregular and they have very large within-class difference and between-class similarity. Considering that leaves have different discriminant performance and contribution to plant recognition task, based on maximum neighborhood margin discriminant projection (MNMDP), we propose a global–local maximum margin discriminant projection (GLMMDP) algorithm for plant recognition. GLMMDP utilizes the local and class information and the global structure of the data to model the intra-class and inter-class neighborhood scatters and a global scatter, obtaining the projection matrix by minimizing the local intra-class scatter and meanwhile maximizing both the local inter-class scatter and the global between-class scatter. Compared with MNMDP, GLMMDP not only can detect the true intrinsic manifold structure of the data, but also can enhance the pattern discrimination between different classes by incorporating the global between-class scatter into MNMDP. The global between-class scatter fully indicates the difference and similarity between classes. The experimental results on the ICL (Intelligent Computing Laboratory) leaf datasets and Leafsnap leaf image datasets demonstrate the effectiveness of the proposed plant recognition method. The recognition accuracy is more than 95% on the ICL datasets and more than 90% on Leafsnap datasets.

Introduction

Nature, the leading international journal, published one paper online under the headline “the world’s largest plant survey reveals astonishing extinction rates”, revealing a startling subject [1]. The global ecological diversity is rapidly declining, due to human habitat changes, such as excessive logging, hunting, excessive usage of pesticides, construction of water conservancy projects, alien biological invasion, etc. The disappearance is not only the loss of species diversity in the ecosystem, but also the loss of the diversity of genetic resources on the earth. The impact might not be intuitive and measurable. Plant species diversity plays a crucial role in Earth ecology. The extinction of a large number of plant species has aroused human initiative to protect the plant species diversity. The conservation of plant species requires to recognize plant species, which is useful to botanists, industrialists, food engineers and physicians. The plant species can be recognized by its different organs, such as leaf, stem, flower and fruit. Botanists can easily identify plant species by distinguishing the shape of leaf, tip, base, leaf margin and leaf vein, as well as the texture of leaf and the arrangement of leaflets of compound leaves. Wäldchen et al. [2] systematically reviewed the existing plant species identification approaches in the 120 peer-reviewed studies published in the ten years from 2005 to 2015 and they classified these methods into two classes,according to the studied plant organs such as leaf, flower and fruit and the studied features such as shape, texture, color, margin, and vein structure. Furthermore, they compared the classification accuracy of these methods achieved on the publicly available datasets. Purohit et al. [3] reviewed the image based plant species identification methods and pointed out that different plant species recognition methods are used based on images of the different parts of plant. Plant leaf is approximately two-dimensional in nature and its shape and texture are two important features for characterizing various plant species. Thus, plant recognition can be achieved by extracting features from its leaf. Many leaf image based plant recognition methods have been presented [4], [5]. Handa et al. [6] compared various plant recognition algorithms, and reviewed the main computational, morphological and image processing methods in recent years. They concluded that the plant recognition can be done by extracting various features from their leaves and there are still room to improve plant species recognition performance through designing a new digital plant recognition system. Jana et al. [7] reviewed the computer vision based approaches for plant species identification, highlighted the main research challenges to overcome in providing feasible tools, and concluded with a discussion of open questions and future research directions. Wang et al. [8] extracted more than 30 leaf features including 16 shape features, 11 texture features and 4 color features, and introduced 8 classifiers for plant recognition. Wu et al. [9] presented a fast and robust method for leaf recognition by identifying leaves based on rotation invariant shape context (RISC) and summed squared differences (SSD) color matching. Unlike the traditional scale and translational invariant of leaf shape based methods, the proposed method can recognize the leaves with different rotational angles, namely rotation invariant. Wu et al. [10] presented a leaf recognition method by combining feature extraction and machine learning. To overcome the weakness exposed in the classical algorithms, the binary Gabor pattern (BGP) with offline manner and extreme learning machine (ELM) are applied to recognizing plant leaves. Different from the traditional neural network like BP and support vector machine (SVM), the method based on ELM only requires to set one parameter, without additional fine-tuning during the leaf recognition. Especially, Medicinal plants are the main source of traditional Chinese medicine, which can provide the basic protection of human health. Jyotismita et al. [11] proposed a plant leaf recognition method by combining texture and shape features. Kan et al. [12] proposed an automatic classification method based on leaf images of medicinal plants to address the limitation of manual classification methods in identifying medicinal plants. In the method, 10 shape features and 5 texture features are extracted and SVM is adopted to classify the leaves of medicinal plants. Lavania et al. [13] presented a leaf based plant recognition algorithm using scalar invariant feature transform (SIFT) and principal component analysis (PCA) with probabilistic neural network (PNN). Jin et al. [14] proposed an automatic species classification method using sparse representation of leaf tooth features. In the method, 4 leaf tooth features (Leaf-num, Leaf-rate, Leaf-sharpness and Leaf-obliqueness) are extracted and concatenated into a feature vector to identify plant species. Chaki et al. [15] proposed a method recognizing plant leaves by combining texture and shape features, where leaf texture is modeled using Gabor filter and gray level co-occurrence matrix (GLCM) while leaf shape is captured using curvelet transform, together with invariant moments. Zhang et al. [16] aimed to solve the difficult problem of plant leaf recognition on the large-scale database and proposed a two-stage local similarity based classification learning method by combining local mean-based clustering method and local sparse representation based classification (SRC). Zeng et al. [17] presented a shape descriptor, namely periodic wavelet descriptor (PWD) to extract plant leaf feature, and constructed a database of PWDs for plant recognition. Zhang et al. [18] proposed a discriminant weighted SRC (DWSRC) algorithm for large-scale plant species classification. Different from the traditional SRC and its improved approaches, DWSRC represented the test sample sparsely on a sub-dictionary, whose basic elements are the training samples of the selected similar classes, instead of using the generic over-complete dictionary on the entire training samples. Thyagharajan et al. [19] reviewed several image processing methods in the feature extraction of leaves, and indicted that feature extraction is a crucial technique in computer vision study.

From the above methods, it is found that the classical leaf image based plant recognition methods generally include the following distinct steps: (1) acquiring leaf images. It is easy to collect the leaf images by cameras, smart phones and other IoT camera devices so that analysis towards classification can be performed; (2) preprocessing. Each original leaf image is preprocessed to enhance the discriminant performance, typically which includes image denoising, image content enhancement, and segmentation; (3) feature extraction. Various classification features are extracted to describe the leaf image; (4) classifying plant leaves. In this step, all extracted features are concatenated into a feature vector for plant species recognition.

Plant leaf recognition has been a hot research spot in recent years, which has produced the improvement in both recognition accuracy and speed. However, many existing methods usually only extract the features of shape and texture of leaf image, and adopt traditional neural network or SVM classifiers to recognize the leaf images. These methods have limitations in recognition accuracy and speed, especially when facing a large leaf database. From the above analysis, we can conclude that the recognition results mainly rely on the extracted features from leaves. However, plant leaf images are various, complex and irregular with a large intra-class difference and inter-class similarity, as shown in Fig. 1. It is difficult to determine which features are optimal, and it is also ineffective to utilize all possible kinds of features to classify the plant species. Thus, many existing classical leaf image feature extraction based plant recognition methods cannot achieve satisfactory results.

In recent years, many manifold-based learning algorithms, such as maximum margin criterion (MMC) and maximum neighborhood margin discriminant projection (MNMDP) [20], [21], [22], have been proposed to discover the intrinsic low-dimensional embedding feature of the original image, and yielded impressive results on artificial and real-world data recognition, even on plant leaf recognition [23]. There are several supervised variants of linear discriminant analysis (LDA) and locality preserving projection (LPP) [24], [25]. LDA takes care of the class information to find the global discriminant information for classification by maximizing the ratio between inter-class and intra-class scatters. MMC is more efficient than LDA for calculating the discriminant vectors since it does not need to calculate the inverse within-class scatter matrix. LPP tries to find an embedding to preserve the local neighborhood information. MNMDP makes use of the class label information for discovering the inherent manifold structure. Shao proposed a supervised global-locality preserving projection (SGLPP) algorithm for plant leaf recognition [26]. SGLPP utilizes the local information and class information of the training samples to construct the global weighted inter-scatter matrix, which can enlarge the distance between different classes in the data and then effectively reveal the intrinsic manifold structure for classification. Based on MMC, MNMDP and SGLPP, we propose a global–local maximum margin discriminant projection (GLMMDP) algorithm for plant species classification. Different from the classical manifold learning methods in constructing the neighborhood weights and the optimal objective function, the objective function of GLMMDP can be constructed by combining the global and local information of the intra-class samples and the discriminating information of the inter-class neighbors of a given sample. It not only can preserve well the intrinsic sub-manifold structure of the data, but also can enhance the discrimination among different classes, which helps improve the classification performance. The experimental results show that GLMMDP is effective on improving classification performance. The contributions of this paper include followings,

(1) A global–local maximum margin discriminant projection (GLMMDP) algorithm is proposed for plant leaf recognition.

(2) The proposed GLMMDP not only can detect the true intrinsic manifold structure of the data, but also can enhance the pattern discrimination between different classes by incorporating a global between-class scatter.

(3) The experimental results on the ICL (Intelligent Computing Laboratory) leaf datasets and Leafsnap leaf image datasets demonstrate the effectiveness of the proposed method. The classification rate is more than 95% on ICL leaf datasets and more than 90% on Leafsnap datasets.

The remainder of this paper is organized as follows. Section 2 reviews MNMDP. GLMMDP is proposed for the plant recognition in Section 3. Section 4 shows the experimental results and comparisons. Finally, this paper is concluded in Section 5.