A multi-division convolutional neural network-based plant identification system

Related works

Recently, many studies based on machine learning algorithms have been conducted for the recognition of the plant species. In most of these studies, leaf-based plant species such as Flavia (Wu et al., 2007), Swedish (Söderkvist, 2001), ICL (Silva, Marcal & Da Silva, 2013) Foliage (Kadir et al., 2011), Folio (Munisami et al., 2015) and LeafSnap (Barré et al., 2017), as well as Flower17 (Nilsback & Zisserman, 2007; Nilsback & Zisserman, 2006) and Flower102 (Nilsback & Zisserman, 2008) real-time datasets have been widely used. The academic studies based on shape, texture and color features using these datasets are summarized in Table 1.

In Table 1, the methods and datasets used in studies related to plant recognition and performance results are presented. Overall, these studies were generally conducted using shape-based methods. According to the results obtained from each study’s experimental works, the highest performance for the Flavia, Foliage, Swedish and Folio datasets was obtained by Sulc & Matas (2015), Šulc & Matas (2017), whereas the highest performance for the real-time datasets Flower17 and Flower102 was calculated as 91.9% (Zhu et al., 2015) and 84.2% (Qi et al., 2014) respectively.

Most of the previous studies undertaken for the recognition of plant species were conducted using shape, texture, and color features. The major disadvantage of these methods is that they require a preprocessing stage and are therefore unsuited for application against real-time systems. Recently, the application of CNNs (Convolutional Neural Networks) have addressed the problems caused by classical learning, and classification performance has been greatly improved as a result. Previous studies that have used deep learning for the classification of plant species are summarized in Table 2.

Table 2 shows the datasets and performance results used in studies based on deep learning for plant recognition. According to the results obtained from deep learning-based studies, approximately a 99% performance level was established for the Flavia, Foliage, Swedish, and Folio datasets. On the other hand, for the Flower17 and Flower102 datasets, (Cıbuk et al., 2019) achieved highest accuracy scores of 96.39% and 95.70%, respectively. As a result, studies based on deep learning have provided superior performance over traditional methods.

Deep architectures

Recently, many deep learning architectures have been developed using Deep Evolutionary Neural Networks and large datasets. These architectures were trained using a subset of the ImageNet dataset in the ILSVRC competition. This dataset contains more than one million images, including 1,000 classes such as keyboard, mouse, pen and many animals. The current study used the ResNet101 and DenseNet201 architectures, which are both pretrained CNN models. The characteristics of these architectures are presented in Table 3.

The ResNet architecture (He et al., 2016) is different from sequential traditional network architectures such as VGGNet and AlexNet. Although this architecture is much deeper than the VGGNet network, the size of the network and the number of parameters are lower. The ResNet architecture is based on adding blocks that feed the next layers of values into the model (see Fig. 1A). The ResNet architecture with this structure has ceased to be a classic model and this architecture contains fewer parameters, although the depth increases (Doğan & Türkoğlu, 2018; Türkoğlu & Hanbay, 2019b).

The DenseNet and ResNet models have similar architectures. However, whilst each ResNet module in a ResNet architecture only receives information from the previous module, the DenseNet architecture is based on receiving information from the previous layers. This difference in DenseNet architecture intensively combines each layer with feedforward (Türkoğlu & Hanbay, 2019b; Nguyen et al., 2018; Chen et al., 2017; Rafi et al., 2018; Liang et al., 2018). The advantages of the DenseNet architecture over ResNet are as follows (Tsang, 2018):

• Provides strong gradient flow;

• Increases efficiency in parameter and calculation;

• More diversified features are obtained.

Principal component analysis

Principal Component Analysis (PCA) is a method that was proposed by Hotelling in 1933. The method refers to the multivariate analysis used in basic spectral decomposition of a coefficient of coordination or covariance matrix. PCA is a statistical method that is widely used in areas such as image processing, speech recognition, text processing, pattern recognition, and image compression. Due to the complexity of the accounts in this method, computer usage was not preferred until it became widespread Hernandez, Alferdo & Türkmen, 2018. The method analyses data defined by several dependent variables associated with each of the observations (Moore, 1981). PCA finds a pattern among the variables in the original data, reducing the size of the data without loss in most of the data (Jollife & Cadima, 2016). In the PCA approach, when the variance of the original variables in the data is at the maximum, it creates a new series of orthogonal axes at an appropriate angle (Jollife & Cadima, 2016; Hernandez, Alferdo & Türkmen, 2018). The greatest change directions of the vertical axis formed in this way are called the main component. Thus, the dimensionality of the data is reduced and the interpretation of the new data becomes less complex following the conversion, as about 20 PCA components can be sufficient to represent the original data with 90% to 95% accuracy. In this regard, PCA is widely preferred for image processing applications since there is relatively little data loss while reducing data.

Support Vector Machine

Support Vector Machine (SVM) classification is one of the most effective and simple methods used (Burges, 1998). SVMs were originally designed to classify two-class linear data. However, later it started to be used in both classification and nonlinear data classifications. Its basic principle is based on finding the best hyperplane to separate into two classes of data (Aslan et al., 2017; Demir et al., 2020). However, whilst there are many hyperplanes that can separate two-class data, SVM is able to find the hyperplane that will maximize the distance between the points closest to it.

Let us assume that SVM has training data in the form of l samples $\left(x_i,y_i\right)=1,2,3,\dots ,l$. Here, x_i shows the N-dimensional space and y_i ∈ { − 1, 1} class labels. Accordingly, the decision function that will find the best hyperplane, which is the purpose of the classification, can be written as shown in Eq. (1). (1)${\displaystyle y_i\left(w^{t}x+b\right)\ge 1}$ where w is the n dimensional weight and b defines the threshold value. The maximum distance $d=max\left(\frac{1}{∥w∥}\right)$ is taken; thus, ${∥w∥}^2$ is minimized and the best hyperplane is found. However, this equation only works if the predicted and actual answers have the same sign. Therefore, the product of both can only happen when there is at least one. This situation defines a major problem for SVM, but can be solved using the Lagrange optimization method of Eq. (2). (2)${\displaystyle L\left(\alpha \right)=\sum _{i=1}^N{\alpha }_i-\frac{1}{2}\sum _{i,j=1}^N{\alpha }_i{\alpha }_j{y}_i{y}_{j}k\left(y_i,y_j\right)}$ where, k(y_i, y_j) is the kernel function and α_i are the Lagrange multipliers.

Datasets

In the current study, we used eight different datasets, namely Flavia, Swedish, ICL, Foliage, Folio, Flower17, Flower102, and LeafSnap, to test the performance of the proposed MD-CNN model. A large number of plant recognition studies have been conducted in the literature using these same datasets. The characteristic features and sample images such as the number of images, number of species, and the image sizes related to these datasets are presented in Table 4.

Table 4:

Characteristics and sample images of datasets.

Dataset	Samples Images			Number of species	Number of species	Image dimensions
Flavia				1,907	32	1,600 × 1,200
Foliage				7,200	60	–
Folio				640	32	2,322 × 4,128
Swedish				1,125	15	–
LeafSnap				7,719	185	–
Flower17				1,360	17	–
Flower102				8,189	102	–

DOI: 10.7717/peerjcs.572/table-4

Performance results

The deep-based system proposed in the current study for the classification of plant species had accuracy scores calculated by using 2 × 2, 3 × 3, 4 × 4 and 5 × 5 matrix division processes. In the experimental studies that were conducted, the highest accuracy score was achieved by using the proposed 2 × 2 division approach for the Flavia, Swedish, Flower17, Flower102, and LeafSnap datasets; while for the ICL, Foliage, and Folio datasets, the highest accuracy score was obtained by using the 3 × 3 division approach. These results and the corresponding feature numbers are presented in Table 5.

As can be seen in Table 5, the highest accuracy scores are shown for eight different datasets using the proposed MD-CNN model. According to these results, a 100% accuracy score was obtained for the Flavia, Swedish, and Folio datasets, while the accuracy scores obtained for the ICL, Foliage, Flower17, Flower102, and LeafSnap datasets were 99.77%, 99.93%, 97.87, 98.03%, and 94.38%, respectively. The complexity matrix of the highest accuracy score obtained for the Flower17 dataset using the MD-CNN model is shown in Fig. 4.

In Table 5, the feature parameter numbers are given using the PCA method for each dataset. For example, with ResNet101 by using the 2 × 2 division approach for the Flavia dataset, a total of 30 features were extracted from each of the four sections obtained from a raw image, making a total of 120 features having been obtained. Similarly, 120 features were obtained for DenseNet201. Then, these features were combined and applied to SVM.

Based on the division approach, extensive experimental studies were conducted for each of the eight datasets. For example, a 100% accuracy score was obtained for the Flavia dataset when using the 2 × 2, 3 × 3, 4 × 4, and 5 × 5 division approaches. In addition, an accuracy score of 99.93% was achieved when using the 3 × 3 division approach for the Foliage dataset, whilst accuracy scores of 99.9%, 99.87%, and 99.72% were obtained for the 2 × 2, 4 × 4, and 5 × 5 division approaches, respectively. Across all of the experimental studies, it was observed that the highest performances for each dataset were obtained from the 2 × 2 and 3 × 3 division approaches.

Comparison of proposed model with other studies

The accuracy scores of the proposed MD-CNN model were compared with the state-of-the-art studies in the literature for the recognition of plant species. These comparative results are presented in Table 6 considering the datasets used in the current study.

Table 6 shows how the experimental results of the proposed MD-CNN model compared to previous studies which had recorded high levels of performance accuracy for the eight plant image datasets used in the current study. Unlike the previous studies, the current study obtained distinctive features for each part of the image by divided the image into nxn window sizes, rather than using the whole leaf image. As can be seen in Table 6, this approach in the current study produced a significantly increased level of classification performance.

According to the results obtained in the current study’s experimental studies, the accuracy scores for the Flavia, Swedish, ICL, Foliage, Folio, Flower17, Flower102, and LeafSnap datasets were 100%, 100%, 99.77%, 99.93%, 100%, 97.87%, 98.03%, and 94.38%, respectively. Based on these results, the highest performance achieved for all eight datasets were from the proposed MD-CNN model when compared to the previous studies.