Abstract

A grading method of potted Anthurium based on machine vision is proposed. A detection system is designed to acquire color images and depth images of potted Anthurium, and the three-dimensional point-cloud image is reconstructed after registration. According to the testing requirements of potted Anthurium, the minimum enclosing rectangle method is used to measure the width of crowns and spathes. The bubble sequencing method is used to measure the plant height, and the clustering segmentation method is used to calculate the number of spathes. Online automatic grading software for potted Anthurium is developed. Compared with manual measurement, the average measurement accuracies of machine vision for crown width, plant height, spathe width, and spathe number are 98.4%, 98.4%, 98.8%, and 86.7%, respectively. The accuracy rate of grading is 85.86%, which can meet the requirements of automatic grading of potted Anthurium.

1. Introduction

With the change of consumption fashion and the improvement of purchasing power, people’s demand for flowers is increasing. Potted Anthurium, a highly ornamental tropical flower, has gained in popularity in recent years. In 2016, the yield of potted Anthurium in China alone was 35 million pots, and China has become the largest consumer of potted Anthurium in the world. Grading is an important part of large-scale production of potted flowers. The traditional manual method is time-consuming and laborious, lacks objective consistency, and may cause damage to flowers, in addition to being difficult to adapt to current large-scale production. Automated equipment is urgently needed to replace manual grading. Nondestructive testing technology based on machine vision has become an inevitable trend of flower grading due to its good objective consistency and high efficiency [110].

Flower grading based on machine vision has been studied by many researchers. Most of the publications are based on two-dimensional (2D) digital images and mainly focus on the defects, colors, and shapes of flowers [1117]. However, limited by its own dimensionality, it is difficult to obtain a flower’s real morphology from a 2D image. In recent years, increasingly, more attempts have been made to study plant phenotypes based on 3D information, but there are few reports of such studies of potted Anthurium [1824]. In 2010, Wang et al. used binocular stereo vision technology to reconstruct a 3D model of corn leaves and measured the leaf length [25]. Uhrmann et al. developed a 3D scanning system for a single plant based on the principle of structured light. The 3D reconstruction of Arabidopsis thaliana was completed, and the plant height, leaf size, and growth angle were measured [26]. Bellasio et al. used structured light and fluorescence imaging 3D reconstruction technology to track the growth process of crops and obtained 3D information of leaf size, leaf angle, and plant height [27]. In 2017, Si et al. used the Harris algorithm to measure the plant height from the depth image of vegetables [28]. In 2019, Si et al. segmented overlapping particles from the registered depth image and obtained accurate walnut counts [29].

The goal of this study is to design an automated grading system for potted Anthurium based on machine vision. The advantage of this system is that the crown width, plant height, spathe width, and spathe number of potted Anthurium can be measured by a single image capture. To achieve this goal, a series of feature parameter acquisition algorithms based on RGB-D images were studied. Grading tests were then carried out to test the accuracy of measurement and grading.

2. Materials and Methods

2.1. Grading Basis and Method

According to the standards GB/T18247.2-2000, DB44/T154-2003, and NY/T1656.3-2008 [3032], potted Anthurium can be divided into two varieties: small flower (spathe width <6 cm) and large flower (spathe width ≥6 cm), and each variety can be divided into three grades. The grading basis is shown in Tables 1 and 2 , respectively. The lowest grade of all evaluation parameters is taken as the grade of potted Anthurium, and potted Anthurium lower than the third level is removed directly. Therefore, the evaluation parameters of potted Anthurium were determined such as crown width, plant height, spathe width, and spathe number.

2.2. Image Capture System

To reconstruct a 3D model of Anthurium in a computer environment, an image capture system was built. The system consists of a conveying unit, image capture unit, control unit, and execution unit. As shown in Figure 1, the testing room size is 1800 mm × 1300 mm × 1800 mm and a KinectV2 camera was used to obtain both color and depth images in the top view. The resolution of RGB images is 1920×1080, and that of depth images is 512 × 424. The distance between the camera and tray is 1271 mm. The work flow of this system is as follows: the potted Anthurium to be tested are placed on trays of the conveyor belt at a certain distance and then transported to the testing room. When they reach the testing point under the camera, a photoelectric sensor is triggered and a signal is sent to the control unit. The control unit triggers the camera to take pictures, processes them, and returns the results. When the flower reaches the corresponding grading point, the photoelectric sensor is triggered, and the control unit drives the corresponding cylinder to push the flower out, completing the grading.

2.3. Image Capture and Preprocessing
2.3.1. Image Capturing and Registration

Since the resolution of the depth and color images is different, the pixels cannot correspond one-to-one, so image registration is necessary. In this study, the depth and color images are registered through the ICoodinateMapper interface provided by Kinect SDK software. The color and registered depth images are shown in Figures 2(a) and 2(b), respectively.

2.3.2. Construction of 3D Point Cloud

The 3D point-cloud images generated from the RGB and registered depth images are shown in Figures 3(a) and 3(b), which can express the information of three dimensions in space. Each pixel in the image contains (X1, Y1, Z, R, G, and B) information. For further parameter measurement, target extraction and noise reduction are conducted, and the results are shown in Figures 3(c)3(f).

2.3.3. Feature Parameter Extraction

(1)Crown widthThe crown width of potted Anthurium refers to the mean value of its crown projection in two vertical directions. In this study, the average side length of the smallest circumscribed rectangle is taken as the crown width. The original RGB image is shown in Figure 4(a), and the calculation steps are the following. First, the second-order color matrix method was used to separate the crown from background, as shown in Figure 4(b). The Ostu threshold segmentation method (threshold T = 10) was then used for binarization, and the results are shown in Figure 4(c). It can be seen that there is still a significant amount of speckle noise. In order to obtain a complete and clear crown contour, two morphological opening operations were used. The first operation is performed to cut off the narrow connections and make the edges smoother, as shown in Figure 4(d). The second operation is conducted to remove speckles, as shown in Figure 4(e). After inverse calculation, the minimum circumscribed rectangle of the crown is drawn, as shown in Figure 4(f), and the average side length calculated.(2)Plant heightPlant height H is the distance from the edge of pot to the plant top, while the data measured by the depth camera are the distances from the plant top to the camera, h1. In this study, the distance from the edge of the pot to the camera is a fixed value of 1240 mm, so the calculation formula for plant height is H = 1240−h1, as shown in Figure 5(a). The calculation steps are the following. First, taking Figure 4(b) as the template, the depth image of the crown is extracted and the depth value of the background is set to 0. The result is shown in Figure 5(b). Then, open operation is used to remove speckled noise, and a complete and clear crown depth image is obtained, as shown in Figure 5(c). Finally, the bubble sort method is used to find the point of the minimum depth value, which is the vertex of the flower, as shown in Figure 5(d), so that the plant height can be obtained.(3)Spathe widthSpathe width is the transverse diameter of the spathe, which represents the size of the flower. The largest spathe width is usually taken. According to the growth characteristics and top advantages of potted Anthurium, the largest flowers are usually spread at the top. Thus, the calculation steps are the following. First, the image of the spathe can be obtained by binarizing Figure 4(b) (threshold T = 160), and the result is shown in Figure 6(a). Second, the morphological opening operation is used to remove the speckle noise to obtain a complete and clear spathe image, as shown in Figure 6(b). Finally, the Canny algorithm is used to extract the contour, and the smallest circumscribed rectangle of the largest spathe is calculated. As shown in Figure 6(c), the short side length is the spathe width.(4)Spathe numberIt can be seen from the above discussion that the difference between spathes and background is obvious, but the spathe number cannot be accurately extracted when the spathes are occluded in the 2D image. However, owing to the different heights of the spathes, they can be accurately segmented by using height difference in 3D point-cloud images. It can be seen from Figures 7(a) and 7(b) that two spathes occluded in the top view are easily segmented by height difference in the side view. Therefore, in this study, 3D point-cloud images are used to calculate spathe number. The spathes were counted by the clustering segmentation method based on color region growth. As shown in Figure 7(c), the points in the same cluster are given the same color and the total number of colors is spathe number.

3. Experimental Results and Discussion

3.1. Grading Experiment

Based on the method detailed above, grading software of potted Anthurium was developed. As shown in Figure 8, this software has a clear interface and simple operation. To verify the effectiveness and accuracy of the measuring method and grading effect of this grading system, machine vision and manual measurement methods were used for grading experiments, and the measurement and grading results were compared and analyzed. The experimental samples were 92 large variety potted Anthurium purchased from the Guangdong Academy of Agricultural Sciences, China.

3.2. Results and Discussion
3.2.1. Comparison of Measurement Results

Tables 36 present the comparison results of machine-vision measurement and manual measurement of crown width, plant height, spathe width, and spathe number, respectively. The results show that the relative errors of the maximum, minimum, and average values of crown width and plant height are small, and the measurement accuracies are all above 98%. The maximum error of the spathe width is slightly larger, which is mainly due to the tilt of the spathe. The machine vision measures the spathe’s projection width, so the value is smaller, which leads to the error, but the average accuracy is still high. The maximum error of spathe number is large, which is mainly because when two spathes are located at the same height and overlap, they cannot be accurately segmented, and a counting error occurs. However, for 92 samples, this situation is rare, and the average measurement accuracy can reach 86.7%.

3.2.2. Comparison of Grading Results

Table 7 shows the comparison results of machine-vision and manual grading. It can be seen from the results that, compared with manual grading, the accuracy of machine-vision grading can reach 85.86%. According to the grading basis, the lowest grade of all evaluation parameters is taken as the grade of potted Anthurium, so the accuracy of grading is close to the lowest accuracy of measurement indicators. If the segmentation problem of overlapping spathes with the same height can be effectively solved, the accuracy of grading will be greatly improved. This shows that the RGB-D digital image processing method is reasonable and feasible for potted Anthurium grading.

4. Conclusions

Based on the grading standard of potted Anthurium, a measurement method based on RGB-D image processing technology is proposed. This machine-vision method determines the grade of Anthurium by measuring the four phenotypic data (crown width, plant height, spathe width, and spathe number). The average measurement accuracies of crown width, plant height, spathe width, and spathe number measured by this method are 98.4%, 98.4%, 98.8%, and 86.7%, respectively. Compared with the manual method, the grading accuracy of the machine-vision method is 85.86%, which shows that the method is effective and feasible for potted Anthurium grading. This study has important reference value for automatic grading of potted flowers.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

The authors would like to acknowledge the financial support from Guangdong Provincial Agricultural Science and Technology Innovation and Extension Project (Grant nos. 2020KJ101 and 2021KJ131), Research and Development Program of Key Fields in Guangdong Province (Grant no. 2019b020222003), Natural Science Foundation of Guangdong Province (Grant no. 2018A0303130218), Characteristic Innovation Projects of Colleges and Universities in Guangdong Province (Grant nos. KA190578823), and Youth Innovative Talent Projects in Universities of Guangdong Province (Grant no. 2019KQNCX049). We thank LetPub (http://www.letpub.com) for its linguistic assistance during the preparation of this manuscript.