Fruit Morphological Measurement Based on Three-Dimensional Reconstruction

Abstract

Three-dimensional (3D) shape information is valuable for fruit quality evaluation. Grading of the fruits is one of the important postharvest tasks that the fruit processing agro-industries do. Although the internal quality of the fruit is important, the external quality of the fruit influences the consumers and the market price significantly. To solve the problem of feature size extraction in 3D fruit scanning, this paper proposes an automatic fruit measurement scheme based on a 2.5-dimensional point cloud with a Kinect depth camera. For getting a complete fruit model, not only the surface point cloud is obtained, but also the bottom point cloud is rotated to the same coordinate system, and the whole fruit model is obtained by iterative closest point algorithm. According to the centroid and principal direction of the fruit, the cut plane of the fruit is made in the x-axis, y-axis, and z-axis respectively to obtain the contour line of the fruit. The experiment is divided into two groups, the first group is various sizes of pears to get the morphological parameters; the second group is the various colors, shapes, and textures of many fruits to get the morphological parameters. Comparing the predicted value with the actual value shows that the automatic extraction scheme of the size information is effective and the methods are universal and provide a reference for the development of the related application.

1. Introduction

In recent years, with the development of machine vision technology, the measurement of fruit shape has become more intelligent and standardized. Many researchers have conducted depth research on fruit measurements in the field of machine vision and have achieved fruitful results [1,2].

Machine vision technology mainly obtains the fruit images through specific industrial cameras and then uses advanced technology such as image processing to detect the shape characteristics of the fruit. Rashidi et al. [3] calculated the roundness ratio and volume of cantaloupes and studied the shape classification of cantaloupes. Su et al. [4] developed a novel method of three-dimensional models for estimating potato mass and shape information, and a new image processing algorithm for depth images including length, width, thickness, and volume as mass prediction related factors. Wang et al. [5] applied the RGB-depth sensor to measure the maximum perimeter and volume of sweet onions, estimated the density of onions using measured parameters and the RMSE of onions volume is 18.5 cm${}^3$. Lu et al. [6] developed phase analysis techniques for reconstruction of the three-dimensional surface of fruit from the pattern images acquired by a structured illumination reflectance imaging system. Fukui et al. [7] based on features extraction from images through a sub-image clustering technique, described images as the number of pixels in various labels in a regression model to estimate the fruit volume. Bin et al. [8] presented a three-dimensional shape enhanced transform to identify apple stem-ends and calyxes from defects. He et al. [9] developed a low-cost multi-view stereo imaging system that captured data from 360° around a target strawberry fruit to estimate berry height, length, width, volume, calyx size, color, and achene number. Zhang et al. [10] developed a high-efficiency Multi-Camera Photography system to measure six variables of nursery paprika plants and investigated the accuracy of 3D models reconstructed from photos. Chaudhury et al. [11] described an automated system to perform 3D plant modeling by a laser scanner mounting on a robot arm to capture plant data. Kochi et al. [12] developed a three-dimensional shape-measuring system using images and a reliable assessment technique to measure shape features with high accuracy and volume. Gaspar et al. [13] presented a volumetric reconstruction of the interior of a pear fruit using parallel slices and successfully overcame the difficulties in the registration of the tomograms by taking advantage of the possibility of scanning both sides. Kabutey et al. [14] used 3D scanner Intel RealSense to determine three-dimension virtual models of avocado, salak, dragon fruit, and mango. Gongal et al. [15] developed a machine vision system consisting of a color CCD camera and a time-of-flight (TOF) light-based 3D camera for estimating apple size in tree canopies and the size of individual pixels along the major axis. Golbach et al. [16] presented a computer-vision system for seedling phenotyping by utilizing a fast three-dimensional reconstruction method. Whan et al. [17] developed a software method to measure grain size and color from images captured with consumer-level flatbed scanners, and the accuracy and precision of the method demonstrated through screening wheat. Yamamoto et al. [18] performed three-dimensional reconstruction of fruits and vegetables by applying a consumer-grade RGB-depth sensor, the generated point cloud was corrected using the ROI of the target fruit to measure the volume and the largest diameter.

Li et al. [19] extracted shape and color features from the apples’ appearance quality by Fourier descriptor and HIS color model, then graded apples using a neural network. Arakeri et al. [20] proposed an automatic and effective tomato fruit grading system based on computer vision techniques to analyze the fruit for defects and ripeness. Fu et al. [21] demonstrated the feasibility of classifying kiwifruit into shape grades by a stepwise multiple linear regression method and to establish corresponding estimation models. Wang et al. [22] developed a potato grading system of weight and shape using image processing systems that use the PCA algorithm. Sofu et al. [23] proposed an automatic apple sorting and quality inspection system, which is based on real-time processing with an average sorting accuracy rate of 73–96%. Vitzrabin et al. [24] combined an adaptive thresholding algorithm with sensor fusion to detect sweet-peppers at highly variable illumination conditions. Tushar et al. [25] presented a nondestructive and accurate fruit grading system based on the volume and maturity feature and estimated the volume of the fruit using the volumetric 3D reconstruction method in the multiple-camera environment. Priti et al. [26] classified tomato images based on color fuzzy rule into three classes unripe, ab2ripe, and ripe, the experimental results established that RandomTree classifier performs well for the classification of tomato firmness. Hu et al. [27] studied bruise detection in apples using 3D imaging with the support vector machine and showed that this method could achieve better classification accuracy. Qureshi et al. [28] proposed two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, then compared the use of these methods relative to existing techniques. Wang et al. [29] developed a one-dimensional filter to remove the fruit pedicles and employed an ellipse fitting method to identify well-separated fruit. Stein et al. [30] presented a novel multi-sensor framework using a multiple viewpoint approach to solve the problem of occlusion to efficiently identify, track, localize, and map every piece of fruit in a commercial mango orchard. Gongal et al. [31] developed the machine vision systems of fruit detection and localization for robotic harvesting, then estimated crop-load of apples, pears, and citrus.

Virtual reality technology is one of the important measurements of reconstructing crop models in three dimensions, which has attracted wide attention from researchers in related fields. The combination of computer technology and agricultural knowledge make the research of morphological structure and physiological functions into the visualization and digitization stage. Due to the complexity of the internal structure of plants, how to quickly and accurately establish a three-dimensional model of plants has been one of the research focuses for computer graphics and agronomy.

At present, the detection of fruit shape is mainly through the digital camera to obtain the information, but two-dimensional images cannot reflect the structure of the surface. Therefore, a measurement method based on RGB-depth (RGB-D) camera which can collect both color image and depth image simultaneously is proposed. RGB images contain color and appearance information, and depth image contains distance information between RGB-D sensors and objects. The recent success of RGB-D cameras such as the Kinect sensor depicts a broad prospect of three-dimensional data-based computer applications [32]. In recent years, with the development of low-cost sensors such as Microsoft Kinect and Intel RealSense, RGB-D images have been widely used in face database [33,34,35], hand gesture [36,37,38,39,40], head pose [41,42,43], and skeleton information [44,45,46]. Compared with the RGB image, the RGB-D image can provide additional information about the object’s three-dimensional geometric structure and provide more effective information for object localization and 3D measurement.

2. Preprocessing of Fruit Point Cloud

In this experiment, a Kinect v2.0 camera and an electric turntable are used to build a three-dimensional fruit model as shown in Figure 1. During the process of scanning, eight pieces of point clouds (point cloud is collected by rotating every 45°) are registered by the iterative closest point (ICP) algorithm to obtain the initial 360° surface point cloud model. According to the rotation matrix, the subsequent point clouds are rotated around the y-axis in Equation (1). It is specified that the direction parallel to the camera coordinate system is the x-axis, the direction vertical to the camera coordinate system is the y-axis, and the direction parallel between the camera and the object is the z-axis.

$$p_i=\left[\begin{array}{ccc}cos\theta & 0& sin\theta \\ 0& 1& 0\\ -sin\theta & 0& cos\theta \end{array}\right]p_{i_{orgin}}$$

(1)

where $p_{i_{orgin}}$ is the point cloud from each direction of the surface, $p_i$ is the processed point cloud by the rotation matrix, and $i\in 1,7$. Based on the first point cloud $p_0$, rotate the other 7 pieces of point clouds by 45°, 90°, 135°, 180°, 225°, 270°, 315°, and the value of $\theta$ is the angle of rotation.

The basic principle of the ICP [47] algorithm is: Match the target point cloud $p_{\left(i\right)}$ from one single direction point cloud P to the source point cloud $q_{\left(i\right)}$ from the adjacent point cloud Q, find the nearest neighbor point $(p_i,q_i)$ according to certain constraints, and then calculate the optimal matching parameters R and t to minimize the error function in Equation (2),

$$E(R,t)=\frac{1}{n}\sum _{i=1}^n{\parallel q_i-(R{p}_i+t)\parallel }^2$$

(2)

The traditional ICP algorithm can be summarized in two steps: compute correspondences between the two-point cloud; calculate the transformation that minimizes the distance between the corresponding points. For the original point cloud of the model, preprocessing is needed, and the outlier points are removed by the statistic outlier filter [48]. In the process of statistics, the number n of the nearest neighbor point is considered, and the distance $d_i$ from the point $p_i$ to all neighboring points p is calculated to determine whether it is less than the threshold standard deviation $\delta$ of outliers. The point whose average distance is outside the standard range will be removed from the data set. After ICP registration, there are overlapped data in the adjacent patches point cloud, so the normal estimation method based on moving least square (MLS) method is used to resample the point cloud.

2.1. Registration the Surface and Bottom Point Cloud of Fruit Model

When the fruit is placed vertically on the turntable, it cannot photograph the bottom point cloud of the fruit in the experiment. The shape of the fruit bottom is different, there are convex ones, such as eggplant and potato, and concave ones, such as tomato and apple. The bottom is a part of the fruit which will affect the accuracy of the result of measuring, so it is necessary to supplement the points from the bottom.

Because the bottom point cloud is collected in the positive direction of the camera, which is different from the coordinate system of the surface point cloud. The collected bottom point cloud $p_b$ should be rotated 90° around the x-axis so that the bottom and the surface point cloud are in the same coordinate system. Equation (3) shows the bottom point cloud in the same coordinate system as the surface point cloud after rotating along the x-axis. Then, move the centroid $c_b$ of the bottom point cloud to the centroid c of the surface point cloud. It shows that the bottom point cloud is in the same coordinate system as the surface point cloud after moving the centroid $c_b$ to c. Through the rotation matrix and translation vector, the bottom point cloud $p_b$ is processed preliminarily. The initial processed point cloud and the surface point cloud are in the same coordinate system. They are all measured with x-axis as the fruit’s length, y-axis as the height, and z-axis as the width.

$$\begin{array}{c}{R}_{x\left(p_b\right)}=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\alpha & -sin\alpha \\ 0& sin\alpha & cos\alpha \end{array}\right]\hfill \\ T_{\left(p_b\right)}=\overrightarrow{centroi{d}_{c_b}-centroi{d}_c}\hfill \\ p_b=R_{x\left(p_b\right)}{p}_{b_{orign}}+T_{\left(p_b\right)}\hfill \end{array}$$

(3)

where $p_{b_{orign}}$ is the initial bottom point cloud, $p_b$ is the bottom point cloud after preliminary matrix operations, $\alpha$ is 90°, it means that the angle of bottom point cloud around the x-axis, and $T_{p_b}$ is the preliminary translation vector. $p_b$ is in the same coordinate system as the surface point cloud, and the direction of the fruit’s bottom point cloud is downward.

ICP algorithm is still needed to accurately register surface point cloud and bottom point cloud. The rotation matrix $R_b$ and translation vector $t_b$ are obtained by the ICP method below 30 mm of the surface point cloud registering with the processed bottom point cloud. The obtained transformation matrix is also applicable to the whole surface point cloud. Then the whole point cloud is obtained after the surface point cloud and the bottom point cloud registered by $R_b$ and $t_b$. As shown in Equation (4), the whole point cloud of fruit is

$$p=p_s+(R_b{p}_b+t_b)$$

(4)

where $p_s$ is the surface point cloud, $p_b$ is the bottom point cloud, and p is the whole point cloud needed in this experiment. Through coarse and precise registration of the surface and bottom points of the point cloud, the whole point cloud is finally obtained.

In Figure 2, eight pieces of continuous point clouds are registered together to obtain a complete 360° frontal model, which contains the surface information of the fruit. At this point, the point cloud of the fruit bottom and the point cloud of the front are respectively obtained, and through preliminary processing, they are in the same world coordinate system, because the fruit surface is the convex body, the surface image and the bottom image taken by the camera can simultaneously acquire the characteristics of the points under the side. Then, through the registration of points collected at the bottom side of the fruit in different coordinate systems, the bottom rotation matrix and the translation vector are obtained, and the bottom model is further processed. Through these steps, a point cloud model with shape-preserving, accurate normal, and small curvature variance is obtained. Finally, the whole fruit model is obtained by the Poisson surface reconstruction method for the preprocessing-smoothed model point cloud p. The fruit’s stalk and pulp point cloud is obtained by frontal surface registering, while the fruit’s bottom point cloud is obtained by image acquisition and coordinate system transformation. The pulp and the bottom of the fruit are combination called fruit pulp in the next subsequent chapters, which is convenient for the subsequent separation of the fruit stalk.

2.2. Segmentation the Fruit Stalk

Based on the 3D model, the stalk cannot be counted as the height of the fruit, so the length of the fruit stalk needs to be removed when calculating the height. In this experiment, the stemmed fruits need further treatment for removing stalks by the locally convex connected patches (LCCP) method [49], which does not depend on the color or texture of point cloud, but on the spatial information and normal vector information. For the segmented model, it is necessary to calculate the convexity-concavity relationship of adjacent patches. The relationship is judged by the extended connectivity criterion and sanity criterion. Extended connectivity criterion uses the angle between the centerline vector $\overrightarrow{x_1}$, $\overrightarrow{x_2}$ and the normal vector $\overrightarrow{n_1}$, $\overrightarrow{n_2}$ of the adjacent two pieces to judge concavity or convexity. If $a_1$ > $a_2$, the relationship of the two patch $\overrightarrow{p_i}$,$\overrightarrow{p_j}$ is concave, otherwise is convex as shown in Figure 3.

Here, $\overrightarrow{d}$ is the difference between the vector $\overrightarrow{x_1}$ and $\overrightarrow{x_2}$, $\overrightarrow{s}$ is the cross product between the vector $\overrightarrow{n_1}$ and $\overrightarrow{n_2}$ as shown in Equation (5),

$$\overrightarrow{d}=\overrightarrow{x_1}-\overrightarrow{x_2},\overrightarrow{s}=\overrightarrow{n_1}\times \overrightarrow{n_2}$$

(5)

If the relationship between the two pieces is convex, it should be further judged by the sanity criterion. When the angle between $\overrightarrow{d}$ and $\overrightarrow{s}$ is greater than the threshold $\theta$, it can be sure that they are convex relations [49] as in Equation (6),

$$\begin{array}{c}\theta (\overrightarrow{p_i},\overrightarrow{p_j})=min(\angle (\overrightarrow{d},\overrightarrow{s}),{180}^{°}-\angle (\overrightarrow{d},\overrightarrow{s}))\hfill \\ SC(\overrightarrow{p_i},\overrightarrow{p_j})=\left\{\begin{array}{ccc}\hfill true& ,\hfill & \hfill \theta (\overrightarrow{d},\overrightarrow{s})\ge \theta ,\\ \hfill false& ,\hfill & \hfill otherwise.\end{array}\right.\hfill \end{array}$$

(6)

After marking the concavity–convexity relationship of each small region, the region growth algorithm is used to cluster the small region into larger objects. This algorithm restricted by this relationship of small regions can clearly distinguish the boundary between stalk and pulp as shown in Figure 4; the red line represents concavity and the white line represents convexity between adjacent patches. There is an obvious concavity and convexity change at the junction of the fruit’s stalk and pulp, which meets the criteria of LCCP segmentation.

The LCCP algorithm can correctly separate fruit stalk and pulp, which is useful to calculate the actual height of the fruit without stalk using the bounding box method. The point cloud is automatically segmented by convexity–concavity relationship between pulp and stalk, and the compared models are shown in Figure 5. At present, the point cloud is divided into two parts, namely $p_{pulp}$ and $p_{stalk}$. The pulp model is retained as the base model for the following measurements.

3. Morphology Measurement of Fruit

3.1. Measurement Length, Height, Width through Principal Component Analysis Bounding-Box Algorithm

When the three-dimensional model is obtained, the length, height, and width of the fruit are calculated first. The steps are as follows: calculating the central point of fruit, where the world coordinate system is the principal direction of point cloud; using the principal direction and centroid establishing the bounding box of the point cloud. As the direction corresponding to the natural axis of the object, the maximum and minimum positions along the three directions of x, y, and z from the vertex are calculated in Equation (7).

$$\begin{array}{c}\hfill \underset{1\le i\le n}{min}X\le x\le \underset{1\le i\le n}{max}X\\ \hfill \underset{1\le i\le n}{min}Y\le y\le \underset{1\le i\le n}{max}Y\\ \hfill \underset{1\le i\le n}{min}Z\le z\le \underset{1\le i\le n}{max}Z\end{array}$$

(7)

By comparing all points $(1,n)$, the minimum coordinate point x, the maximum x, the minimum y, the maximum y, the minimum z, and the maximum z have been obtained. Then, combine the minimum values as the minimum point $(x_{min},y_{min},z_{min})$ and the maximum value as the maximum point $(x_{max},y_{max},z_{max})$. So, the minimum three-dimensional point and the maximum three-dimensional point in the three-dimensional coordinate system of fruit can be obtained. The center point is the midpoint of the two extreme points, representing the centroid of the bounding box, the center point’s three-dimensional coordinates $(a,b,c)$ is shown in Equation (8).

$$\begin{array}{c}\hfill a=\frac{x_{min}+x_{max}}{2}\\ \hfill b=\frac{y_{min}+y_{max}}{2}\\ \hfill c=\frac{z_{min}+z_{max}}{2}\end{array}$$

(8)

According to the matrix coordinate position of the bounding-box extracted the region of fruit. The effective region includes x, y, and z of fruit, which represent the direction of length, height, and width of fruit respectively. Take the minimum point $(x_{min},y_{min},z_{min})$ as the starting point, and the maximum point $(x_{max},y_{max},z_{max})$ as the end-point and draw a line from left to right along the main direction x, draw a line from up to down along the main direction y, draw a line from front to back along the main direction z, which represents the length, height, and width of the fruit, respectively. The length of fruit can be obtained by subtracting the left and right limit values of x intersection points, the height can be obtained by subtracting the limit values of y intersection points, and the width can be obtained by subtracting the limit values of z intersection points. The length l, height h, and width w of the bounding box can be obtained by the differences of the three directions $(x,y,z)$ of the maximum three-dimensional point and the minimum three-dimensional point in Equation (9).

$$\begin{array}{c}\hfill l=x_{max}-x_{min}\\ \hfill h=y_{max}-y_{min}\\ \hfill w=z_{max}-z_{min}\end{array}$$

(9)

Based on the bounding-box method, the three-dimensional fruit plane is displayed in Figure 6. It is specified that the x-axis corresponds to the length l and height h of fruit, the y-axis corresponds to the width w and height h of fruit, and the z-axis corresponds to the width w and length l of fruit. The display of the z-direction surface is divided into top and bottom surfaces to observe the top and bottom details.

3.2. Measurement Perimeters of Fruit

Further, analyzing the contour of the model, and taking the centroid of the model as the research object to calculate the perimeter of each section is another task for fruit. The distribution of three-dimensional continuous image $f(x,y,z)$ on any plane $P:ax+by+cz+d=0$ is called the section of f along plane P. The outline of multi-layer slices can be represented in three-dimensional coordinates. In this experiment, the segmentation point is obtained according to the symmetry of the fruit. The tangent planes are made parallel to $xoy$, $yoz$, and $xoz$, respectively according to the $intersectio{n}_{(}{p}_x,p_y,p_z)$, where section A, B, and C are parallel to plane $yoz$, plane $xoz$, and plane $xoy$, respectively, the points made up of these planes are shown in Equation (10),

$$\begin{array}{c}{p}_A=\left\{p_i\right|(p_x,y_i,z_i)\}\hfill \\ p_B=\left\{p_i\right|(x_j,p_y,z_j)\}\hfill \\ p_C=\left\{p_i\right|(x_k,y_k,p_z)\}\hfill \end{array}$$

(10)

Based on the three sections parallel to $x,y,z$-axis through the centroid, the intersection point of the fruit model is the set of the required point cloud, which constitutes the contour of the fruit, i.e., the perimeter. The perimeter is of great significance to identify the shape of the fruit. As shown in Figure 7, the contour obtained by plane A tangent to the model is shown in red, the contour obtained by plane B tangent to the model is shown in blue, and the contour obtained by plane C tangent to the model is shown in green. Each plane centered on the intersection is divided into four quadrants, and the intersection of these lines in the two-dimensional plane represents the outline of the fruit.

These points are discrete in two-dimensional space, so the spline interpolation method is used to connect them together. As the surface of the fruit tends to be a sphere or ellipsoid, the obtained points are improved by the cubic Bezier curve spline [50] in Equation (11),

$$\begin{array}{ccc}\hfill & B\left(t\right)=P_0{(1-t)}^3+3P_{1}t{(1-t)}^2+3P_2{t}^2(1-t)+P_3{t}^3\hfill & \hfill t\in [0,1]\end{array},$$

(11)

where $P_i$ is the boundary points of the contour, t is the scale value, the smooth curve is drawn according to the coordinates of four adjacent points. Here, we use the method in [51], which not only retains the characteristics of the cubic Bezier curve but also passes through all the points in the experiment, that is, the control point $P_1$, $P_2$ can be expressed by unit vector and tension parameter. For the continuous curve L on the section, its density distribution function is $B\left(t\right)$. Because the function $B\left(t\right)$ is a piecewise curve, the perimeter is obtained by the curve integral in Equation (12) on each small curve [a, b].

$$s={\int }_a^b\sqrt{1+B^{\prime }\left(t\right)}dtt\in [a,b]$$

(12)

The surface profile is obtained by the intersection of the tangent plane and the parallel section as shown in Figure 8, which are described as the value points of the contour line.

4. Experiment

Two groups of fruits were tested in this paper. In the first group, the predicted value is compared with the measured value after modeling the same type of fruits (pears in this experiment) in different sizes; in the second group, the predicted value is compared with the measured value after modeling many types of fruits with different shapes, colors, textures, and stalks. When practically measured, for the length, width, and height of the fruit, the actual value measured by the ruler is compared with the predicted value; for the perimeter of the fruit, the average value obtained by the soft ruler within 3 mm of the centroid is compared with the predicted value.

4.1. Measurement of the Morphological Parameters of the Pear

Forty pears are used as experimental objects to verify the validity of the experiment by modeling, measuring and comparing with the actual value. The measured results are compared with predicted values of length, height, width, and perimeter to get the minimum error, maximum error, coefficient of determination ($R^2$), and root mean square error ($RMSE$). The results are shown in Figure 9. In the coordinate graph, the x-axis represents the predicted value, and the y-axis represents the measured value of pears.

The other methods [5,18,52] are compared with the algorithm proposed in this paper, and the results are shown in

Table 1. Comparison of the method in this paper with other methods.

Experiment	${\mathit{R}}^2$	RMSE (mm)
Onion maximum diameter [5]	0.96	2.0
Apple largest diameter [18]	0.968	0.8
Rape pod diameter [52]	-	0.48
Average of vertical pear diameter	0.925	1.17
Average of horizontal pear diameter	0.977	1.03
Pear x-axis perimeters	0.938	3.85
Pear y-axis perimeters	0.911	3.78
Pear z-axis perimeters	0.928	3.72

. The value obtained by averaging the length and width is taken as vertical diameters. The results show that average of vertical pear diameter $RMSE$ is 1.17 mm and the $R^2$ is 92.5%; the horizontal pear diameter $RMSE$ is 1.03 mm and the $R^2$ is 97.7%; the perimeters of the pears are calculated, the smallest error is from the pears’ z-axis perimeters, $RMSE$ is 3.72 mm; the largest error is the pears’ x-axis perimeters, $RMSE$ is 3.85 mm.

4.2. Measurements of the Morphological Parameters of Many Kinds of Fruit

A variety of vegetables and fruits are used as experimental objects to establish a three-dimensional model and calculate the morphological parameters. For onion, pear, kiwifruit, and other fruits with stalks, the LCCP algorithm is used to remove its stalk and calculate the height of the pulp. For fruits without a stalk, like tomato, orange, eggplant, the model calculates the height directly. The measured values and predicted values are shown in Figure 10, the yellow bar represents the predicted value, the green bar represents the measured value. Among various fruit models, the error between the measured value and the predicted value of the green pepper model is larger than other fruits. The shape of the green pepper model is irregular, some details are removed during the point cloud registration process, so the model error of the green pepper is larger than that of other fruits. The details of the model still need to be further studied, which is also the focus of the next work.

To show the model more clearly, when collecting the point cloud, the Kinect camera coordinate system is used to calibrate the color and depth images, so the collected point cloud has the attributes of color and coordinate position. The fruit model is displayed from the front, bottom, and side, respectively, in the experiment. In Figure A1, from left to right is the front image of fruit, the registering model of the fruit point cloud, the front of 3D model, the bottom image of fruit, the bottom of the 3D model, and the overall appearance of the 3D model. Through the observation of the model by the bounding-box method, it can be seen that the method provided in this experiment can correctly fit the 3D model of fruit. It can not only measure the morphological parameters, such as length, height, width, perimeter but also reflect the color and texture characteristics.

5. Conclusions

The three-dimensional fruit model is obtained by point cloud modeling from different coordinate systems. Eight surface point clouds are registered by the ICP algorithm, the bottom point cloud coordinate system and the surface point cloud coordinate system are unified to get the whole point cloud of the fruit. The fruit model is classified according to its stalk, the height of the fruit without stalk is calculated directly, while the height of fruit with stalk is calculated after further processing. The parameters of length, height, width, and centroid are obtained by the bounding-box method. Through the tangent planes to the fruit model of the centroid and main direction x, y, and z, the contours of the fruit is obtained.

The innovation of this experiment is to obtain a complete model, not just a front surface model of the fruit. Secondly, for the measurement of fruit with stalk, the concavity–convexity relationship between adjacent patches is used as the segmentation judgment standard to remove the fruit stalk to improve the accuracy of height measurement. Finally, the effectiveness of the algorithm is verified by comparing the measured values of the length of the bounding box method and the perimeter of the fruit with the actual values.

In this paper, the length, height, width and three-dimensional perimeters of the fruit are measured and compared with the actual values. In the first group of the experiment, pears with stalks are taken as the research subjects, the concavity and convexity of the LCCP method are used to remove the stalk, then the phenotype parameters of the model are calculated. In the second group, a variety of fruits are taken as the research objects, and the phenotype parameters of fruits with different shapes, color, and texture are measured by our method.

Through the experimental results, it can be seen that the experimental method in this paper is universal and can complete the phenotype parameters measurement of many kinds of fruits.