当前位置: X-MOL 学术ISPRS J. Photogramm. Remote Sens. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Three-dimensional photogrammetry with deep learning instance segmentation to extract berry fruit harvestability traits
ISPRS Journal of Photogrammetry and Remote Sensing ( IF 10.6 ) Pub Date : 2020-12-10 , DOI: 10.1016/j.isprsjprs.2020.11.010
Xueping Ni , Changying Li , Huanyu Jiang , Fumiomi Takeda

Fruit cluster characteristics such as compactness, maturity, berry number, and berry size, are important phenotypic traits associated with harvestability and yield of blueberry genotypes and can be used to monitor berry development and improve crop management. The goal of this study was to develop a complete framework of 3D segmentation for individual blueberries as they develop in clusters and to extract blueberry cluster traits. To achieve this goal, an image-capturing system was developed to capture blueberry images to facilitate 3D reconstruction and a 2D-3D projection-based photogrammetric pipeline was proposed to extract berry cluster traits. The reconstruction was performed for four southern highbush blueberry cultivars (‘Emerald’, ‘Farthing’, ‘Meadowlark’ and ‘Star’) with 10 cluster samples for each cultivar based on photogrammetry. A minimum bounding box was created to surround a 3D blueberry cluster to calculate compactness as the ratio of berry volume and minimum bounding box volume. Mask R-CNN was used to segment individual blueberries with the maturity property from 2D images and the instance masks were projected onto 3D point clouds to establish 2D-3D correspondences. The developed trait extraction algorithm was used to segment individual 3D blueberries to obtain berry number, individual berry volume, and berry maturity. Berry maturity was used to calculate cluster maturity as the ratio of the mature berry (blue colored fruit) number and the total berry (blue, reddish, and green colored fruit) number comprising the cluster. The accuracy of determining the fruit number in a cluster is 97.3%. The linear regression for cluster maturity has a R2 of 0.908 with a RMSE of 0.068. The cluster berry volume has a RMSE of 2.92 cm3 compared with the ground truth, indicating that the individual berry volume has an error of less than 0.292 cm3 for clusters with a berry number greater than 10. The statistical analyses of the traits for the four cultivars reveals that, in the middle of April, ‘Emerald’ and ‘Farthing’ were more compact than ‘Meadowlark’ and ‘Star’, and the mature berry volume of ‘Farthing’ was greater than ‘Emerald’ and ‘Meadowlark’, while ‘Star’ had the smallest mature berry size. This study develops an effective method based on 3D photogrammetry and 2D instance segmentation that can determine blueberry cluster traits accurately from a large number of samples and can be used for fruit development monitoring, yield estimation, and harvest time prediction.



中文翻译:

三维摄影测量与深度学习实例分割以提取浆果的可收获性状

果实簇的特征,例如紧密度,成熟度,浆果数和浆果大小,是与蓝莓基因型的可收获性和产量相关的重要表型性状,可用于监测浆果的发育并改善作物管理。这项研究的目的是为单个蓝莓在簇中发育时开发一个完整的3D分割框架,并提取蓝莓簇的特征。为了实现此目标,开发了一种图像捕获系统来捕获蓝莓图像以促进3D重建,并提出了一种基于2D-3D投影的摄影测量管道来提取浆果簇特征。基于摄影测量,对四个南部高灌木蓝莓品种(“翡翠”,“远星”,“ Meadowlark”和“星”)进行了重建,每个品种有10个簇样本。创建了包围3D蓝莓簇的最小边界框,以浆果的体积与最小边界框的体积之比来计算紧密度。遮罩R-CNN用于从2D图像中分割具有成熟属性的蓝莓,并将实例遮罩投影到3D点云上以建立2D-3D对应关系。所开发的特征提取算法用于对单个3D蓝莓进行细分,以获得浆果数量,单个浆果体积和浆果成熟度。浆果成熟度用于计算群集成熟度,即成熟浆果(蓝色水果)数量与构成群集的总浆果(蓝色,红色和绿色水果)数量之比。确定簇中果实数目的准确性为97.3%。聚类成熟度的线性回归具有[R20.908,RMSE为0.068。群集浆果卷的RMSE为2.92厘米3 与地面真相比较,表明单个浆果体积的误差小于0.292 厘米3对于浆果数大于10的集群,对这四个品种的性状的统计分析表明,在4月中旬,“翡翠”和“远缘”比“ Meadowlark”和“ Star”更紧凑,而“ Farthing”的成熟浆果体积大于“ Emerald”和“ Meadowlark”,而“ Star”具有最小的成熟浆果尺寸。这项研究开发了一种基于3D摄影测量和2D实例分割的有效方法,该方法可以从大量样本中准确确定蓝莓的群集特征,并可以用于水果发育监控,产量估算和收获时间预测。

更新日期:2020-12-10
down
wechat
bug