Fresh market apples are one of the high‐value crops in the United States. Washington alone has produced two‐thirds of the annual national production in the past 10 years. However, the availability of seasonal labor is increasingly uncertain. Shake‐and‐catch automated harvesting solutions have, therefore, become attractive for addressing this challenge. One of the significant challenges in applying this harvesting system is effectively positioning the end‐effector at appropriate excitation locations. A computer vision system was used for automatically identifying appropriate locations. Convolutional neural networks (CNNs) were utilized to identify the tree trunks and branches for supporting the automated excitation locations determination. Three CNN architectures were employed: Deeplab v3+ ResNet‐18, VGG‐16, and VGG‐19. Four pixel classes were predefined as branches, trunks, apples, and leaves to segment the canopies trained to simple, narrow, accessible, and productive tree architectures with varying foliage density. Results on Fuji cultivar showed that ResNet‐18 outperformed the VGGs in identifying branches and trunks based on all three evaluation measures: per‐class accuracy (PcA), intersection over union (IoU), and boundary‐F1 score (BFScore). ResNet‐18 achieved a PcA of 97%, IoU of 0.69, and BFScore of 0.89. The ResNet‐18 was further evaluated for its robustness with other test canopy images. When applied this method to one of the highest density cultivars of Scifresh, results showed it can achieve IoUs of 0.41 and 0.62 and BFScores of 0.71 and 0.86 for branches and trunks. Such identification result helped to get a 72% of auto‐determined shaking points being the “good” category identified by human experts.

中文翻译：

---

基于计算机视觉的树干和树枝识别以及茂密植物冠层中的摇动点检测，可自动收获苹果

新鲜市场苹果是美国的高价值农作物之一。在过去的十年中，仅华盛顿一国就生产了该国国民生产总值的三分之二。但是，季节性劳动力的可用性越来越不确定。因此，摇动捕集式自动收割解决方案对于应对这一挑战变得具有吸引力。应用这种收割系统的重大挑战之一是将末端执行器有效地定位在适当的激发位置。使用计算机视觉系统来自动识别适当的位置。卷积神经网络（CNN）用于识别树干和树枝，以支持自动激发位置的确定。使用了三种CNN架构：Deeplab v3 + ResNet-18，VGG-16和VGG-19。预先定义了四个像素类别，分别是树枝，树干，苹果和树叶，以将树冠分割成具有不同树叶密度的简单，狭窄，可访问且富有生产力的树木结构。富士品种的结果表明，在所有三种评估方法的基础上，ResNet‐18在识别分支和主干方面优于VGG，这是基于每类准确性（PcA），交集相交（IoU）和边界F1分数（BFScore）。ResNet-18的PcA为97％，IoU为0.69，BFScore为0.89。ResNet-18与其他测试顶篷图像相比，经过了进一步的鲁棒性评估。当将该方法应用于Scifresh的一个最高密度的品种之一时，结果表明它可以实现分支和主干的IoU为0.41和0.62，BFScore为0.71和0.86。