For robotic precision spray application in vegetable farms, simultaneous accurate instance segmentation and robust tracking of plants are of great importance and a prerequisite for the following spray action. With onboard cameras, agricultural robots can apply Multiple Object Tracking and Segmentation (MOTS) methods, for instance, segmentation and tracking of plants. By assigning a unique identification for each vegetable, it ensures the robot to spray each vegetable exactly once, while traversing along the farm rows. Conventional MOTS methods, which are mostly designed for tracking pedestrians or vehicles, usually extract their color and texture features for associating different targets in consecutive images. However, vegetable plants of the same species normally show similar color and texture, which leads to degraded performance when conventional MOTS methods are used. To solve the challenging problem of associating vegetables with similar color and texture in consecutive images, in this paper, a novel MOTS method that exploits contour and blob features is proposed, for instance, segmentation and tracking of multiple vegetable plants. The method takes advantage of the fact that different plants normally possess different shape contours and blob properties. With images captured on top of them, these features of the same plant show little difference in consecutively captured images. Comprehensive experiments including ablation studies are conducted, which prove its superior performance over two state-of-the-art MOTS methods. Compared with the conventional MOTS methods, the proposed method is able to re-identify objects which have gone out of the camera field of view and re-appear again using the proposed data association strategy, which is important to ensure each vegetable be sprayed only once when the robot travels back and forth. Although the method is tested on lettuce farm, it can be applied to other similar vegetables, such as broccoli and canola. Both the code and the dataset of this paper are publicly released for the benefit of the community: https://github.com/NanH5837/LettuceMOTS.

中文翻译：

---

利用蔬菜形状特征进行蔬菜植株分割与跟踪，实现农业机器人精准喷洒

对于蔬菜农场中的机器人精准喷雾应用，同步精确的实例分割和稳健的植物跟踪非常重要，也是后续喷雾动作的先决条件。借助机载摄像头，农业机器人可以应用多目标跟踪和分割（MOTS）方法，例如植物的分割和跟踪。通过为每种蔬菜分配唯一的标识，它可以确保机器人在沿着农场行进时对每种蔬菜精确喷洒一次。传统的 MOTS 方法主要用于跟踪行人或车辆，通常提取其颜色和纹理特征以关联连续图像中的不同目标。然而，同一物种的蔬菜植物通常表现出相似的颜色和纹理，这导致使用传统 MOTS 方法时性能下降。为了解决在连续图像中关联具有相似颜色和纹理的蔬菜的挑战性问题，本文提出了一种利用轮廓和斑点特征的新颖的 MOTS 方法，例如对多个蔬菜植物的分割和跟踪。该方法利用了不同植物通常具有不同形状轮廓和斑点特性的事实。通过在它们顶部捕获的图像，同一植物的这些特征在连续捕获的图像中几乎没有差异。进行了包括消融研究在内的综合实验，证明其性能优于两种最先进的 MOTS 方法。与传统的 MOTS 方法相比，所提出的方法能够使用所提出的数据关联策略重新识别已经超出相机视野并再次出现的物体，这对于确保每种蔬菜仅喷洒一次非常重要当机器人来回移动时。尽管该方法是在生菜农场进行测试的，但它也可以应用于其他类似的蔬菜，例如西兰花和油菜。本文的代码和数据集均已公开发布，以造福社区：https://github.com/NanH5837/LettuceMOTS。