Reliable obstacle detection and classification in rough and unstructured terrain such as agricultural fields or orchards remains a challenging problem. These environments involve large variations in both geometry and appearance, challenging perception systems that rely on only a single sensor modality. Geometrically, tall grass, fallen leaves, or terrain roughness can mistakenly be perceived as nontraversable or might even obscure actual obstacles. Likewise, traversable grass or dirt roads and obstacles such as trees and bushes might be visually ambiguous. In this paper, we combine appearance- and geometry-based detection methods by probabilistically fusing lidar and camera sensing with semantic segmentation using a conditional random field. We apply a state-of-the-art multimodal fusion algorithm from the scene analysis domain and adjust it for obstacle detection in agriculture with moving ground vehicles. This involves explicitly handling sparse point cloud data and exploiting both spatial, temporal, and multimodal links between corresponding 2D and 3D regions. The proposed method was evaluated on a diverse data set, comprising a dairy paddock and different orchards gathered with a perception research robot in Australia. Results showed that for a two-class classification problem (ground and nonground), only the camera leveraged from information provided by the other modality with an increase in the mean classification score of 0.5%. However, as more classes were introduced (ground, sky, vegetation, and object), both modalities complemented each other with improvements of 1.4% in 2D and 7.9% in 3D. Finally, introducing temporal links between successive frames resulted in improvements of 0.2% in 2D and 1.5% in 3D.

中文翻译：

---

具有条件随机场的非结构化环境中的多模态障碍检测

在农田或果园等粗糙和非结构化地形中进行可靠的障碍物检测和分类仍然是一个具有挑战性的问题。这些环境涉及几何形状和外观的巨大变化，对仅依赖单一传感器模式的感知系统提出了挑战。从几何上讲，高草、落叶或地形崎岖可能会被错误地视为不可穿越，甚至可能掩盖实际障碍。同样，可穿越的草地或土路以及树木和灌木丛等障碍物在视觉上可能不明确。在本文中，我们通过使用条件随机场将激光雷达和相机传感与语义分割进行概率融合，将基于外观和几何的检测方法相结合。我们从场景分析领域应用了最先进的多模态融合算法，并对其进行了调整，以用于具有移动地面车辆的农业中的障碍物检测。这涉及明确处理稀疏点云数据并利用相应 2D 和 3D 区域之间的空间、时间和多模态链接。所提出的方法在不同的数据集上进行了评估，包括一个奶牛场和不同的果园，这些数据集是由澳大利亚的感知研究机器人收集的。结果表明，对于两类分类问题（地面和非地面），只有相机利用了其他模态提供的信息，平均分类得分增加了 0.5%。然而，随着更多类的引入（地面、天空、植被和物体），两种模式相互补充，改进为 1。2D 为 4%，3D 为 7.9%。最后，在连续帧之间引入时间链接导致 2D 和 3D 分别提高 0.2% 和 1.5%。