Excavators are widely used for material handling applications in unstructured environments, including mining and construction. Operating excavators in a real-world environment can be challenging due to extreme conditions—such as rock sliding, ground collapse, or excessive dust—and can result in fatalities and injuries. Here, we present an autonomous excavator system (AES) for material loading tasks. Our system can handle different environments and uses an architecture that combines perception and planning. We fuse multimodal perception sensors, including LiDAR and cameras, along with advanced image enhancement, material and texture classification, and object detection algorithms. We also present hierarchical task and motion planning algorithms that combine learning-based techniques with optimization-based methods and are tightly integrated with the perception modules and the controller modules. We have evaluated AES performance on compact and standard excavators in many complex indoor and outdoor scenarios corresponding to material loading into dump trucks, waste material handling, rock capturing, pile removal, and trenching tasks. We demonstrate that our architecture improves the efficiency and autonomously handles different scenarios. AES has been deployed for real-world operations for long periods and can operate robustly in challenging scenarios. AES achieves 24 hours per intervention, i.e., the system can continuously operate for 24 hours without any human intervention. Moreover, the amount of material handled by AES per hour is closely equivalent to an experienced human operator.

挖掘机广泛用于非结构化环境中的物料搬运应用，包括采矿和建筑。由于极端条件（例如岩石滑动、地面坍塌或灰尘过多），在真实环境中操作挖掘机可能具有挑战性，并且可能导致人员伤亡。在这里，我们展示了一种用于材料装载任务的自主挖掘机系统 (AES)。我们的系统可以处理不同的环境，并使用结合感知和规划的架构。我们融合了多模态感知传感器，包括 LiDAR 和相机，以及先进的图像增强、材料和纹理分类以及物体检测算法。我们还提出了分层任务和运动规划算法，将基于学习的技术与基于优化的方法相结合，并与感知模块和控制器模块紧密集成。我们已经在许多复杂的室内和室外场景中评估了紧凑型和标准挖掘机的 AES 性能，这些场景对应于将材料装载到自卸车、废料处理、岩石捕获、清除桩和挖沟任务。我们证明我们的架构提高了效率并自主处理不同的场景。AES 已长期部署用于实际操作，并且可以在具有挑战性的场景中稳健运行。AES 实现了每次干预 24 小时，即系统可以在没有任何人为干预的情况下连续运行 24 小时。而且，