Automating building processes through robotic systems has the potential to address the need for safer, more efficient, and sustainable construction operations. While ongoing research effort often targets the use of prefabricated materials in controlled environments, here we focus on utilizing objects found on-site, such as irregularly shaped rocks and rubble, as a way of enabling novel types of construction in remote and extreme environments, where standard building materials might not be easily accessible. In this article, we present a perception and grasp pose planning pipeline for autonomous manipulation of objects of interest with a robotic walking excavator. The system incrementally builds a LiDAR-based map of the robot's surroundings and provides the ability to register externally reconstructed point clouds of the scene, for example, from images captured by a drone-borne camera, which helps increasing map coverage. In addition, object-like instances, such as stones, are segmented out of this map. Based on this information, collision-free grasping poses for the robotic manipulator are planned to enable picking and placing of these objects, while keeping track of them during the manipulation. The approach is validated in a real setting on an architectural relevant scale by segmenting and manipulating boulders of several hundred kilograms, which is a first step towards the full automation of dry-stack wall building processes. Video – https://youtu.be/4bc5n2-zj3Q

中文翻译：

---

走向自动化构建任务：大规模对象映射、分割和操作

通过机器人系统自动化建筑流程有可能满足对更安全、更高效和可持续的建筑运营的需求。虽然正在进行的研究工作通常针对在受控环境中使用预制材料，但在这里我们专注于利用现场发现的物体，例如形状不规则的岩石和瓦砾，作为在偏远和极端环境中实现新型建筑的一种方式，其中标准建筑材料可能不容易获得。在本文中，我们提出了一种感知和掌握姿势规划管道，用于使用机器人步行挖掘机自主操纵感兴趣的对象。该系统逐步构建机器人周围环境的基于激光雷达的地图，并提供注册外部重建场景点云的能力，例如，来自无人机携带的相机拍摄的图像，这有助于增加地图覆盖范围。此外，从该地图中分割出类似物体的实例，例如石头。根据这些信息，机器人操纵器的无碰撞抓取姿势计划能够拾取和放置这些物体，同时在操纵过程中跟踪它们。通过分割和操作数百公斤的巨石，该方法在建筑相关规模的真实环境中得到验证，这是实现干堆墙建造过程完全自动化的第一步。机器人操纵器的无碰撞抓取姿势计划能够拾取和放置这些物体，同时在操纵过程中跟踪它们。通过分割和操作数百公斤的巨石，该方法在建筑相关规模的真实环境中得到验证，这是实现干堆墙建造过程完全自动化的第一步。机器人操纵器的无碰撞抓取姿势计划能够拾取和放置这些物体，同时在操纵过程中跟踪它们。通过分割和操作数百公斤的巨石，该方法在建筑相关规模的真实环境中得到验证，这是实现干堆墙建造过程完全自动化的第一步。视频– https://youtu.be/4bc5n2-zj3Q