In this paper, we propose the problem of collaborative perception, where robots can combine their local observations with those of neighboring agents in a learnable way to improve accuracy on a perception task. Unlike existing work in robotics and multi-agent reinforcement learning, we formulate the problem as one where learned information must be shared across a set of agents in a bandwidth-sensitive manner to optimize for scene understanding tasks such as semantic segmentation. Inspired by networking communication protocols, we propose a multi-stage handshake communication mechanism where the neural network can learn to compress relevant information needed for each stage. Specifically, a target agent with degraded sensor data sends a compressed request, the other agents respond with matching scores, and the target agent determines who to connect with (i.e., receive information from). We additionally develop the AirSim-CP dataset and metrics based on the AirSim simulator where a group of aerial robots perceive diverse landscapes, such as roads, grasslands, buildings, etc. We show that for the semantic segmentation task, our handshake communication method significantly improves accuracy by approximately 20% over decentralized baselines, and is comparable to centralized ones using a quarter of the bandwidth.

中文翻译：

---

Who2com：通过可学习的握手通信实现协作感知

在本文中，我们提出了协作感知问题，其中机器人可以以一种可学习的方式将其局部观察与相邻代理的观察结合起来，以提高感知任务的准确性。与机器人和多智能体强化学习领域的现有工作不同，我们将问题表述为必须以带宽敏感的方式在一组智能体之间共享学习信息，以优化场景理解任务，例如语义分割。受网络通信协议的启发，我们提出了一种多阶段握手通信机制，神经网络可以学习压缩每个阶段所需的相关信息。具体来说，具有降级传感器数据的目标代理发送压缩请求，其他代理以匹配的分数响应，并且目标代理确定与谁联系（即，从那里接收信息）。我们另外开发了基于 AirSim 模拟器的 AirSim-CP 数据集和指标，其中一组空中机器人感知不同的景观，如道路、草原、建筑物等。我们表明，对于语义分割任务，我们的握手通信方法显着提高准确度比去中心化基线提高约 20%，与使用四分之一带宽的集中式基线相当。