We investigate whether a robot arm can learn to pick and throw arbitrary rigid objects into selected boxes quickly and accurately. Throwing has the potential to increase the physical reachability and picking speed of a robot arm. However, precisely throwing arbitrary objects in unstructured settings presents many challenges: from acquiring objects in grasps suitable for reliable throwing, to handling varying object-centric properties (e.g., mass distribution, friction, shape) and complex aerodynamics. In this work, we propose an end-to-end formulation that jointly learns to infer control parameters for grasping and throwing motion primitives from visual observations (RGB-D images of arbitrary objects in a bin) through trial and error. Within this formulation, we investigate the synergies between grasping and throwing (i.e., learning grasps that enable more accurate throws) and between simulation and deep learning (i.e., using deep networks to predict residuals on top of control parameters predicted by a physics simulator). The resulting system, TossingBot, is able to grasp and successfully throw arbitrary objects into boxes located outside its maximum reach range at 500+ mean picks per hour (600+ grasps per hour with 85% throwing accuracy); and generalizes to new objects and target locations.

中文翻译：

---

TossingBot：学习用残差物理扔任意物体

我们研究了机器人手臂是否可以学会快速准确地将任意刚性物体拾取并扔进选定的盒子中。投掷有可能提高机械臂的物理可达性和拾取速度。然而，在非结构化环境中精确投掷任意物体会带来许多挑战：从抓取适合可靠投掷的物体，到处理以物体为中心的各种属性（例如，质量分布、摩擦、形状）和复杂的空气动力学。在这项工作中，我们提出了一种端到端的公式，该公式通过反复试验共同学习从视觉观察（垃圾箱中任意对象的 RGB-D 图像）中推断用于抓取和投掷运动基元的控制参数。在这个公式中，我们研究了抓握和投掷之间的协同作用（即，学习掌握可以实现更准确的投掷）以及模拟和深度学习（即，使用深度网络预测物理模拟器预测的控制参数之上的残差）。由此产生的系统 TossingBot 能够以每小时 500+ 次平均拾取（每小时 600+ 次抓取，投掷准确度为 85%）的速度抓取并成功地将任意物体扔进位于其最大触及范围之外的盒子中；并推广到新的对象和目标位置。能够以每小时 500+ 次平均拾取（每小时 600+ 次抓取，85% 的投掷准确率）的速度抓取并成功地将任意物体扔进位于其最大触及范围之外的盒子中；并推广到新的对象和目标位置。能够以每小时 500+ 次平均拾取（每小时 600+ 次抓取，85% 的投掷准确率）的速度抓取并成功地将任意物体扔进位于其最大触及范围之外的盒子中；并推广到新的对象和目标位置。