Robot grasping and manipulation require estimation of 3D object poses. Recently, a number of methods and datasets for vision-based pose estimation have been proposed. However, it is unclear how well the performance measures developed for visual pose estimation predict success in robot manipulation. In this work, we introduce an approach that connects error in pose and success in robot manipulation, and propose a probabilistic performance measure of the task success rate. A physical setup is needed to estimate the probability densities from real world samples, but evaluation of pose estimation methods is offline using captured test images, ground truth poses and the estimated densities. We validate the approach with four industrial manipulation tasks and evaluate a number of publicly available pose estimation methods. The popular pose estimation performance measure, Average Distance of Corresponding model points (ADC), does not offer any quantitatively meaningful indication of the frequency of success in robot manipulation. Our measure is instead quantitatively informative: e.g., a score of 0.24 corresponds to average success probability of 24%.

机器人抓取和操纵需要估计 3D 对象姿势。最近，已经提出了许多用于基于视觉的姿势估计的方法和数据集。然而，目前尚不清楚为视觉姿态估计开发的性能指标如何预测机器人操作的成功。在这项工作中，我们引入了一种将姿势错误与机器人操作成功联系起来的方法，并提出了任务成功率的概率性能度量。需要物理设置来估计真实世界样本的概率密度，但使用捕获的测试图像、地面实况姿势和估计的密度对姿势估计方法的评估是离线的。我们通过四个工业操作任务验证了该方法，并评估了许多公开可用的姿势估计方法。流行的姿态估计性能指标，对应模型点的平均距离 (ADC)，并没有提供任何关于机器人操作成功频率的定量有意义的指示。我们的衡量标准是定量提供信息：例如，0.24 的分数对应于 24% 的平均成功概率。