One of the most basic skills a robot should possess is predicting the effect of physical interactions with objects in the environment. This enables optimal action selection to reach a certain goal state. Traditionally, these dynamics are described by physics-based analytical models, which may however be very hard to find for complex problems. More recently, we have seen learning approaches that can predict the effect of more complex physical interactions directly from sensory input. However, it is an open question how far these models generalize beyond their training data. In this work, we analyse how analytical and learned models can be combined to leverage the best of both worlds. As physical interaction task, we use planar pushing, for which there exists a well-known analytical model and a large real-world dataset. We propose to use a neural network to convert the raw sensory data into a suitable representation that can be consumed by the analytical model and compare this approach to using neural networks for both, perception and prediction. Our results show that the combined method outperforms the purely learned version in terms of accuracy and generalization to push actions not seen during training. It also performs comparable to the analytical model applied on ground truth input values, despite using raw sensory data as input.

中文翻译：

---

结合学习模型和分析模型，根据感官数据预测动作效果

机器人应具备的最基本技能之一是预测与环境中物体的物理交互效果。这使得最佳动作选择能够达到某个目标状态。传统上，这些动力学由基于物理的分析模型描述，但是对于复杂问题可能很难找到。最近，我们看到了可以直接从感官输入预测更复杂物理交互影响的学习方法。然而，这些模型在训练数据之外的泛化程度是一个悬而未决的问题。在这项工作中，我们分析了如何将分析模型和学习模型结合起来以利用两全其美。作为物理交互任务，我们使用平面推送，为此存在一个众所周知的分析模型和一个大型的现实世界数据集。我们建议使用神经网络将原始感官数据转换为分析模型可以使用的合适表示，并将这种方法与使用神经网络进行感知和预测进行比较。我们的结果表明，组合方法在准确性和泛化方面优于纯学习版本，以推动训练期间未见的动作。尽管使用原始感官数据作为输入，它的性能也与应用于地面实况输入值的分析模型相当。我们的结果表明，组合方法在准确性和泛化方面优于纯学习版本，以推动训练期间未见的动作。尽管使用原始感官数据作为输入，它的性能也与应用于地面实况输入值的分析模型相当。我们的结果表明，组合方法在准确性和泛化方面优于纯学习版本，以推动训练期间未见的动作。尽管使用原始感官数据作为输入，它的性能也与应用于地面实况输入值的分析模型相当。