Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates a rich variety of scenarios including different object sizes, object numbers and manipulation actions. We approach dynamics learning by proposing an object-centric graph representation and two modules which are Active Prediction Module (APM) and Position Prediction Module (PPM) based on graph neural networks with an encode-process-decode architecture. At the inference stage, we build a two-stage model based on the learned modules for single time step prediction. We combine modules with different prediction horizons into a mixed-horizon model which addresses long-term prediction. In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks. Supplementary material is available at https://github.com/wengzehang/deformable_rigid_interaction_prediction

中文翻译：

---

基于图的特定于任务的预测模型，用于可变形对象与刚性对象之间的相互作用

捕获场景动态并预测将来的场景状态具有挑战性，但对于机器人操纵任务来说是必不可少的，特别是当场景同时包含刚性和可变形对象时。在这项工作中，我们贡献了一个模拟环境并生成了一个新的数据集，用于特定任务的处理，涉及刚性物体和可变形袋之间的相互作用。数据集包含多种场景，包括不同的对象大小，对象编号和操作动作。我们通过提出一种以对象为中心的图形表示和两个模块（基于模型神经网络和编码过程解码架构的主动预测模块（APM）和位置预测模块（PPM））来进行动力学学习。在推论阶段，我们基于学习的模块构建了一个两阶段模型，用于单时步预测。我们将具有不同预测范围的模块组合到一个混合水平模型中，该模型可解决长期预测问题。在消融研究中，我们展示了两阶段模型对单时间步长预测的好处，以及混合水平模型对长期预测任务的有效性。补充材料可在https://github.com/wengzehang/deformable_rigid_interaction_prediction获得