We address the problem of shaping deformable plastic materials using non-prehensile actions. Shaping plastic objects is challenging, because they are difficult to model and to track visually. We study this problem, by using kinetic sand, a plastic toy material that mimics the physical properties of wet sand. Inspired by a pilot study where humans shape kinetic sand, we define two types of actions: pushing the material from the sides and tapping from above. The chosen actions are executed with a robotic arm using image-based visual servoing. From the current and desired view of the material, we define states based on visual features such as the outer contour shape and the pixel luminosity values. These are mapped to actions, which are repeated iteratively to reduce the image error until convergence is reached. For pushing, we propose three methods for mapping the visual state to an action. These include heuristic methods and a neural network, trained from human actions. We show that it is possible to obtain simple shapes with the kinetic sand, without explicitly modeling the material. Our approach is limited in the types of shapes it can achieve. A richer set of action types and multi-step reasoning is needed to achieve more sophisticated shapes.

中文翻译：

---

可变形塑料材料的无模型视觉成型

我们解决了使用非抓握动作塑造可变形塑料材料的问题。塑造塑料物体具有挑战性，因为它们难以建模和视觉跟踪。我们通过使用动力沙来研究这个问题，这是一种模拟湿沙物​​理特性的塑料玩具材料。受人类塑造动态沙子的试点研究的启发，我们定义了两种类型的动作：从侧面推动材料和从上方敲击。选择的动作是使用基于图像的视觉伺服的机械臂执行的。从材料的当前和所需视图中，我们根据外部轮廓形状和像素亮度值等视觉特征定义状态。这些被映射到动作，这些动作被迭代地重复以减少图像误差直到达到收敛。为了推动，我们提出了三种将视觉状态映射到动作的方法。这些包括启发式方法和从人类行为训练的神经网络。我们表明可以使用动力砂获得简单的形状，而无需对材料进行明确建模。我们的方法在它可以实现的形状类型方面受到限制。需要更丰富的动作类型和多步推理才能实现更复杂的形状。