The images captured by vision-based tactile sensors carry information about high-resolution tactile fields, such as the distribution of the contact forces applied to their soft sensing surface. However, extracting the information encoded in the images is challenging and often addressed with learning-based approaches, which generally require a large amount of training data. This article proposes a strategy to generate tactile images in simulation for a vision-based tactile sensor based on an internal camera that tracks the motion of spherical particles within a soft material. The deformation of the material is simulated in a finite element environment under a diverse set of contact conditions, and spherical particles are projected to a simulated image. Features extracted from the images are mapped to the three-dimensional contact force distribution, with the ground truth also obtained using finite-element simulations, with an artificial neural network that is therefore entirely trained on synthetic data avoiding the need for real-world data collection. The resulting model exhibits high accuracy when evaluated on real-world tactile images, is transferable across multiple tactile sensors without further training, and is suitable for efficient real-time inference.

中文翻译：

---

用于高分辨率光学触觉传感的 Sim-to-Real：从图像到三维接触力分布

基于视觉的触觉传感器捕获的图像携带有关高分辨率触觉场的信息，例如施加到其软传感表面的接触力的分布。然而，提取图像中编码的信息具有挑战性，并且通常使用基于学习的方法来解决，这通常需要大量的训练数据。本文提出了一种在模拟中为基于视觉的触觉传感器生成触觉图像的策略，该传感器基于跟踪软材料内球形粒子运动的内部摄像头。材料的变形在有限元环境中在各种接触条件下进行模拟，并将球形颗粒投影到模​​拟图像上。从图像中提取的特征被映射到三维接触力分布，使用有限元模拟也获得了基本事实，人工神经网络因此完全接受了合成数据的训练，从而避免了对现实世界数据收集的需要。所得模型在对真实世界的触觉图像进行评估时表现出高精度，无需进一步训练即可跨多个触觉传感器转移，并且适用于有效的实时推理。