Artificial intelligence provides new feasibilities to the control of dexterous prostheses. To achieve suitable grasps over various objects, a novel computer vision-based classification method assorting objects into different grasp patterns is proposed in this paper. This method can be applied in the autonomous control of the multi-fingered prosthetic hand, as it can help users rapidly complete “reach-and-pick up” tasks on various daily objects with low demand on the myoelectric control. Firstly, an RGB-D image database (121 objects) was established according to four important grasp patterns (cylindrical, spherical, tripod, and lateral). The image samples in the RGB-D dataset were acquired on a large variety of daily objects of different sizes, shapes, postures (16), as well as different illumination conditions (4) and camera positions (4). Then, different inputs and structures of the discrimination model (multilayer CNN) were tested in terms of the classification success rate through cross-validation. Our results showed that depth data play an important role in grasp pattern recognition. The bimodal data (Gray-D) integrating both grayscale and depth information about the objects can improve the classification accuracy acquired from the RGB images (> 10%) effectively. Within the database, the network could achieve the classification with high accuracy (98%); it also has a strong generalization capability on novel samples (93.9 ± 3.0%). We finally applied the method on a dexterous prosthetic hand and tested the whole system on performing the “reach-and-pick up” tasks. The experiments showed that the proposed computer vision-based myoelectric control method (Vision-EMG) could significantly improve the control effectiveness (6.4 s) , with comparison to the traditional coding-based myoelectric control method (Coding-EMG, 13 ${s}$ ).

中文翻译：

---

基于计算机视觉的抓握模式识别及其在右手假体肌电控制中的应用

人工智能为灵巧假体的控制提供了新的可行性。为了实现对各种物体的合适抓握，提出了一种基于计算机视觉的分类方法，将物体分为不同的抓握模式。该方法可用于多指假肢手的自主控制，因为它可以帮助用户快速完成对各种日常对象的“伸手拿起”任务，而对肌电控制的需求却很少。首先，根据四个重要的抓取模式（圆柱，球形，三脚架和侧面）建立了RGB-D图像数据库（121个对象）。RGB-D数据集中的图像样本是从各种大小，形状，姿势（16），以及不同的照明条件（4）和相机位置（4）的日常对象中获取的。然后，通过交叉验证，对分类模型（多层CNN）的不同输入和结构进行了分类成功率的测试。我们的结果表明，深度数据在把握模式识别中起着重要作用。集成了有关对象的灰度和深度信息的双峰数据（Gray-D）可以有效地提高从RGB图像（> 10％）获得的分类精度。在数据库内，网络可以实现高精度分类（98％）；它还对新样本具有很强的泛化能力（93.9±3.0％）。最后，我们将该方法应用于灵巧的假肢上，并在执行“伸手拿起”任务时测试了整个系统。s） ，与传统的基于编码的肌电控制方法（Coding-EMG，13 $ {s} $ ）。