Manipulative action recognition is one of the most important and challenging topic in the fields of image processing. In this paper, three kinds of sensor modules are used for motion, force and object information capture in the manipulative actions. Two fusion methods are proposed. Further, the recognition accuracy can be improved by using object as context. For the feature-level fusion method, significant features are chosen first. Then the Hidden Markov Models are built with these selected features to characterize the temporal sequence. For the decision-level fusion method, HMMs are built for each feature group. Then the decisions are fused. On top of these two fusion methods, the object/action context is modeled using Bayesian network. Assembly tasks are used for algorithm evaluation. The experimental results prove that the proposed approach is effective on manipulative action recognition task. The recognition accuracy of the decision-level, feature-level fusion methods and the Bayesian model are 72%, 80% and 90% respectively.

操纵动作识别是图像处理领域中最重要和最具挑战性的主题之一。在本文中，三种传感器模块用于在操纵操作中捕获运动，力和对象信息。提出了两种融合方法。此外，通过使用对象作为上下文可以提高识别精度。对于特征级融合方法，首先选择重要特征。然后利用这些选定的特征构建隐马尔可夫模型，以表征时间序列。对于决策级融合方法，将为每个功能组构建HMM。然后将决策融合在一起。在这两种融合方法之上，使用贝叶斯网络对对象/动作上下文进行建模。组装任务用于算法评估。实验结果证明了该方法对操纵动作识别任务的有效性。决策级，特征级融合方法和贝叶斯模型的识别准确率分别为72％，80％和90％。