As the environments that human live are complex and uncontrolled, the object manipulation with humanoid robots is regarded as one of the most challenging tasks. Learning a manipulation skill from human Demonstration (LfD) is one of the popular methods in the artificial intelligence and robotics community. This paper introduces a deep learning based teleoperation system for humanoid robots that imitate the human operator’s object manipulation behavior. One of the fundamental problems in LfD is to approximate the robot trajectories obtained by means of human demonstrations with high accuracy. The work introduces novel models based on Convolutional Neural Networks (CNNs), CNNs-Long Short-Term Memory (LSTM) models combining the CNN LSTM models, and their scaled variants for object manipulation with humanoid robots by using LfD. In the proposed LfD system, six models are employed to estimate the shoulder roll position of the humanoid robot. The data are first collected in terms of teleoperation of a real Robotis-Op3 humanoid robot and the models are trained. The trajectory estimation is then carried out by the trained CNNs and CNN-LSTM models on the humanoid robot in an autonomous way. All trajectories relating the joint positions are finally generated by the model outputs. The results relating to the six models are compared to each other and the real ones in terms of the training and validation loss, the parameter number, and the training and testing time. Extensive experimental results show that the proposed CNN models are well learned the joint positions and especially the hybrid CNN-LSTM models in the proposed teleoperation system exhibit a more accuracy and stable results.

由于人类生活的环境复杂且不受控制，人形机器人的物体操纵被认为是最具挑战性的任务之一。从人类演示 (LfD) 中学习操作技能是人工智能和机器人社区中流行的方法之一。本文介绍了一种基于深度学习的仿人机器人遥操作系统，该系统模仿人类操作者的物体操作行为。LfD 的基本问题之一是高精度地逼近通过人类演示获得的机器人轨迹。这项工作介绍了基于卷积神经网络 (CNN) 的新模型、结合了 CNN LSTM 模型的 CNN-长短期记忆 (LSTM) 模型，以及它们的缩放变体，用于使用 LfD 对类人机器人进行对象操作。在所提出的 LfD 系统中，采用六个模型来估计仿人机器人的肩部侧倾位置。首先根据真实 Robotis-Op3 人形机器人的遥控操作收集数据，并对模型进行训练。然后由训练有素的 CNN 和 CNN-LSTM 模型在仿人机器人上以自主方式进行轨迹估计。与关节位置相关的所有轨迹最终由模型输出生成。将与六个模型相关的结果在训练和验证损失、参数数量以及训练和测试时间方面相互比较并与真实模型进行比较。