This work develops a novel high-dimensional inverse reinforcement learning (IRL) algorithm for human motion analysis in medical, clinical, and robotics applications. The method is based on the assumption that a surgical robot operators’ skill or a patient’s motor skill is encoded into the innate reward function during motion planning and recovered by an IRL algorithm from motion demonstrations. This class of applications is characterized by high-dimensional sensory data, which is computationally prohibitive for most existing IRL algorithms. We propose a novel function approximation framework and reformulate the Bellman optimality equation to handle high-dimensional state spaces efficiently. We compare different function approximators in simulated environments, and adopt a deep neural network as the function approximator. The technique is applied to evaluating human patients with spinal cord injuries under spinal stimulation, and the skill levels of surgical robot operators. The results demonstrate the efficiency and effectiveness of the proposed method.

中文翻译：

---

通过高维逆强化学习在医疗机器人中进行人体运动分析

这项工作开发了一种新颖的高维逆强化学习 (IRL) 算法，用于医学、临床和机器人应用中的人体运动分析。该方法基于以下假设：手术机器人操作员的技能或患者的运动技能在运动规划期间被编码到先天奖励函数中，并通过 IRL 算法从运动演示中恢复。这类应用程序的特点是高维感官数据，这对于大多数现有的 IRL 算法来说在计算上是令人望而却步的。我们提出了一种新的函数逼近框架，并重新制定了 Bellman 最优方程以有效处理高维状态空间。我们在模拟环境中比较不同的函数逼近器，并采用深度神经网络作为函数逼近器。该技术用于评估脊髓刺激下脊髓损伤的人类患者，以及手术机器人操作员的技能水平。结果证明了所提出方法的效率和有效性。