High-level spinal cord injuries often result in paralysis of all four limbs, leading to decreased patient independence and quality of life. Coordinated functional electrical stimulation (FES) of paralyzed muscles can be used to restore some motor function in the upper extremity. To coordinate functional movements, FES controllers should be developed to exploit the complex characteristics of human movement and produce the intended movement kinematics and/or kinetics. Here, we demonstrate the ability of a controller trained using reinforcement learning to generate desired movements of a horizontal planar musculoskeletal model of the human arm with 2 degrees of freedom and 6 actuators. The controller is given information about the kinematics of the arm, but not the internal state of the actuators. In particular, we demonstrate that a technique called “hindsight experience replay” can improve controller performance while also decreasing controller training time.

中文翻译：

---

事后经验回放改进了用于控制人臂 MIMO 肌肉骨骼模型的强化学习


高位脊髓损伤通常会导致四肢瘫痪，导致患者独立性和生活质量下降。对瘫痪肌肉进行协调功能性电刺激 (FES) 可用于恢复上肢的部分运动功能。为了协调功能运动，应开发 FES 控制器来利用人体运动的复杂特征并产生预期的运动运动学和/或动力学。在这里，我们展示了使用强化学习训练的控制器能够生成具有 2 个自由度和 6 个执行器的人类手臂水平平面肌肉骨骼模型所需的运动。控制器获得有关手臂运动学的信息，但不知道执行器的内部状态。特别是，我们证明了一种称为“事后经验重播”的技术可以提高控制器性能，同时减少控制器训练时间。