To diagnose, plan, and treat musculoskeletal pathologies, understanding and reproducing muscle recruitment for complex movements is essential. With muscle activations for movements often being highly redundant, nonlinear, and time dependent, machine learning can provide a solution for their modeling and control for anatomy-specific musculoskeletal simulations. Sophisticated biomechanical simulations often require specialized computational environments, being numerically complex and slow, hindering their integration with typical deep learning frameworks. In this work, a deep reinforcement learning (DRL) based inverse dynamics controller is trained to control muscle activations of a biomechanical model of the human shoulder. In a generalizable end-to-end fashion, muscle activations are learned given current and desired position-velocity pairs. A customized reward functions for trajectory control is introduced, enabling straightforward extension to additional muscles and higher degrees of freedom. Using the biomechanical model, multiple episodes are simulated on a cluster simultaneously using the evolving neural models of the DRL being trained. Results are presented for a single-axis motion control of shoulder abduction for the task of following randomly generated angular trajectories.

中文翻译：

---

从功能模拟强化学习肌肉骨骼控制

为了诊断、计划和治疗肌肉骨骼病变，了解和再现复杂运动的肌肉募集是必不可少的。由于运动的肌肉激活通常是高度冗余的、非线性的和时间依赖性的，机器学习可以为其建模和控制特定解剖结构的肌肉骨骼模拟提供解决方案。复杂的生物力学模拟通常需要专门的计算环境，其数值复杂且速度缓慢，阻碍了它们与典型深度学习框架的集成。在这项工作中，训练基于深度强化学习 (DRL) 的逆动力学控制器来控制人体肩部生物力学模型的肌肉激活。以可推广的端到端方式，在给定当前和所需位置-速度对的情况下学习肌肉激活。引入了用于轨迹控制的定制奖励函数，可以直接扩展到额外的肌肉和更高的自由度。使用生物力学模型，使用正在训练的 DRL 的进化神经模型在集群上同时模拟多个事件。给出了肩外展的单轴运动控制的结果，用于跟随随机生成的角轨迹的任务。