We present a differentiable soft-body physics simulator that can be composed with neural networks as a differentiable layer. In contrast to other differentiable physics approaches that use explicit forward models to define state transitions, we focus on implicit state transitions defined via function minimization. Implicit state transitions appear in implicit numerical integration methods, which offer the benefits of large time steps and excellent numerical stability, but require a special treatment to achieve differentiability due to the absence of an explicit differentiable forward pass. In contrast to other implicit differentiation approaches that require explicit formulas for the force function and the force Jacobian matrix, we present an energy-based approach that allows us to compute these derivatives automatically and in a matrix-free fashion via reverse-mode automatic differentiation. This allows for more flexibility and productivity when defining physical models and is particularly important in the context of neural network training, which often relies on reverse-mode automatic differentiation (backpropagation). We demonstrate the effectiveness of our differentiable simulator in policy optimization for locomotion tasks and show that it achieves better sample efficiency than model-free reinforcement learning.

中文翻译：

---

可微隐式人体物理

我们提出了一种可微分的物理物理学模拟器，该模拟器可以与神经网络组成可微分的层。与使用显式正向模型定义状态转移的其他微分物理学方法相反，我们集中于通过函数最小化定义的隐式状态转移。隐式状态转换出现在隐式数值积分方法中，该方法具有较大的时间步长和出色的数值稳定性，但由于没有显式的可微弱的前向通过，因此需要进行特殊处理以实现可微性。与其他隐式微分方法不同，这些隐式微分方法需要针对力函数和力雅可比矩阵的显式公式，我们提出了一种基于能量的方法，该方法使我们能够通过反向模式自动微分以无矩阵的方式自动计算这些导数。这在定义物理模型时提供了更大的灵活性和生产率，并且在神经网络训练的背景下尤其重要，而神经网络训练通常依赖于反向模式自动微分（反向传播）。我们证明了可区分模拟器在运动任务策略优化中的有效性，并表明与无模型的强化学习相比，它可以实现更好的样本效率。通常依赖于反向模式自动微分（反向传播）。我们证明了可区分模拟器在运动任务策略优化中的有效性，并表明与无模型的强化学习相比，它可以实现更好的样本效率。通常依赖于反向模式自动微分（反向传播）。我们证明了可区分模拟器在针对运动任务进行策略优化中的有效性，并表明与无模型强化学习相比，它可实现更好的样本效率。