A reflex is a simple closed-loop control approach that tries to minimize an error but fails to do so because it will always react too late. An adaptive algorithm can use this error to learn a forward model with the help of predictive cues. For example, a driver learns to improve steering by looking ahead to avoid steering in the last minute. In order to process complex cues such as the road ahead, deep learning is a natural choice. However, this is usually achieved only indirectly by employing deep reinforcement learning having a discrete state space. Here, we show how this can be directly achieved by embedding deep learning into a closed-loop system and preserving its continuous processing. We show in z-space specifically how error backpropagation can be achieved and in general how gradient-based approaches can be analyzed in such closed-loop scenarios. The performance of this learning paradigm is demonstrated using a line follower in simulation and on a real robot that shows very fast and continuous learning.

中文翻译：

---

闭环深度学习：使用反向传播生成前向模型

反射是一种简单的闭环控制方法，它试图将错误最小化，但未能这样做，因为它总是反应太晚。自适应算法可以在预测线索的帮助下使用此错误来学习前向模型。例如，驾驶员通过向前看以避免在最后一分钟转向来学习改进转向。为了处理诸如前方道路之类的复杂线索，深度学习是一个自然的选择。然而，这通常只能通过采用具有离散状态空间的深度强化学习来间接实现。在这里，我们展示了如何通过将深度学习嵌入闭环系统并保持其连续处理来直接实现这一点。我们在 z 空间中具体展示了如何实现误差反向传播，以及一般如何在此类闭环场景中分析基于梯度的方法。这种学习范式的性能在模拟中使用线跟随器和在显示出非常快速和持续学习的真实机器人上进行了演示。