The majority of efforts in the field of sim-to-real Deep Reinforcement Learning focus on robot manipulators, which is justified by their importance for modern production plants. However, there are only a few studies for a more extensive use in manufacturing processes. In this paper, we contribute to this by automating a complex manufacturing-like process using simulation-based Deep Reinforcement Learning. The setup and workflow presented here are designed to mimic the characteristics of real manufacturing processes and proves that Deep Reinforcement Learning can be applied to physical systems built from industrial drive and control components by transferring policies learned in simulation to the real machine. Aided by domain randomization, training in a virtual environment is crucial due to the benefit of accelerated training speed and the desire for safe Reinforcement Learning. Our key contribution is to demonstrate the applicability of simulation-based Deep Reinforcement Learning in industrial automation technology. We introduce an industrial drive and control system, based on the classic pub game Foosball, from both an engineering and a simulation perspective, describing the strategies applied to increase transfer robustness. Our approach allowed us to train a self-learning agent to independently learn successful control policies for demanding Foosball tasks based on sparse reward signals. The promising results prove that state-of-the-art Deep Reinforcement Learning algorithms are able to produce models trained in simulation, which can successfully control industrial use cases without using the actual system for training beforehand.

在模拟到真实的深度强化学习领域中，大多数工作都集中在机器人操纵器上，这对于它们在现代生产工厂中的重要性是有道理的。但是，只有很少的研究在制造过程中有更广泛的用途。在本文中，我们通过使用基于仿真的“深度强化学习”来自动化类似于制造的复杂过程，从而对此做出了贡献。这里介绍的设置和工作流程旨在模仿实际制造过程的特征，并证明通过将在模拟中学习的策略转移到实际机器上，深度强化学习可以应用于由工业驱动和控制组件构建的物理系统。借助域随机化，在虚拟环境中进行培训至关重要，这归因于提高的培训速度和对安全强化学习的渴望。我们的主要贡献是证明基于仿真的深度强化学习在工业自动化技术中的适用性。我们从工程学和仿真的角度介绍了基于经典酒吧游戏Foosball的工业驱动和控制系统，描述了用于提高传输鲁棒性的策略。我们的方法使我们能够训练自学代理，以基于稀疏的奖励信号独立学习成功的控制策略，以完成要求苛刻的Foosball任务。令人鼓舞的结果证明，最新的深度强化学习算法能够生成经过模拟训练的模型，