Torque control of electric drives is a challenging task, as high dynamics need to be achieved despite different input and state constraints while also pursuing secondary objectives, e.g., maximizing power efficiency. Whereas most state-of-the-art methods generally necessitate thorough knowledge about the system model, a model-free deep reinforcement learning torque controller is proposed. In particular, the deep Q-learning algorithm is utilized which has been successfully used in different application scenarios with a finite action set in the recent past. This nicely fits the considered system, a permanent magnet synchronous motor supplied by a two-level voltage source inverter, since the latter is a power supply unit with a limited amount of distinct switching states. This contribution investigates the deep Q-learning finite control set framework and its design, including the conception of a reward function that incorporates the demands concerning torque tracking, efficiency maximization and compliance with operation limits. In addition, a comprehensive hyperparameter optimization is presented, which addresses the many degrees of freedom of the deep Q-learning algorithm striving for an optimal controller configuration. Advantages and remaining challenges of the proposed algorithm are disclosed through an extensive validation, which includes a direct comparison with a state-of-the-art model predictive direct torque controller.

中文翻译：

---

用于永磁同步电机的深度 Q 学习直接扭矩控制器


电力驱动器的扭矩控制是一项具有挑战性的任务，因为尽管输入和状态约束不同，但仍需要实现高动态性，同时还要追求次要目标，例如最大化功率效率。鉴于大多数最先进的方法通常需要对系统模型有透彻的了解，因此提出了一种无模型的深度强化学习扭矩控制器。特别是，采用了深度Q学习算法，该算法近年来已成功应用于具有有限动作集的不同应用场景。这非常适合所考虑的系统，即由两级电压源逆变器供电的永磁同步电机，因为后者是具有有限数量的不同开关状态的电源单元。本论文研究了深度 Q 学习有限控制集框架及其设计，包括奖励函数的概念，该函数结合了扭矩跟踪、效率最大化和遵守操作限制等要求。此外，还提出了全面的超参数优化，解决了深度 Q 学习算法争取最佳控制器配置的多个自由度问题。通过广泛的验证揭示了所提出算法的优点和剩余挑战，其中包括与最先进的模型预测直接扭矩控制器的直接比较。