To address the uncertainty of production environment in assembly job shop, in combination of the real-time feature of reinforcement learning, a dual Q-learning (D-Q) method is proposed to enhance the adaptability to environmental changes by self-learning for assembly job shop scheduling problem. On the basis of the objective function of minimising the total weighted earliness penalty and completion time cost, the top level Q-learning is focused on localised targets in order to find the dispatching policy which can minimise machine idleness and balance machine loads, and the bottom level Q-learning is focused on global targets in order to learn the optimal scheduling policy which can minimise the overall earliness of all jobs. Some theoretical results and simulation experiments indicate that the proposed algorithm achieves generally better results than the single Q-learning (S-Q) and other scheduling rules, under the arrival frequency of product with different conditions, and show good adaptive performance.

Abbreviations: AFSSP, assembly flow shop scheduling problem; AJSSP, assembly job shop scheduling problem; RL, reinforcement learning; TASP, two-stage assembly scheduling problem

针对装配车间生产环境的不确定性，结合强化学习的实时性，提出了一种双Q学习（DQ）方法，通过自学习提高装配车间对环境变化的适应性调度问题. 在最小化总加权提前惩罚和完成时间成本的目标函数的基础上，顶层 Q-learning 专注于本地化目标，以找到可以最小化机器空闲和平衡机器负载的调度策略，以及底层 Q-learning level Q-learning 专注于全局目标，以学习可以最小化所有作业的整体提前期的最佳调度策略。一些理论结果和仿真实验表明，该算法在不同条件下产品到达频率下，总体上比单一Q-learning（SQ）等调度规则取得了更好的效果，并表现出良好的自适应性能。

缩写： AFSSP，装配流水车间调度问题；AJSSP，装配作业车间调度问题；RL，强化学习；TASP，两阶段装配调度问题