The dynamic scheduling problem of semiconductor manufacturing systems (SMSs) is becoming more complicated and challenging due to internal uncertainties and external demand changes. To this end, this paper addresses integrated release control and production scheduling problems with uncertain processing times and urgent orders and proposes a convolutional neural network and asynchronous advanced actor critic-based method (CNN-A3C) that involves a training phase and a deployment phase. In the training phase, actor–critic networks are trained to predict the evaluation of scheduling decisions and to output the optimal scheduling decision. In the deployment phase, the most appropriate release control and scheduling decisions are periodically generated according to the current production status based on the networks. Furthermore, we improve the four key points in the deep reinforcement learning (DRL) algorithm, state space, action space, reward function, and network structure and design four mechanisms: a slide-window-based two-dimensional state perception mechanism, an adaptive reward function that considers multiple objectives and automatically adjusts to dynamic events, a continuous action space based on composite dispatching rules (CDR) and release strategies, and actor–critic networks based on convolutional neural networks (CNNs). To verify the feasibility and effectiveness of the proposed dynamic scheduling method, it is implemented on a simplified SMS. The simulation experimental results show that the proposed method outperforms the unimproved A3C-based method and the common dispatching rules under the new uncertain scenarios.

由于内部不确定性和外部需求变化，半导体制造系统（SMS）的动态调度问题变得越来越复杂和具有挑战性。为此，本文解决了具有不确定处理时间和紧急订单的集成发布控制和生产调度问题，并提出了一种涉及训练阶段和部署阶段的卷积神经网络和异步高级actorcritic-based方法（CNN-A3C）。在训练阶段，actor-critic 网络被训练来预测调度决策的评估并输出最优调度决策。在部署阶段，根据网络的当前生产状态，周期性地生成最合适的发布控制和调度决策。此外，我们改进了深度强化学习（DRL）算法的四个关键点，即状态空间、动作空间、奖励函数和网络结构，并设计了四种机制：基于滑动窗口的二维状态感知机制、自适应奖励函数它考虑了多个目标并自动适应动态事件、基于复合调度规则 (CDR) 和发布策略的连续动作空间，以及基于卷积神经网络 (CNN) 的演员-评论家网络。为了验证所提出的动态调度方法的可行性和有效性，它在一个简化的 SMS 上实现。仿真实验结果表明，在新的不确定场景下，该方法优于未改进的基于 A3C 的方法和常见的调度规则。