The chemical mechanical polishing (CMP) process usually suffers from drift and shift in the Run-to-Run material removal process due to the wear and replacement of the polishing pad, lacking of in-suit measurements of the product quality of interest and other environment variations. This paper proposed a deep reinforcement learning (DRL)-based run-to-run (R2R) controller for the CMP process. Firstly, deep reinforcement learning is effectively utilized as a training algorithm of the R2R controller, which is a model-free controller to take a decision with infinite horizon information and thus improves the control performance; Secondly, a novel policy network is embeded to the DRL model, which divides the network into linear and nonlinear part explicitly to improve the prediction performance of the R2R controller on process changes. Finally, a special reward function is proposed to improve the training of the R2R controller, which trades off between target tracing and fluctuations of production parameters. The effectiveness of the proposed controller is validated on a CMP process. The testing results illustrate that the DRL-based R2R controller can precisely trace the desired target of material removal rate (MRR) and is very effective to control various process variations online.

中文翻译：

---

基于深度强化学习的化学机械抛光过程逐次控制

由于抛光垫的磨损和更换，化学机械抛光 (CMP) 过程通常会在连续材料去除过程中出现漂移和偏移，缺乏对感兴趣的产品质量和其他环境的适合测量变化。本文提出了一种用于 CMP 过程的基于深度强化学习 (DRL) 的运行到运行 (R2R) 控制器。首先，有效利用深度强化学习作为 R2R 控制器的训练算法，R2R 控制器是一种无模型控制器，可以在无限范围信息下进行决策，从而提高控制性能；其次，将一种新颖的策略网络嵌入到 DRL 模型中，将网络明确地划分为线性和非线性部分，以提高 R2R 控制器对过程变化的预测性能。最后，提出了一个特殊的奖励函数来改进 R2R 控制器的训练，它在目标跟踪和生产参数的波动之间进行权衡。提议的控制器的有效性在 CMP 过程中得到验证。测试结果表明，基于 DRL 的 R2R 控制器可以精确跟踪所需的材料去除率 (MRR) 目标，并且非常有效地在线控制各种工艺变化。