The cybertwin and 6G-enabled Industrial Internet of Things (6G-IIoT) are the critical technologies that create the digital counterparts for physical systems and enable the near-instant interconnectivity in the industrial domain. It is in demand but challenging to conduct the integrated design for 6G-IIoT, which intertwines the cyber subsystems, such as control, communication, computing (3C), and the physical industrial factories and plants. Therefore, the cybertwin, which synchronizes between the digital counterparts and its physical entities during the system runtime, is the ideal proving ground for conducting the integrated design on the highly intertwined 3C of 6G-IIoT. However, the cybertwin lacks artificial intelligence to capacitate the automated integrated design for the 6G-IIoT. In this article, we first demonstrate the architecture of the machine-learning-based cybertwin for 6G-IIoT. Then, we leverage deep reinforcement learning (DRL) to conduct the integrated design via systematic trial and error in the cybertwin model, which is otherwise costly and dangerous in real industrial systems. Moreover, we invent the adaptive observation window for deep QQ -network (AOW-DQN), which generates system states adaptive to the control system’s physical dynamics. Finally, the experimental results demonstrate the effectiveness and efficiency of our approach. To the best of our knowledge, we are the first to present the machine-learning-based cybertwin for carrying out the integrated design on the 3C for 6G-IIoT.

中文翻译：

---

基于深度强化学习的 6G IIoT Cyber​​twin 架构：控制、通信和计算的集成设计


cybertwin 和支持 6G 的工业物联网 (6G-IIoT) 是为物理系统创建数字对应物并实现工业领域近乎即时互连的关键技术。 6G-IIoT 的集成设计将控制、通信、计算 (3C) 等网络子系统与物理工业工厂和工厂交织在一起，因此需求旺盛，但也具有挑战性。因此，在系统运行时在数字对应物与其物理实体之间进行同步的cybertwin是在6G-IIoT高度交织的3C上进行集成设计的理想试验场。然而，Cyber​​twin 缺乏人工智能来支持 6G-IIoT 的自动化集成设计。在本文中，我们首先演示了 6G-IIoT 的基于机器学习的网络孪生的架构。然后，我们利用深度强化学习（DRL）通过 Cyber​​twin 模型中的系统试错来进行集成设计，否则这在实际工业系统中成本高昂且危险。此外，我们发明了深度 QQ 网络的自适应观察窗口（AOW-DQN），它生成适应控制系统物理动力学的系统状态。最后，实验结果证明了我们方法的有效性和效率。据我们所知，我们是第一个推出基于机器学习的cybertwin，用于在6G-IIoT的3C上进行集成设计。