The Industrial Internet of Things (IIoT), also known as Industry 4.0, empowers manufacturing and production processes by leveraging automation and Internet of Things (IoT) technologies. In IIoT, the information communication technologies enabled by IoT could greatly improve the efficiency and timeliness of information exchanges between both vertical and horizontal system integrations. Likewise, machine learning algorithms, particularly Deep Reinforcement Learning (DRL), are viable for assisting in automated control of complex IIoT systems, with the support of distributed edge computing infrastructure. Despite noticeable performance improvements, the security threats brought by massive interconnections in IoT and the vulnerabilities of deep neural networks used in DRL must be thoroughly investigated and mitigated before widespread deployment. Thus, in this paper we first design a DRL-based controller that could be deployed at edge computing server to enable automated control in an IIoT context. We then investigate malicious behaviors of adversaries with two attacks: (i) function-based attacks that can be launched during training phase and (ii) performance-based attacks that can be launched after training phase, to study the security impacts of vulnerable DRL-based controllers. From the adversary’s perspective, maximum entropy Inverse Reinforcement Learning (IRL) is used to approximate a reward function through observation of system trajectories under the control of trained DRL-based controllers. The approximated reward function is then used to launch attacks by the adversary against the Deep Q Network (DQN)-based controller. Via simulation, we evaluate the impacts of our two investigated attacks, finding that attacks are increasingly successful with increasing accuracy of the control model. Furthermore, we discuss some tradeoffs between control performance and security performance of DRL-based IIoT controllers, and outline several future research directions to secure machine learning use in IIoT systems.

工业物联网（IIoT），也称为工业4.0，通过利用自动化和物联网（IoT）技术来支持制造和生产过程。在工业物联网中，物联网支持的信息通信技术可以大大提高纵向和横向系统集成之间信息交换的效率和及时性。同样，在分布式边缘计算基础架构的支持下，机器学习算法（尤其是深度强化学习（DRL））可用于协助自动控制复杂的IIoT系统。尽管性能有了显着改善，但在广泛部署之前，必须彻底研究和缓解IoT大规模互连带来的安全威胁以及DRL中使用的深度神经网络的漏洞。从而，在本文中，我们首先设计了一种基于DRL的控制器，该控制器可部署在边缘计算服务器上，以实现IIoT环境中的自动控制。然后，我们通过两种攻击调查对手的恶意行为：（i）可以在培训阶段启动的基于功能的攻击，以及（ii）可以在培训阶段之后启动的基于性能的攻击，以研究易受攻击的DRL-基于控制器。从对手的角度来看，最大熵逆强化学习（IRL）用于在受过训练的基于DRL的控制器的控制下，通过观察系统轨迹来近似奖励函数。然后，近似的奖励功能被攻击者针对基于Deep Q Network（DQN）的控制器发起攻击。通过模拟，我们评估了两次调查的攻击的影响，发现随着控制模型准确性的提高，攻击越来越成功。此外，我们讨论了基于DRL的IIoT控制器的控制性能和安全性能之间的权衡，并概述了一些未来的研究方向，以确保IIoT系统中机器学习的安全。