Given the always increasing size of computer systems, manually protecting them in case of attacks is unfeasible and error-prone. For this reason, until now, several model-based Intrusion Response Systems (IRSs) have been proposed with the purpose of limiting the amount of work of the system administrators. However, since the most advanced IRSs adopt a stateful approach, they are subject to what Richard Bellman defined as the curse of dimensionality. Furthermore, modern computer systems are non-stationary, that is, they are subject to frequent changes in their configuration and in their software base, which in turn could make a model-based approach ineffective due to deviations in system behavior with respect to the model. In this paper we propose, to the best of our knowledge, the first approach based on deep reinforcement learning for the implementation of a hybrid model-free IRS. Experimental results show that the proposed IRS is able to deal with non-stationary systems, while reducing the time needed for the computation of the defense policies by orders of magnitude with respect to model-based approaches, and being still able to provide near-optimal rewards.

鉴于计算机系统的规模总是在不断扩大，因此在受到攻击时手动保护它们是不可行的，而且容易出错。因此，迄今为止，已经提出了几种基于模型的入侵响应系统（IRS），目的是限制系统管理员的工作量。但是，由于最先进的IRS采用有状态方法，因此受制于Richard Bellman定义的维数诅咒。此外，现代计算机系统是非固定的，也就是说，它们的配置和软件库经常更改，由于系统行为相对于模型的偏差，使得基于模型的方法无效。 。在本文中，我们就我们所知，提出了一种基于深度强化学习的第一种方法，用于实现无模型IRS的混合。实验结果表明，提出的IRS能够处理非平稳系统，同时相对于基于模型的方法而言，将防御策略的计算所需的时间减少了几个数量级，并且仍然能够提供近乎最佳的状态奖励。