Future smart grids can and will be subject of systematic attacks that can result in monetary costs and reduced system stability. These attacks are not necessarily malicious, but can be economically motivated as well. Emerging flexibility markets are of interest here, because they can incite attacks if market design is flawed. The dimension and danger potential of such strategies is still unknown. Automatic analysis tools are required to systematically search for unknown strategies and their respective countermeasures. We propose deep reinforcement learning to learn attack strategies autonomously to identify underlying systemic vulnerabilities this way. As a proof-of-concept, we apply our approach to a reactive power market setting in a distribution grid. In the case study, the attacker learned to exploit the reactive power market by using controllable loads. That was done by systematically inducing constraint violations into the system and then providing countermeasures on the flexibility market to generate profit, thus finding a hitherto unknown attack strategy. As a weak-point, we identified the optimal power flow that was used for market clearing. Our general approach is applicable to detect unknown attack vectors, to analyze a specific power system regarding vulnerabilities, and to systematically evaluate potential countermeasures.

中文翻译：

---

在电力经济系统中进行脆弱性分析的强化学习

未来的智能电网可能并且将会受到系统性攻击，这可能导致金钱成本和系统稳定性降低。这些攻击不一定是恶意的，但也可能出于经济动机。新兴的灵活性市场在这里很受关注，因为如果市场设计有缺陷，它们会煽动攻击。此类策略的规模和潜在危险仍然未知。需要自动分析工具来系统地搜索未知策略及其各自的对策。我们提出深度强化学习来自主学习攻击策略，从而以这种方式识别潜在的系统漏洞。作为概念验证，我们将我们的方法应用于配电网中的无功功率市场设置。在案例研究中，攻击者学会了通过使用可控负载来利用无功功率市场。这是通过系统地将约束违​​规引入系统，然后在灵活性市场上提供对策以产生利润来完成的，从而找到了前所未有的攻击策略。作为弱点，我们确定了用于市场出清的最佳潮流。我们的一般方法适用于检测未知的攻击向量，分析特定电力系统的漏洞，并系统地评估潜在的对策。我们确定了用于市场出清的最佳潮流。我们的一般方法适用于检测未知的攻击向量，分析特定电力系统的漏洞，并系统地评估潜在的对策。我们确定了用于市场出清的最佳潮流。我们的一般方法适用于检测未知的攻击向量，分析特定电力系统的漏洞，并系统地评估潜在的对策。