Most of the existing log anomaly detection methods suffer from scalability and numerous false positives. Besides, they cannot rank the severity level of abnormal events. This paper proposes a log anomaly detection based on Q-learning, namely QLLog, which can detect multiple types of system anomalies and rank the severity level of abnormal events. We first build a mathematical model of log anomaly detection, proving that log anomaly detection is a sequential decision problem. Second, we use the Q-learning algorithm to build the core of the anomaly detection model. This allows QLLog to automatically learn directed acyclic graph log patterns from normal execution and adjust the training model according to the reward value. Then, QLLog combines the advantages of the Q-learning algorithm and the specially designed rules to detect anomalies when log patterns deviate from the model trained from log data under normal execution. Besides, we provide a feedback mechanism and build an abnormal level table. Therefore, QLLog can adapt to new log states and log patterns. Experiments on real datasets show that the method can quickly and effectively detect system anomalies. Compared with the state of the art, QLLog can detect numerous real problems with high accuracy 95%, and its scalability outperforms other existing log-based anomaly detection methods.

现有的大多数对数异常检测方法都存在可伸缩性和大量误报的问题。此外，他们无法对异常事件的严重程度进行排名。本文提出了一种基于Q学习的日志异常检测方法，即QLLog，它可以检测多种类型的系统异常，并对异常事件的严重程度进行排序。我们首先建立了对数异常检测的数学模型，证明了对数异常检测是一个顺序决策问题。其次，我们使用Q学习算法来构建异常检测模型的核心。这允许QLLog从正常执行中自动学习有向无环图日志模式，并根据奖励值调整训练模型。然后，QLLog结合了Q学习算法的优势和专门设计的规则，可在正常执行情况下在日志模式偏离从日志数据训练的模型时检测异常。此外，我们提供了一种反馈机制并建立了一个异常级别表。因此，QLLog可以适应新的日志状态和日志模式。在真实数据集上的实验表明，该方法可以快速有效地检测系统异常。与现有技术相比，QLLog可以以95％的高精度检测大量实际问题，并且其可扩展性优于其他现有的基于日志的异常检测方法。在真实数据集上的实验表明，该方法可以快速有效地检测系统异常。与现有技术相比，QLLog可以以95％的高精度检测大量实际问题，并且其可扩展性优于其他现有的基于日志的异常检测方法。在真实数据集上的实验表明，该方法可以快速有效地检测系统异常。与现有技术相比，QLLog可以以95％的高精度检测大量实际问题，并且其可扩展性优于其他现有的基于日志的异常检测方法。