Despite highly accurate intrusion detection schemes based on machine learning (ML) reported in the literature, changes in network traffic behavior quickly yield low accuracy rates. An intrusion detection model update is not easily feasible due to the enormous amount of network traffic to be processed in near real-time for high-speed networks, in particular, under big data settings. In this paper, we propose a new scalable long-lasting intrusion detection architecture for the processing of network content and the building of a reliable ML-based intrusion detection model. Experiments performed through the analysis of five years of network traffic, about 20 TB of data, have shown that our approach extends the lifespan of our model by up to six weeks. That occurs because the average accuracy rate of our proposal lasted eight weeks after the training phase, and traditional ones reached only two weeks after the model building. Additionally, our proposal achieves up to 10 Gbps of detection throughput in a 20-core big data processing cluster.

中文翻译：

---

大数据时代的机器学习入侵检测：更长模型寿命的多目标方法

尽管文献中报道了基于机器学习（ML）的高度精确的入侵检测方案，但网络流量行为的变化很快会导致较低的准确率。由于要为高速网络（尤其是在大数据设置下）近乎实时地处理大量网络流量，入侵检测模型更新不容易实现。在本文中，我们提出了一种新的可扩展的持久入侵检测体系结构，用于处理网络内容和建立可靠的基于ML的入侵检测模型。通过分析五年的网络流量和大约20 TB的数据进行的实验表明，我们的方法将模型的寿命延长了六周。发生这种情况是因为我们的建议的平均准确率在训练阶段后持续了八周，而传统的准确率仅在模型构建后的两周才达到。此外，我们的建议在20核大数据处理集群中实现了高达10 Gbps的检测吞吐量。