DoS and DDoS attacks have been growing in size and number over the last decade and existing solutions to mitigate these attacks are largely inefficient. Compared to other types of malicious cyber attacks, DoS and DDoS attacks are particularly challenging to combat. Because of their ability to mask themselves as legitimate traffic, it has proven difficult to develop methods to detect these types of attacks on a packet or flow level. In this paper, we explore the potential of Variational Autoencoders to serve as a component within an intelligent security solution that differentiates between normal and malicious traffic. The motivation behind resorting to Variational Autoencoders is that unlike normal encoders that would code an input flow as a single point, they encode a flow as a distribution over the latent space which avoids overfitting. Intuitively, this allows a Variational Autoencoder to not only learn latent representations of seen input features, but to generalize in a way that allows for an interpretation of unseen flows and flow features with slight variations.

Two methods based on the ability of Variational Autoencoders to learn latent representations from network traffic flows of both benign and malicious traffic, are proposed. The first method resorts to a classifier based on the latent encodings obtained from Variational Autoencoders learned from traffic traces. The second method is an anomaly detection method, where the Variational Autoencoder is used to learn the abstract feature representations of exclusively legitimate traffic. Anomalies are then filtered out by relying on the reconstruction loss of the Variational Autoencoder. In this sense, the construction loss of the autoencoder is fed as input to a classifier that outputs the class of the traffic including benign and malign, and eventually the attack type. Thus, the second approach operates with two separate training processes on two separate data sources: the first training involving only legitimate traffic, and the second training involving all traffic classes. This is different from the first approach which operates only a single training process on the whole traffic dataset. Thus, the autoencoder of the first approach aspires to learn a general feature representation of the flows while the autoencoder of the second approach aims to exclusively learn a representation of the benign traffic. The second approach is thus more susceptible to finding zero day attacks and discovering new attacks as anomalies.

Both of the proposed methods have been thoroughly tested on two separate datasets with a similar feature space. The results show that both methods are promising, with the classifier-based method being slightly superior to the anomaly-based one.

在过去十年中，DoS 和 DDoS 攻击的规模和数量一直在增长，缓解这些攻击的现有解决方案在很大程度上效率低下。与其他类型的恶意网络攻击相比，DoS 和 DDoS 攻击尤其具有挑战性。由于它们能够将自己伪装成合法流量，因此已经证明很难开发方法来检测数据包或流级别上的这些类型的攻击。在本文中，我们探讨了变分自编码器作为智能安全解决方案中的一个组件的潜力，可区分正常和恶意流量。诉诸变分自动编码器背后的动机是，与将输入流编码为单个点的普通编码器不同，它们将流编码为潜在空间上的分布，以避免过度拟合。

提出了两种基于变分自动编码器从良性和恶意流量的网络流量中学习潜在表示的能力的方法。第一种方法采用基于从流量跟踪中学习的变分自动编码器获得的潜在编码的分类器。第二种方法是异常检测方法，其中使用变分自编码器来学习完全合法流量的抽象特征表示。然后依靠变分自动编码器的重建损失过滤掉异常。从这个意义上说，自动编码器的构造损失作为输入提供给分类器，该分类器输出流量的类别，包括良性和恶意，最终是攻击类型。因此，第二种方法在两个单独的数据源上使用两个单独的训练过程：第一个训练只涉及合法流量，第二个训练涉及所有流量类别。这与第一种方法不同，第一种方法仅在整个交通数据集上运行单个训练过程。因此，第一种方法的自动编码器渴望学习流的一般特征表示，而第二种方法的自动编码器旨在专门学习良性流量的表示。因此，第二种方法更容易发现零日攻击并将新攻击发现为异常。这与第一种方法不同，第一种方法仅在整个交通数据集上运行单个训练过程。因此，第一种方法的自动编码器渴望学习流的一般特征表示，而第二种方法的自动编码器旨在专门学习良性流量的表示。因此，第二种方法更容易发现零日攻击并将新攻击发现为异常。这与第一种方法不同，第一种方法仅在整个交通数据集上运行单个训练过程。因此，第一种方法的自动编码器渴望学习流的一般特征表示，而第二种方法的自动编码器旨在专门学习良性流量的表示。因此，第二种方法更容易发现零日攻击并将新攻击发现为异常。

所提出的两种方法都已在具有相似特征空间的两个独立数据集上进行了彻底测试。结果表明，这两种方法都有前景，基于分类器的方法略优于基于异常的方法。