Detecting the flows with abnormally large spreads over big network data can help us identify network attacks, such as DDoS attacks and scanners. Most per-flow measurement studies use compact data structures to reduce their memory requirements, fitting in the limited on-chip memory and catching up with the line rate. In this paper, we study a novel problem called spread estimation among multi-periods to measure the total number of distinct elements or the number of distinct k-persistent elements in a flow among multiple traffic measurement periods. In our design, we use an on-chip/off-chip model to record the per-flow traffic information, which uses small on-chip memory and matches the line rate, i.e., we use on-chip memory to filter out the duplicates, sample the elements, and store the sampled traffic data in off-chip memory. By performing the set operations on the sampled traffic data, we can derive the total number of distinct elements and the number of distinct k-persistent elements among multiple periods based on probability analysis. The experimental results on real Internet traffic traces show that, when performing spread estimation among multiple periods, our estimator is efficient in memory usage and estimation accuracy and can efficiently detect the stealthy DDoS attack and scanners.

检测到大网络数据上分布异常大的流量可以帮助我们识别网络攻击，例如DDoS攻击和扫描程序。大多数每流测量研究都使用紧凑的数据结构来减少其内存需求，以适应有限的片上内存并赶上线速。在本文中，我们研究了一个新的问题，称为多周期之间的扩散估计，以测量多个流量测量周期之间的流中不同元素的总数或不同k持久元素的数量。在我们的设计中，我们使用片上/片外模型来记录每流流量信息，该模型使用小的片内存储器并匹配线路速率，即，我们使用片上存储器来过滤出重复项，对元素进行采样，并将采样的流量数据存储在片外存储器中。通过对采样的交通数据执行设置操作，我们可以基于概率分析得出多个时段中不同元素的总数和不同k持久元素的数量。在真实Internet流量跟踪上的实验结果表明，当在多个时间段之间进行传播估计时，我们的估计器在内存使用和估计准确性方面都很有效，并且可以有效地检测到隐身的DDoS攻击和扫描程序。