Many network operations, ranging from attack investigation and mitigation to traffic management, require answering network-wide flow queries in seconds. Although flow records are collected at each router, using available traffic capture utilities, querying the resulting datasets from hundreds of routers across sites and over time, remains a significant challenge due to the sheer traffic volume and distributed nature of flow records. In this article, we investigate how to improve the response time for a priori unknown network-wide queries. We present Flowyager, a system that is built on top of existing traffic capture utilities. Flowyager generates and analyzes tree data structures, that we call Flowtrees, which are succinct summaries of the raw flow data available by capture utilities. Flowtrees are self-adjusted data structures that drastically reduce space and transfer requirements, by 75% to 95%, compared to raw flow records. Flowyager manages the storage and transfers of Flowtrees, supports Flowtree operators, and provides a structured query language for answering flow queries across sites and time periods. By deploying a Flowyager prototype at both a large Internet Exchange Point and a Tier-1 Internet Service Provider, we showcase its capabilities for networks with hundreds of router interfaces. Our results show that the query response time can be reduced by an order of magnitude when compared with alternative data analytics platforms. Thus, Flowyager enables interactive network-wide queries and offers unprecedented drill-down capabilities to, e.g., identify DDoS culprits, pinpoint the involved sites, and determine the length of the attack.

中文翻译：

---

使用 Flowyager 探索全网流数据

许多网络操作，从攻击调查和缓解到流量管理，都需要在几秒钟内回答网络范围的流查询。尽管流记录是在每个路由器上收集的，但由于流记录的庞大流量和分布式特性，使用可用的流量捕获实用程序、跨站点和随着时间的推移查询来自数百个路由器的结果数据集仍然是一个重大挑战。在本文中，我们研究如何改进先验未知网络范围查询的响应时间。我们展示了 Flowyager，这是一个建立在现有流量捕获实用程序之上的系统。Flowyager 生成并分析树数据结构，我们称之为 Flowtrees，它是捕获实用程序可用的原始流数据的简洁摘要。Flowtrees 是自我调整的数据结构，与原始流记录相比，可显着减少空间和传输需求，减少 75% 到 95%。Flowyager 管理 Flowtree 的存储和传输，支持 Flowtree 算子，并提供结构化查询语言来回答跨站点和时间段的 Flow 查询。通过在大型互联网交换点和一级互联网服务提供商部署 Flowyager 原型，我们展示了其在具有数百个路由器接口的网络中的功能。我们的结果表明，与其他数据分析平台相比，查询响应时间可以减少一个数量级。因此，Flowyager 支持交互式全网查询并提供前所未有的深入分析功能，例如识别 DDoS 罪魁祸首、查明相关站点、