当前位置: X-MOL 学术arXiv.cs.PF › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Poster: Benchmarking Financial Data Feed Systems
arXiv - CS - Performance Pub Date : 2020-10-29 , DOI: arxiv-2010.15534
Manuel Coenen, Christoph Wagner, Alexander Echler, Sebastian Frischbier

Data-driven solutions for the investment industry require event-based backend systems to process high-volume financial data feeds with low latency, high throughput, and guaranteed delivery modes. At vwd we process an average of 18 billion incoming event notifications from 500+ data sources for 30 million symbols per day and peak rates of 1+ million notifications per second using custom-built platforms that keep audit logs of every event. We currently assess modern open source event-processing platforms such as Kafka, NATS, Redis, Flink or Storm for the use in our ticker plant to reduce the maintenance effort for cross-cutting concerns and leverage hybrid deployment models. For comparability and repeatability we benchmark candidates with a standardized workload we derived from our real data feeds. We have enhanced an existing light-weight open source benchmarking tool in its processing, logging, and reporting capabilities to cope with our workloads. The resulting tool wrench can simulate workloads or replay snapshots in volume and dynamics like those we process in our ticker plant. We provide the tool as open source. As part of ongoing work we contribute details on (a) our workload and requirements for benchmarking candidate platforms for financial feed processing; (b) the current state of the tool wrench.

中文翻译:

海报:基准金融数据馈送系统

投资行业的数据驱动解决方案需要基于事件的后端系统来处理具有低延迟、高吞吐量和有保证的交付模式的大容量金融数据馈送。在 vwd,我们每天平均处理来自 500 多个数据源的 180 亿个传入事件通知,处理 3000 万个符号,峰值速率为每秒 1+ 百万个通知,使用定制平台保留每个事件的审计日志。我们目前正在评估现代开源事件处理平台,如 Kafka、NATS、Redis、Flink 或 Storm,以在我们的股票工厂中使用,以减少跨领域关注点的维护工作并利用混合部署模型。为了可比性和可重复性,我们使用从真实数据源中得出的标准化工作负载对候选人进行基准测试。我们增强了现有轻量级开源基准测试工具的处理、日志记录和报告功能,以应对我们的工作负载。由此产生的工具扳手可以模拟工作负载或重放体积和动态的快照,就像我们在自动收报机工厂中处理的那样。我们以开源形式提供该工具。作为正在进行的工作的一部分,我们提供了以下方面的详细信息:(b) 工具扳手的当前状态。作为正在进行的工作的一部分,我们提供了以下方面的详细信息:(a) 我们的工作量和对金融信息处理候选平台进行基准测试的要求;(b) 工具扳手的当前状态。作为正在进行的工作的一部分,我们提供了以下方面的详细信息:(b) 工具扳手的当前状态。
更新日期:2020-10-30
down
wechat
bug