Distributed streaming applications, i.e., applications that process massive streams of data in a distributed fashion, are becoming increasingly popular to tame the velocity and the volume of Big Data . Nevertheless, the widespread adoption of data-intensive processing is still limited by the non-trivial design paradigms involved, which deal with the unboundedness and volume of involved data streams and by the many distributed streaming platforms, each with its own characteristics and APIs. In this article, we present StreamGen, a Model-Driven Engineering tool to simplify the design of such streaming applications and automatically generate the corresponding code. StreamGen is able to automatically generate fully working and processing-ready code for different target platforms (e.g., Apache Spark, Apache Flink). Evaluation shows that (i) StreamGen is general enough to model and generate the code, offering comparable performance against a preexisting similar and well-known application; (ii) the tool is fully compliant with streaming concepts defined as part of the Google Dataflow Model; and (iii) users with little computer science background and limited experience with big data have been able to work with StreamGen and create/refactor an application in a matter of minutes.

中文翻译：

---

流生成

分布式流应用程序，即以分布式方式处理大量数据流的应用程序，正变得越来越流行，以控制数据流的速度和数量。大数据. 尽管如此，数据密集型处理的广泛采用仍然受到所涉及的非平凡设计范式的限制，这些范式处理所涉及的数据流的无限性和数量，以及许多分布式流媒体平台，每个平台都有自己的特点和 API。在本文中，我们介绍了 StreamGen，这是一种模型驱动的工程工具，用于简化此类流应用程序的设计并自动生成相应的代码。StreamGen 能够为不同的目标平台（例如，Apache Spark、Apache Flink）自动生成完全工作和处理就绪的代码。评估表明 (i) StreamGen 足够通用，可以建模和生成代码，提供与先前存在的类似和知名应用程序相当的性能；(ii) 该工具完全符合定义为 Google 数据流模型一部分的流概念；(iii) 几乎没有计算机科学背景和大数据经验有限的用户已经能够使用 StreamGen 并在几分钟内创建/重构应用程序。