当前位置: X-MOL 学术Future Gener. Comput. Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
WfCommons: A framework for enabling scientific workflow research and development
Future Generation Computer Systems ( IF 6.2 ) Pub Date : 2021-10-06 , DOI: 10.1016/j.future.2021.09.043
Tainã Coleman 1, 2 , Henri Casanova 3 , Loïc Pottier 1 , Manav Kaushik 2 , Ewa Deelman 1, 2 , Rafael Ferreira da Silva 1, 2
Affiliation  

Scientific workflows are a cornerstone of modern scientific computing. They are used to describe complex computational applications that require efficient and robust management of large volumes of data, which are typically stored/processed on heterogeneous, distributed resources. The workflow research and development community has employed a number of methods for the quantitative evaluation of existing and novel workflow algorithms and systems. In particular, a common approach is to simulate workflow executions. In previous works, we have presented a collection of tools that have been adopted by the community for conducting workflow research. Despite their popularity, they suffer from several shortcomings that prevent easy adoption, maintenance, and consistency with the evolving structures and computational requirements of production workflows. In this work, we present WfCommons , a framework that provides a collection of tools for analyzing workflow executions, for producing generators of synthetic workflows, and for simulating workflow executions. We demonstrate the realism of the generated synthetic workflows by comparing their simulated executions to real workflow executions. We also contrast these results with results obtained when using the previously available collection of tools. We find that the workflow generators that are automatically constructed by our framework not only generate representative same-scale workflows (i.e., with structures and task characteristics distributions that resemble those observed in real-world workflows), but also do so at scales larger than that of available real-world workflows. Finally, we conduct a case study to demonstrate the usefulness of our framework for estimating the energy consumption of large-scale workflow executions.



中文翻译:

WfCommons:支持科学工作流研究和开发的框架

科学工作流是现代科学计算的基石。它们用于描述复杂的计算应用程序,这些应用程序需要对大量数据进行高效和稳健的管理,这些数据通常在异构分布式资源上存储/处理。工作流研究和开发社区采用了多种方法对现有和新颖的工作流算法和系统进行定量评估。特别是,一种常见的方法是模拟工作流执行。在之前的工作中,我们展示了一系列被社区用于进行工作流研究的工具。尽管它们很受欢迎,但它们仍存在一些缺点,无法轻松采用、维护以及与生产工作流程不断发展的结构和计算要求保持一致。WfCommons,一个框架,它提供了一组工具,用于分析工作流执行、生成合成工作流的生成器以及模拟工作流执行。我们通过将模拟执行与真实工作流执行进行比较来证明生成的合成工作流的真实性。我们还将这些结果与使用以前可用的工具集合时获得的结果进行对比。我们发现由我们的框架自动构建的工作流生成器不仅生成具有代表性的相同规模的工作流(即,结构和任务特征分布类似于现实世界工作流中观察到的分布),而且还以比实际工作流更大的规模生成可用的实际工作流程。最后,

更新日期:2021-10-14
down
wechat
bug