Performance evaluation is crucial to understanding the behavior of scientific workflows. In this study, we target an emerging type of workflow, called in situ workflows. These workflows tightly couple components such as simulation and analysis to improve overall workflow performance. To understand the tradeoffs of various configurable parameters for coupling these heterogeneous tasks, namely simulation stride, and component placement, separately monitoring each component is insufficient to gain insights into the entire workflow behavior. Through an analysis of the state-of-the-art research, we propose a lightweight metric, derived from a defined in situ step, for assessing resource usage efficiency of an in situ workflow execution. By applying this metric to a synthetic workflow, which is parameterized to emulate behaviors of a molecular dynamics simulation, we explore two possible scenarios (Idle Simulation and Idle Analyzer) for the characterization of in situ workflow execution. In addition to preliminary results from a recently published study [11], we further exploit the proposed metric to evaluate a practical in situ workflow with a real molecular dynamics application, i.e. GROMACS. Experimental results show that the in transit placement (analytics on dedicated nodes) sustains a higher frequency for performing in situ analysis compared to the helper-core configuration (analytics co-allocated with simulation).

绩效评估对于了解科学工作流程的行为至关重要。在这项研究中，我们针对一种新兴的工作流，称为原位工作流。这些工作流程将模拟和分析等组件紧密耦合在一起，以提高整体工作流程性能。要了解用于耦合这些异构任务（即仿真步幅和组件放置）的各种可配置参数的权衡，单独监视每个组件不足以深入了解整个工作流程行为。通过国家的最先进的研究分析，我们提出了一个轻量级的度量标准，从定义的派生原位步骤，评估的资源使用效率原位工作流程执行。通过将此指标应用于合成工作流程，该工作流程已参数化以模拟分子动力学仿真的行为，我们探索了两种可能的情况（空闲仿真和空闲分析器）来表征原位工作流程执行。除了最近发表的一项研究[11]的初步结果外，我们还利用拟议的指标来评估具有实际分子动力学应用程序即GROMACS的实际原位工作流程。实验结果表明，与辅助核心配置（分析与模拟共同分配）相比，运输中的放置（专用节点上的分析）在执行原位分析时具有更高的频率。