In situ visualization on high-performance computing systems allows us to analyze simulation results that would otherwise be impossible, given the size of the simulation data sets and offline post-processing execution time. We develop an in situ adaptor for Paraview Catalyst and Nek5000, a massively parallel Fortran and C code for computational fluid dynamics. We perform a strong scalability test up to 2048 cores on KTH’s Beskow Cray XC40 supercomputer and assess in situ visualization’s impact on the Nek5000 performance. In our study case, a high-fidelity simulation of turbulent flow, we observe that in situ operations significantly limit the strong scalability of the code, reducing the relative parallel efficiency to only \(\approx 21\%\) on 2048 cores (the relative efficiency of Nek5000 without in situ operations is \(\approx 99\%\)). Through profiling with Arm MAP, we identified a bottleneck in the image composition step (that uses the Radix-kr algorithm) where a majority of the time is spent on MPI communication. We also identified an imbalance of in situ processing time between rank 0 and all other ranks. In our case, better scaling and load-balancing in the parallel image composition would considerably improve the performance of Nek5000 with in situ capabilities. In general, the result of this study highlights the technical challenges posed by the integration of high-performance simulation codes and data-analysis libraries and their practical use in complex cases, even when efficient algorithms already exist for a certain application scenario.

考虑到仿真数据集的大小和离线后处理执行时间，高性能计算系统上的原位可视化使我们能够分析原本不可能的仿真结果。我们为 Paraview Catalyst 和 Nek5000 开发了一个原位适配器，这是一种用于计算流体动力学的大规模并行 Fortran 和 C 代码。我们在 KTH 的 Beskow Cray XC40 超级计算机上执行高达 2048 个内核的强大可扩展性测试，并评估原位可视化对 Nek5000 性能的影响。在我们的研究案例中，湍流的高保真模拟，我们观察到原位操作显着限制了代码的强大可扩展性，将相对并行效率降低到只有\(\约 21\%\)在 2048 个内核上（Nek5000 在没有原位操作的情况下的相对效率为\(\约 99\%\)）。通过使用 Arm MAP 进行分析，我们确定了图像合成步骤（使用 Radix-kr 算法）中的一个瓶颈，其中大部分时间都花在了 MPI 通信上。我们还发现 rank 0 和所有其他 rank 之间的原位处理时间不平衡。在我们的案例中，并行图像合成中更好的缩放和负载平衡将显着提高具有原位功能的 Nek5000 的性能。总的来说，这项研究的结果突出了高性能仿真代码和数据分析库的集成以及它们在复杂情况下的实际使用所带来的技术挑战，即使在特定应用场景中已经存在有效算法的情况下也是如此。