In this work, we propose a highly scalable parallel double binned ghost particle (DBGP) algorithm for direct-forcing immersed boundary spectral element method for multiphase flow simulations. In particular, the DBGP algorithm is designed to obtain fully distributed data storage and scalable data transfer across hundreds of thousands of processors. The proposed algorithm uses a queen and worker data structure for fully resolved particles to demarcate particle-level and marker-level quantities and communication. In the DBGP algorithm, each particle’s centroid is represented by a queen marker and the particle surface is covered with a uniform distribution of surface worker markers. The queen marker contains information on the translational and rotational motion of a particle and integrates the force and torque computed at all the worker markers, while the worker marker implements the fluid–particle interaction. Ghost queen and ghost worker markers are generated for each real queen and real worker marker during computation for particle-level and marker-level communications, respectively. A double Cartesian binning process is introduced that divides the physical domain into a coarse queen-bin and a fine worker-bin structure in three dimensions. The queen-bin and worker-bin sizes are determined by their zone of influence at the particle-level and marker-level communication, respectively. Bin-to-rank maps that relate each queen-bin and worker-bin to all the MPI ranks that they interact with are created. By using the queen/worker marker representation and two-layer bin-to-rank maps, data communication across very large number of MPI ranks is efficiently carried out. A scaling analysis has been conducted, showing excellent performance of the DBGP algorithm for up to 16,384 MPI ranks in both weak and strong scaling studies. The proposed method has been demonstrated to accurately predict sedimentation of particle clouds. The simulated correlation between the mean settling velocity and volume fraction is in good agreement with empirical correlations from previous studies.

中文翻译：

---

一种用于谱元多相流模拟的直接强迫浸入边界法的可扩展并行算法

在这项工作中，我们提出了一种高度可扩展的并行双合并鬼粒子（DBGP）算法，用于多相流模拟的直接强迫浸入边界谱元方法。特别是，DBGP 算法旨在在数十万个处理器之间获得完全分布式的数据存储和可扩展的数据传输。所提出的算法使用皇后和工人数据结构来完全解析粒子来划分粒子级和标记级数量和通信。在 DBGP 算法中，每个粒子的质心由一个皇后标记表示，粒子表面覆盖有均匀分布的表面工人标记。后标记包含有关粒子平移和旋转运动的信息，并整合在所有工作标记处计算的力和扭矩，而工人标记实现流体 - 粒子相互作用。在计算粒子级和标记级通信期间，分别为每个真正的皇后和真正的工人标记生成幽灵女王和幽灵工人标记。引入了双笛卡尔分箱过程，将物理域在三个维度上分为粗皇后箱和细工箱结构。Queen-bin 和 worker-bin 大小分别由它们在粒子级和标记级通信的影响区域决定。创建了将每个 Queen-bin 和 worker-bin 与它们交互的所有 MPI 等级相关联的 bin-to-rank 映射。通过使用女王/工人标记表示和两层 bin-to-rank 映射，可以有效地执行大量 MPI 等级之间的数据通信。进行了缩放分析，显示了 DBGP 算法在弱和强缩放研究中高达 16,384 MPI 等级的出色性能。所提出的方法已被证明可以准确预测粒子云的沉降。平均沉降速度和体积分数之间的模拟相关性与先前研究的经验相关性非常一致。