SIAM Journal on Scientific Computing, Volume 42, Issue 3, Page C69-C96, January 2020.
High-order discontinuous Galerkin (DG) methods promise to be an excellent discretization paradigm for hyperbolic differential equation solvers running on supercomputers, since they combine high arithmetic intensity with localized data access, since they straightforwardly translate into nonoverlapping domain decomposition, and since they facilitate dynamic adaptivity without the need for conformal meshes. An efficient parallel evaluation of DG weak formulation in an MPI+X setting, however, remains nontrivial as dependency graphs over dynamically adaptive meshes change with each mesh refinement or coarsening, as resolution transitions yield nontrivial data flow dependencies, and as data sent along domain boundaries through message passing (MPI) have to be triggered in the correct order. Domain decomposition with MPI alone starts to become insufficient if the mesh changes very frequently, if mesh changes cannot be predicted, and if limiters and nonlinear per-cell solves yield unpredictable costs per cell. We introduce enclave tasking as a task invocation technique for shared memory and MPI+X: It does not assemble any task graph; instead the mesh traversal spawns ready tasks directly. A marker-and-cell approach ensures that tasks feeding into MPI or triggering mesh modifications as well as latency-sensitive or bandwidth-demanding tasks are processed with high priority. The remaining cell tasks form enclaves, i.e., groups of tasks that can be processed in the background. Enclave tasking introduces high concurrency which is homogeneously distributed over the mesh traversal, it mixes memory-intensive volumetric DG calculations with compute-bound Riemann solves, and it helps to overlap communication with computations. Our work focuses on ADER-DG and patch-based finite volumes. Yet, we discuss how the paradigm can be generalized to the whole DG family and finite volume stand-alone solvers.

中文翻译：

---

动态自适应网格上DG方法的Enclave Tasking

SIAM科学计算杂志，第42卷，第3期，第C69-C96页，2020年1月。
对于在超级计算机上运行的双曲型微分方程求解器，高阶不连续Galerkin（DG）方法有望成为出色的离散化范例，因为它们将高算术强度与局部数据访问结合在一起，因为它们直接转化为非重叠域分解，并且由于它们促进了动态不需要保形网格的适应性。然而，在MPI + X设置中，对DG弱公式的有效并行评估仍然是不平凡的，因为随着网格的细化或粗化，动态自适应网格上的依存关系图会发生变化，因为分辨率转换会产生非平凡的数据流依存关系，并且数据沿域边界发送通过消息传递（MPI）必须以正确的顺序触发。如果网格变化非常频繁，无法预测网格变化以及限制器和每单元非线性解决方案，则仅使用MPI进行域分解就变得不够充分，如果限制器和非线性每个单元解决了每个单元无法预测的成本。我们将安全区任务作为共享内存和MPI + X的任务调用技术引入：它不组装任何任务图；相反，网格遍历会直接生成就绪任务。标记和单元方法可确保以高优先级处理送入MPI或触发网格修改的任务以及对延迟敏感或对带宽有要求的任务。其余的单元任务形成了几个区域，即可以在后台处理的任务组。飞地任务分配引入了高并发性，该并发性均匀地分布在网格遍历上，它将内存密集型体积DG计算与受计算限制的Riemann解决方案混合在一起，并有助于将通信与计算重叠。我们的工作集中在ADER-DG和基于补丁的有限体积上。但是，我们讨论了如何将范式推广到整个DG系列和有限体积的独立求解器。