To fully utilize the resources of a main memory database cluster, we additionally take the independent parallelism into account to parallelize multiple pipelines of one query. However, scheduling resources to multiple pipelines is an intractable problem. Traditional static approaches to this problem may lead to a serious waste of resources and suboptimal execution order of pipelines, because it is hard to predict the actual data distribution and fluctuating workloads at compile time. In response, we propose a dynamic scheduling algorithm, List with Filling and Preemption (LFPS), based on two novel techniques. (1) Adaptive filling improves resource utilization by issuing more extra pipelines to adaptively fill idle resource “holes” during execution. (2) Rank-based preemption strictly guarantees scheduling the pipelines on the critical path first at run time. Interestingly, the latter facilitates the former filling idle “holes” with best efforts to finish multiple pipelines as soon as possible. We implement LFPS in our prototype database system. Under the workloads of TPC-H, experiments show our work improves the finish time of parallelizable pipelines from one query up to 2.5X than a static approach and 2.1X than a serialized execution.

中文翻译：

---

将资源调度到主内存数据库集群中一个查询的多个管道

为了充分利用主存数据库集群的资源，我们额外考虑了独立并行性来并行化一个查询的多个管道。然而，将资源调度到多个管道是一个棘手的问题。解决这个问题的传统静态方法可能会导致资源的严重浪费和流水线的次优执行顺序，因为在编译时很难预测实际数据分布和波动的工作负载。作为回应，我们提出了一种基于两种新技术的动态调度算法，即带填充和抢占的列表（LFPS）。(1) 自适应填充通过在执行期间发出更多额外管道来自适应填充空闲资源“空洞”来提高资源利用率。(2) 基于等级的抢占严格保证在运行时首先在关键路径上调度管道。有趣的是，后者有助于前者尽最大努力尽快完成多条管道填补闲置的“漏洞”。我们在原型数据库系统中实现了 LFPS。在 TPC-H 的工作负载下，实验表明，我们的工作将并行化管道的完成时间从一个查询提高到静态方法的 2.5 倍和序列化执行的 2.1 倍。