Network communication is the slowest component of many operators in distributed parallel databases deployed for large-scale analytics. Whereas considerable work has focused on speeding up databases on modern hardware, communication reduction has received less attention. Existing parallel DBMSs rely on algorithms designed for disks with minor modifications for networks. A more complicated algorithm may burden the CPUs but could avoid redundant transfers of tuples across the network. We introduce track join, a new distributed join algorithm that minimizes network traffic by generating an optimal transfer schedule for each distinct join key. Track join extends the trade-off options between CPU and network. Track join explicitly detects and exploits locality, also allowing for advanced placement of tuples beyond hash partitioning on a single attribute. We propose a novel data placement algorithm based on track join that minimizes the total network cost of multiple joins across different dimensions in an analytical workload. Our evaluation shows that track join outperforms hash join on the most expensive queries of real workloads regarding both network traffic and execution time. Finally, we show that our data placement optimization approach is both robust and effective in minimizing the total network cost of joins in analytical workloads.

中文翻译：

---

用于最小网络流量的分布式连接和数据放置

在为大规模分析部署的分布式并行数据库中，网络通信是许多运营商中最慢的组件。尽管大量工作集中在加速现代硬件上的数据库，但减少通信受到的关注较少。现有的并行 DBMS 依赖于为磁盘设计的算法，对网络进行了少量修改。更复杂的算法可能会给 CPU 带来负担，但可以避免元组在网络上的冗余传输。我们引入了跟踪连接，这是一种新的分布式连接算法，通过为每个不同的连接键生成最佳传输计划来最小化网络流量。Track join 扩展了 CPU 和网络之间的权衡选项。Track join 显式检测和利用局部性，还允许在单个属性上进行哈希分区之外的元组高级放置。我们提出了一种基于轨道连接的新型数据放置算法，该算法可以最小化分析工作负载中跨不同维度的多个连接的总网络成本。我们的评估表明，在涉及网络流量和执行时间的真实工作负载的最昂贵查询上，track join 优于 hash join。最后，我们展示了我们的数据放置优化方法在最小化分析工作负载中连接的总网络成本方面既稳健又有效。