To provide low-latency and high-throughput guarantees, most large key-value stores keep the data in the memory of many servers. Despite the natural parallelism across lookups, the load imbalance, introduced by heavy skew in the popularity distribution of keys, limits performance. To avoid violating tail latency service-level objectives, systems tend to keep server utilization low and organize the data in micro-shards, which provides units of migration and replication for the purpose of load balancing. These techniques reduce the skew but incur additional monitoring, data replication, and consistency maintenance overheads. In this work, we introduce RackOut, a memory pooling technique that leverages the one-sided remote read primitive of emerging rack-scale systems to mitigate load imbalance while respecting service-level objectives. In RackOut, the data are aggregated at rack-scale granularity, with all of the participating servers in the rack jointly servicing all of the rack’s micro-shards. We develop a queuing model to evaluate the impact of RackOut at the datacenter scale. In addition, we implement a RackOut proof-of-concept key-value store, evaluate it on two experimental platforms based on RDMA and Scale-Out NUMA, and use these results to validate the model. We devise two distinct approaches to load balancing within a RackOut unit, one based on random selection of nodes—RackOut_static—and another one based on an adaptive load balancing mechanism—RackOut_adaptive. Our results show that RackOut_static increases throughput by up to 6× for RDMA and 8.6&times for Scale-Out NUMA compared to a scale-out deployment, while respecting tight tail latency service-level objectives. RackOut_adaptive improves the throughput by 30% for workloads with 20% of writes over RackOut_static.

中文翻译：

---

使用机架级内存池缓解分布式数据服务中的负载不平衡

为了提供低延迟和高吞吐量的保证，大多数大型键值存储将数据保存在许多服务器的内存中。尽管查找之间具有自然的并行性，但由于键的流行分布中的严重偏差引入的负载不平衡限制了性能。为避免违反尾延迟服务级别目标，系统倾向于保持较低的服务器利用率并将数据组织在微分片中，微分片提供迁移和复制单元以达到负载平衡的目的。这些技术减少了偏差，但会产生额外的监控、数据复制和一致性维护开销。在这项工作中，我们介绍了 RackOut，这是一种内存池技术，它利用新兴机架规模系统的单边远程读取原语来减轻负载不平衡，同时尊重服务级别目标。在 RackOut 中，数据以机架规模的粒度聚合，机架中的所有参与服务器共同为机架的所有微分片提供服务。我们开发了一个排队模型来评估 RackOut 在数据中心范围内的影响。此外，我们实现了一个 RackOut 概念验证键值存储，在基于 RDMA 和 Scale-Out NUMA 的两个实验平台上对其进行评估，并使用这些结果来验证模型。我们设计了两种不同的方法来在 RackOut 单元内进行负载平衡，一种基于随机选择节点 - RackOut_static，另一种基于自适应负载平衡机制 - RackOut_adaptive。我们的结果表明，与横向扩展部署相比，RackOut_static 将 RDMA 的吞吐量提高了 6 倍，将横向扩展 NUMA 提高了 8.6 倍，同时遵守严格的尾部延迟服务级别目标。