In this article, we show that key-value stores backed by a log-structured merge-tree (LSM-tree) exhibit an intrinsic tradeoff between lookup cost, update cost, and main memory footprint, yet all existing designs expose a suboptimal and difficult to tune tradeoff among these metrics. We pinpoint the problem to the fact that modern key-value stores suboptimally co-tune the merge policy, the buffer size, and the Bloom filters’ false-positive rates across the LSM-tree’s different levels. We present Monkey, an LSM-tree based key-value store that strikes the optimal balance between the costs of updates and lookups with any given main memory budget. The core insight is that worst-case lookup cost is proportional to the sum of the false-positive rates of the Bloom filters across all levels of the LSM-tree. Contrary to state-of-the-art key-value stores that assign a fixed number of bits-per-element to all Bloom filters, Monkey allocates memory to filters across different levels so as to minimize the sum of their false-positive rates. We show analytically that Monkey reduces the asymptotic complexity of the worst-case lookup I/O cost, and we verify empirically using an implementation on top of RocksDB that Monkey reduces lookup latency by an increasing margin as the data volume grows (50--80% for the data sizes we experimented with). Furthermore, we map the design space onto a closed-form model that enables adapting the merging frequency and memory allocation to strike the best tradeoff among lookup cost, update cost and main memory, depending on the workload (proportion of lookups and updates), the dataset (number and size of entries), and the underlying hardware (main memory available, disk vs. flash). We show how to use this model to answer what-if design questions about how changes in environmental parameters impact performance and how to adapt the design of the key-value store for optimal performance.

中文翻译：

---

LSM 树的最优布隆过滤器和自适应合并

在本文中，我们展示了由日志结构合并树 (LSM-tree) 支持的键值存储在查找成本、更新成本和主内存占用之间表现出内在的权衡，但所有现有设计都暴露出次优和困难调整这些指标之间的权衡。我们将问题归结为现代键值存储在 LSM-tree 的不同级别上对合并策略、缓冲区大小和 Bloom 过滤器的误报率进行了次优的协同调整。我们展示了 Monkey，这是一种基于 LSM-tree 的键值存储，它可以在任何给定的主内存预算的情况下在更新和查找成本之间取得最佳平衡。核心观点是，最坏情况的查找成本与 LSM 树所有级别的 Bloom 过滤器的误报率之和成正比。与将每个元素的固定位数分配给所有 Bloom 过滤器的最先进的键值存储相反，Monkey 将内存分配给不同级别的过滤器，以最小化其误报率的总和。我们分析表明，Monkey 降低了最坏情况查找 I/O 成本的渐近复杂性，并且我们使用 RocksDB 之上的实现经验验证，随着数据量的增长（50--80我们实验的数据大小的百分比）。此外，我们将设计空间映射到一个封闭形式的模型上，该模型能够调整合并频率和内存分配，以根据工作负载（查找和更新的比例）在查找成本、更新成本和主内存之间取得最佳折衷。数据集（条目的数量和大小），和底层硬件（可用的主内存、磁盘与闪存）。我们展示了如何使用该模型来回答有关环境参数变化如何影响性能以及如何调整键值存储设计以实现最佳性能的假设设计问题。