For a decade, the Ceph distributed file system followed the conventional wisdom of building its storage backend on top of local file systems. This is a preferred choice for most distributed file systems today, because it allows them to benefit from the convenience and maturity of battle-tested code. Ceph’s experience, however, shows that this comes at a high price. First, developing a zero-overhead transaction mechanism is challenging. Second, metadata performance at the local level can significantly affect performance at the distributed level. Third, supporting emerging storage hardware is painstakingly slow. Ceph addressed these issues with BlueStore, a new backend designed to run directly on raw storage devices. In only two years since its inception, BlueStore outperformed previous established backends and is adopted by 70% of users in production. By running in user space and fully controlling the I/O stack, it has enabled space-efficient metadata and data checksums, fast overwrites of erasure-coded data, inline compression, decreased performance variability, and avoided a series of performance pitfalls of local file systems. Finally, it makes the adoption of backward-incompatible storage hardware possible, an important trait in a changing storage landscape that is learning to embrace hardware diversity.

中文翻译：

---

分布式存储系统中自定义存储后端的案例

十年来，Ceph 分布式文件系统遵循在本地文件系统之上构建其存储后端的传统智慧。这是当今大多数分布式文件系统的首选，因为它允许它们从经过实战测试的代码的便利性和成熟度中受益。然而，Ceph 的经验表明，这是要付出高昂代价的。首先，开发零开销交易机制具有挑战性。其次，本地级别的元数据性能会显着影响分布式级别的性能。第三，支持新兴存储硬件非常缓慢。Ceph 使用 BlueStore 解决了这些问题，BlueStore 是一种旨在直接在原始存储设备上运行的新后端。成立仅两年时间，BlueStore 优于以前建立的后端，并被 70% 的用户在生产中采用。通过在用户空间运行并完全控制 I/O 堆栈，它实现了节省空间的元数据和数据校验和、快速覆盖纠删码数据、内联压缩、降低性能可变性，并避免了本地文件的一系列性能陷阱系统。最后，它使采用向后不兼容的存储硬件成为可能，这是在不断变化的存储环境中学习拥抱硬件多样性的一个重要特征。