Fail-slow hardware is an under-studied failure mode. We present a study of 114 reports of fail-slow hardware incidents, collected from large-scale cluster deployments in 14 institutions. We show that all hardware types such as disk, SSD, CPU, memory, and network components can exhibit performance faults. We made several important observations such as faults convert from one form to another, the cascading root causes and impacts can be long, and fail-slow faults can have varying symptoms. From this study, we make suggestions to vendors, operators, and systems designers.

中文翻译：

---

大规模失败

Fail-slow 硬件是一种未被充分研究的故障模式。我们对来自 14 个机构的大规模集群部署的 114 份故障缓慢硬件事件报告进行了研究。我们展示了所有硬件类型，如磁盘、SSD、CPU、内存和网络组件都可能出现性能故障。我们进行了一些重要的观察，例如故障从一种形式转换为另一种形式，级联的根本原因和影响可能很长，并且故障缓慢的故障可能有不同的症状。根据这项研究，我们向供应商、运营商和系统设计人员提出建议。