Benchmarking systems is difficult. Mistakes can compromise guarantees and threaten reproducibility and comparability. We conduct a study to show that benchmarking flaws are widespread in systems security defense papers, even at tier-1 venues. We aim to raise awareness and provide recommendations for safeguarding the scientific process in our community.

中文翻译：

---

基准缺陷破坏安全研究

基准系统是困难的。错误会影响保证并威胁可重复性和可比性。我们进行了一项研究，以表明基准测试缺陷在系统安全防御论文中很普遍，即使在一级场所也是如此。我们旨在提高认识并提供建议，以保护我们社区的科学过程。