Over the past few years, SMT string solvers have found their applications in an increasing number of domains, such as program analyses in mobile and Web applications, which require the ability to reason about string values. A series of research has been carried out to find quality issues of string solvers in terms of its correctness and performance. Yet, none of them has considered the performance regressions happening across multiple versions of a string solver. To fill this gap, in this paper, we focus on solver performance regressions (SPRs), i.e., unintended slowdowns introduced during the evolution of string solvers. To this end, we develop SPRFinderto not only generate test cases demonstrating SPRs, but also localize the probable causes of them, in terms of commits. We evaluated the effectiveness of SPRFinderon three state-of-the-art string solvers, i.e., Z3Seq, Z3Str3, and CVC4. The results demonstrate that SPRFinderis effective in generating SPR-inducing test cases and also able to accurately locate the responsible commits. Specifically, the average running time on the target versions is 13.2× slower than that of the reference versions. Besides, we also conducted the first empirical study to peek into the characteristics of SPRs, including the impact of random seed configuration for SPR detection, understanding the root causes of SPRs, and characterizing the regression test cases through case studies. Finally, we highlight that 149 unique SPR-inducing commits were discovered in total by SPRFinder, and 27of them have been confirmed by the corresponding developers.

中文翻译：

---

揭秘字符串求解器中的性能回归


在过去的几年中，SMT 字符串求解器已在越来越多的领域中得到应用，例如移动和 Web 应用程序中的程序分析，这需要能够推理字符串值。为了发现字符串求解器的正确性和性能方面的质量问题，已经进行了一系列研究。然而，他们都没有考虑过字符串求解器的多个版本中发生的性能回归。为了填补这一空白，在本文中，我们重点关注求解器性能回归（SPR），即字符串求解器演化过程中引入的意外减速。为此，我们开发了 SPRFinder，不仅生成展示 SPR 的测试用例，而且还根据提交来定位它们的可能原因。我们评估了 SPRFinder 在三种最先进的字符串求解器（即 Z3Seq、Z3Str3 和 CVC4）上的有效性。结果表明，SPRFinder 可以有效生成 SPR 引发的测试用例，并且还能够准确定位负责的提交。具体来说，目标版本的平均运行时间比参考版本慢 13.2 倍。此外，我们还进行了首次实证研究来探究SPR的特征，包括随机种子配置对SPR检测的影响、了解SPR的根本原因以及通过案例研究来表征回归测试用例。最后，我们强调，SPRFinder 总共发现了 149 个独特的 SPR 诱导提交，其中 27 个已得到相应开发人员的确认。