当前位置: X-MOL 学术IEEE Trans. Softw. Eng. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Accelerating Continuous Integration by Caching Environments and Inferring Dependencies
IEEE Transactions on Software Engineering ( IF 6.5 ) Pub Date : 2020-12-31 , DOI: 10.1109/tse.2020.3048335
Keheliya Gallaba 1 , John Ewart 2 , Yves Junqueira 2 , Shane McIntosh 3
Affiliation  

To facilitate the rapid release cadence of modern software (on the order of weeks, days, or even hours), software development organizations invest in practices like Continuous Integration (CI), where each change submitted by developers is built (e.g., compiled, tested, linted) to detect problematic changes early. A fast and efficient build process is crucial to provide timely CI feedback to developers. If CI feedback is too slow, developers may switch contexts to other tasks, which is known to be a costly operation for knowledge workers. Thus, minimizing the build execution time for CI services is an important task. While recent work has made several important advances in the acceleration of CI builds, optimizations often depend upon explicitly defined build dependency graphs (e.g., make, Gradle, CloudBuild, Bazel). These hand-maintained graphs may be (a) underspecified, leading to incorrect build behaviour; or (b) overspecified, leading to missed acceleration opportunities. In this paper, we propose Kotinos—a language-agnostic approach to infer data from which build acceleration decisions can be made without relying upon build specifications. After inferring this data, our approach accelerates CI builds by caching the build environment and skipping unaffected build steps. Kotinos is at the core of a commercial CI service with a growing customer base. To evaluate Kotinos, we mine 14,364 historical CI build records spanning three proprietary and seven open-source software projects. We find that: (1) at least 87.9 percent of the builds activate at least one Kotinos acceleration; and (2) 74 percent of accelerated builds achieve a speed-up of two-fold with respect to their non-accelerated counterparts. Moreover, (3) the benefits of Kotinos can also be replicated in open source software systems; and (4) Kotinos imposes minimal resource overhead (i.e., $<$< 1 percent median CPU usage, 2 MB – 2.2 GB median memory usage, and 0.4 GB – 5.2 GB median storage overhead) and does not compromise build outcomes. Our results suggest that migration to Kotinos yields substantial benefits with minimal investment of effort (e.g., no migration of build systems is necessary).

中文翻译:

通过缓存环境和推断依赖关系加速持续集成

为了促进现代软件的快速发布节奏(数周、数天甚至数小时),软件开发组织投资于持续集成 (CI) 等实践,其中构建开发人员提交的每个更改(例如,编译、测试, linted) 以及早发现有问题的更改。快速高效的构建过程对于向开发人员提供及时的 CI 反馈至关重要。如果 CI 反馈太慢,开发人员可能会将上下文切换到其他任务,这对于知识工作者来说是一项成本高昂的操作。因此,最小化 CI 服务的构建执行时间是一项重要任务。虽然最近的工作在加速 CI 构建方面取得了一些重要进展,但优化通常依赖于明确定义的构建依赖关系图(例如,make、Gradle、CloudBuild、Bazel)。这些手动维护的图表可能 (a) 未指定,导致不正确的构建行为;或 (b) 过度指定,导致错过加速机会。在本文中,我们提出Kotinos——一种与语言无关的方法来推断数据,可以从中做出构建加速决策,而无需依赖构建规范。在推断出这些数据后,我们的方法通过缓存构建环境并跳过不受影响的构建步骤来加速 CI 构建。Kotinos是具有不断增长的客户群的商业 CI 服务的核心。为了评估Kotinos,我们挖掘了跨越三个专有软件项目和七个开源软件项目的 14,364 个历史 CI 构建记录。我们发现:(1)至少 87.9% 的构建激活了至少一个Kotinos加速;(2) 74% 的加速构建相对于非加速构建实现了两倍的加速。此外,(3) 的好处Kotinos也可以在开源软件系统中复制;(4) Kotinos施加最小的资源开销(即,$<$<1% 的 CPU 使用中值、2 MB - 2.2 GB 的内存使用中值和 0.4 GB - 5.2 GB 的中值存储开销)并且不会影响构建结果。我们的结果表明,迁移到Kotinos 会以最少的工作投入产生巨大的收益(例如,不需要迁移构建系统)。
更新日期:2020-12-31
down
wechat
bug