Comparing the effectiveness of capture and replay against automatic input generation for Android graphical user interface testing,Software Testing, Verification and Reliability

当前位置： X-MOL 学术 › Softw. Test. Verif. Reliab. › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

Comparing the effectiveness of capture and replay against automatic input generation for Android graphical user interface testing
Software Testing, Verification and Reliability ( IF 1.5 ) Pub Date : 2020-10-16 , DOI: 10.1002/stvr.1754
Sergio Di Martino ₁ , Anna Rita Fasolino ₁ , Luigi Libero Lucio Starace ₁ , Porfirio Tramontana ₁

Affiliation

Exploratory testing and fully automated testing tools represent two viable and cheap alternatives to traditional test‐case‐based approaches for graphical user interface (GUI) testing of Android apps. The former can be executed by capture and replay tools that directly translate execution scenarios registered by testers in test cases, without requiring preliminary test‐case design and advanced programming/testing skills. The latter tools are able to test Android GUIs without tester intervention. Even if these two strategies are widely employed, to the best of our knowledge, no empirical investigation has been performed to compare their performance and obtain useful insights for a project manager to establish an effective testing strategy. In this paper, we present two experiments we carried out to compare the effectiveness of exploratory testing approaches using a capture and replay tool (Robotium Recorder) against three freely available automatic testing tools (AndroidRipper, Sapienz, and Google Robo). The first experiment involved 20 computer engineering students who were asked to record testing executions, under strict temporal limits and no access to the source code. Results were slightly better than those of fully automated tools, but not in a conclusive way. In the second experiment, the same students were asked to improve the achieved testing coverage by exploiting the source code and the coverage obtained in the previous tests, without strict temporal constraints. The results of this second experiment showed that students outperformed the automated tools especially for long/complex execution scenarios. The obtained findings provide useful indications for deciding testing strategies that combine manual exploratory testing and automated testing.

中文翻译：

比较捕获和重放与针对Android图形用户界面测试的自动输入生成的有效性

探索性测试和全自动测试工具代表了对Android应用程序进行图形用户界面（GUI）测试的传统基于测试案例的方法的两种可行且廉价的选择。前者可以通过捕获和重播工具执行，这些工具可以直接转换测试人员在测试用例中注册的执行方案，而无需初步的测试用例设计和高级的编程/测试技能。后者工具无需测试人员干预即可测试Android GUI。即使广泛采用了这两种策略，就我们所知，也没有进行任何实证研究来比较它们的性能并为项目经理建立有效的测试策略提供有用的见解。在本文中，我们提供了两个实验，以比较使用捕获和重播工具（Robotium Recorder）与三个免费提供的自动测试工具（AndroidRipper，Sapienz和Google Robo）进行探索性测试方法的有效性。第一个实验涉及20位计算机工程专业的学生，他们被要求在严格的时间限制下并且无法访问源代码来记录测试执行情况。结果比全自动工具的结果略好，但不是结论性的。在第二个实验中，要求相同的学生在没有严格的时间限制的情况下，通过利用源代码和先前测试中获得的覆盖率来提高所达到的测试覆盖率。第二个实验的结果表明，特别是对于长/复杂执行场景，学生的表现优于自动化工具。获得的发现为确定结合手动探索性测试和自动化测试的测试策略提供了有用的指示。

更新日期：2020-10-16

点击分享查看原文

点击收藏

阅读更多本刊最新论文