Various software-engineering problems have been solved by crowdsourcing. In many projects, the software outsourcing process is streamlined on cloud-based platforms. Among software engineering tasks, test-case development is particularly suitable for crowdsourcing, because a large number of test cases can be generated at little monetary cost. However, the numerous test cases harvested from crowdsourcing can be high- or low-quality. Owing to the large volume, distinguishing the high-quality tests by traditional techniques is computationally expensive. Therefore, crowdsourced testing would benefit from an efficient mechanism distinguishes the qualities of the test cases. This paper introduces an automated approach — TCQA — to evaluate the quality of test cases based on the onsite coding history. Quality assessment by TCQA proceeds through three steps: (1) modeling the code history as a time series, (2) extracting the multiple relevant features from the time series, and (3) building a model that classifies the test cases based on their qualities. Step (3) is accomplished by feature-based machine-learning techniques. By leveraging the onsite coding history, TCQA can assess the test-case quality without performing expensive source-code analysis or executing the test cases. Using the data of nine test-development tasks involving more than 400 participants, we evaluated TCQA from multiple perspectives. The TCQA approach assessed the quality of the test cases with higher precision, faster speed, and lower overhead than conventional test-case quality-assessment techniques. Moreover, TCQA provided yield real-time insights on test-case quality before the assessment was finished.

众包解决了各种软件工程问题。在许多项目中，在基于云的平台上简化了软件外包过程。在软件工程任务中，测试用例开发特别适合于众包，因为可以用很少的金钱成本生成大量的测试用例。但是，从众包中获得的大量测试用例可以是高质量的，也可以是劣质的。由于体积大，通过传统技术区分高质量测试的计算量很大。因此，众包测试将受益于区分测试用例质量的有效机制。本文介绍了一种自动方法TCQA，用于根据现场编码历史来评估测试用例的质量。TCQA进行质量评估的过程分为三个步骤：（1）将代码历史记录建模为一个时间序列，（2）从该时间序列中提取多个相关特征，以及（3）建立一个基于测试用例的质量对其进行分类的模型。步骤（3）通过基于特征的机器学习技术完成。通过利用现场编码历史记录，TCQA可以评估测试用例的质量，而无需执行昂贵的源代码分析或执行测试用例。利用涉及400多个参与者的9个测试开发任务的数据，我们从多个角度评估了TCQA。TCQA方法比传统的测试用例质量评估技术以更高的精度，更快的速度和更低的开销评估了测试用例的质量。此外，TCQA在评估完成之前提供了有关测试用例质量的实时洞察。