In crowdsourced testing, a large number of test reports will be generated in a short time. How to efficiently inspect these reports becomes one of the critical steps in the testing process. In recent years, many automated techniques like clustering, classification, and prioritization have emerged to provide an automated inspection order over test reports. Even though these methods have achieved good performance, they did not consider the priority to image and text information. Simultaneously, existing prioritization approaches only focus on the rate of detecting faults but ignore the severity of the faults. In fact, bug severity is a vital indicator that the users provide to flag the criticality of a bug, so developers can then use it to set their priority for the resolution process. For these reasons, this paper presents a novel prioritization approach for crowdsourcing test reports. It extracts features from text and screenshot information of the test reports, uses the hash technique to index test reports, and finally designs a prioritization algorithm. To validate our approach, we conducted experiments on six industrial projects. The results and the hypotheses analysis show that our approach can detect all faults faster in a limited time and can prioritize reports that have higher severity faults compared with the existing methods.

在众包测试中，会在短时间内生成大量的测试报告。如何有效地检查这些报告成为测试过程中的关键步骤之一。近年来，出现了许多自动化技术，如聚类、分类和优先级排序，以提供对测试报告的自动化检查顺序。尽管这些方法都取得了很好的性能，但他们没有考虑图像和文本信息的优先级。同时，现有的优先级排序方法只关注故障检测率，而忽略了故障的严重性。事实上，错误严重性是用户提供的一个重要指标，用于标记错误的严重程度，因此开发人员可以使用它来设置解决过程的优先级。由于这些原因，本文提出了一种新的众包测试报告优先排序方法。它从测试报告的文本和屏幕截图信息中提取特征，使用哈希技术对测试报告进行索引，最后设计出优先级排序算法。为了验证我们的方法，我们对六个工业项目进行了实验。结果和假设分析表明，与现有方法相比，我们的方法可以在有限的时间内更快地检测到所有故障，并且可以优先考虑具有更高严重性故障的报告。