This paper discusses the differences in perceived usability evaluations between crowdsourcing platform and laboratory testing. A retrospective evaluation measurement method was adopted. A SUS quality inspection method with two new sets of inspection items was established to more precisely observe certain low-quality feedback. We also tested a combination of the traditional check method (Items 2 and 3) and all inspection item pairs to observe the cleaning effect. The results of this study suggests that (i) crowdsourcing platform scores (using SUS) are higher than laboratory test scores, though their quality may not be as good, while the difference in CGS was only one grade. (ii) Using one inspection item pair is an acceptable cleaning method, but it may lead to over-cleaning. (iii) The internal relevance of the inspection item pairs is very high. The SUS quality inspection method proposed in this paper can improve the accuracy of perceived usability evaluations. (iv) The completion-time-based cleaning method may not require setting a maximum time in retrospective usability testing.

本文讨论了众包平台和实验室测试在感知可用性评估方面的差异。采用回顾性评估测量方法。建立了具有两套新检查项目的SUS质量检查方法，以更精确地观察某些低质量反馈。我们还测试了传统检查方法（项目2和3）和所有检查项目对的组合，以观察清洁效果。这项研究的结果表明：（i）众包平台评分（使用SUS）高于实验室测试评分，尽管它们的质量可能不那么好，而CGS的差异仅为一个等级。（ii）使用一对检查项目是可接受的清洁方法，但是可能导致过度清洁。（iii）检查项目对的内部相关性很高。本文提出的SUS质量检查方法可以提高感知的可用性评估的准确性。（iv）基于完成时间的清洁方法可能不需要在追溯可用性测试中设置最长时间。