Online reviews contain many vital insights for quality management, but the volume of content makes identifying defect-related discussion difficult. This paper critically assesses multiple approaches for detecting defect-related discussion, ranging from out-of-the-box sentiment analyses to supervised and unsupervised machine-learned defect terms. We examine reviews from 25 product and service categories to assess each method’s performance. We examine each approach across the broad cross-section of categories as well as when tailored to a singular category of study. Surprisingly, we found that negative sentiment was often a poor predictor of defect-related discussion. Terms generated with unsupervised topic modeling tended to correspond to generic product discussions rather than defect-related discussion. Supervised learning techniques outperformed the other text analytic techniques in our cross-category analysis, and they were especially effective when confined to a single category of study. Our work suggests a need for category-specific text analyses to take full advantage of consumer-driven quality intelligence.

在线评论包含许多对质量管理至关重要的见解，但是内容的数量使识别与缺陷相关的讨论变得困难。本文对评估与缺陷相关的讨论的多种方法进行了严格的评估，从开箱即用的情感分析到有监督和无监督的机器学习缺陷术语，不一而足。我们检查来自25种产品和服务类别的评论，以评估每种方法的性能。我们研究了跨大类的每种方法以及针对单个研究类别量身定制的方法。令人惊讶的是，我们发现负面情绪通常不能很好地预测与缺陷相关的讨论。使用无监督主题建模生成的术语倾向于对应于通用产品讨论，而不是与缺陷相关的讨论。在我们的跨类别分析中，有监督的学习技术优于其他文本分析技术，当仅限于单个研究类别时，它们尤其有效。我们的工作表明需要特定类别的文本分析，以充分利用消费者驱动的质量智能。