ABSTRACT

In recognition of the capability of text-mining models to quantify aspects of language use, some creativity researchers have adopted text-mining models as a mechanism to objectively and efficiently score the Originality of open-ended responses to verbal divergent thinking tasks. With the increasing use of text-mining models in divergent thinking research, concerns have been raised about how text-mining-based Originality estimates can be confounded by other dimensions of divergent thinking, especially Elaboration. Since automated Originality estimates can be influenced by varying amounts of Elaboration, or the number of words a participant uses in a response, some researchers question whether Originality scores are psychometrically valid or if the reliability of Originality scores is dependent on the variance in Elaboration. Using partial correlation procedures, we investigate whether Originality scores generated by a freely available text-mining system are significantly influenced by the amount of Elaboration a participant exhibits in their response to a divergent thinking task. Then, the reliability of Originality scores, before and after the variance accounted for by Elaboration is partialled out, are compared. Results from this brief analysis reveal that, when recent methodological recommendations for automatic Originality scoring are applied, Originality scores generated via the GloVe 840B text-mining system are not meaningfully confounded by Elaboration. We conclude that, even when the variance attributed to Elaboration is partialled out, this scoring method is capable of producing reliable Originality scores.

摘要

认识到文本挖掘模型可以量化语言使用方面的能力，一些创造力研究人员已采用文本挖掘模型作为一种客观有效地评分针对言语分歧思维任务的开放式响应的原创性的机制。随着发散思维研究中越来越多地使用文本挖掘模型，人们越来越担心基于文本挖掘的原始性估计会如何与发散思维的其他维度（尤其是精化）相混淆。由于自动原创性估计会受到复杂程度的变化或参与者在回答中使用的单词数的影响，因此一些研究人员质疑原创性评分在心理上是否有效，或者原创性评分的可靠性是否取决于精心设计的方差。使用偏相关程序，我们调查了由自由可用的文本挖掘系统生成的原创性分数是否受到参与者对不同思维任务的反应所表现出的细化程度的显着影响。然后，比较细化所占方差被分拆前后的原始性得分的可靠性。简短分析的结果表明，当应用最新的自动原创性评分方法建议时，通过GloVe 840B文本挖掘系统生成的原创性分数不会被Elaboration有意义地混淆。我们得出结论，即使部分归因于细化的方差，该评分方法也能够产生可靠的原创性评分。我们调查了可自由使用的文本挖掘系统生成的原创性分数是否受参与者对不同思维任务的反应所表现出的细化程度显着影响。然后，比较了细化所解释的方差被分开之前和之后，独创性得分的可靠性。简短分析的结果表明，当应用最新的自动原创性评分方法建议时，通过GloVe 840B文本挖掘系统生成的原创性分数不会被Elaboration有意义地混淆。我们得出结论，即使部分归因于细化的方差，该评分方法也能够产生可靠的原创性评分。我们调查了可自由使用的文本挖掘系统生成的原创性分数是否受参与者对不同思维任务的反应所表现出的细化程度显着影响。然后，比较了细化所解释的方差被分开之前和之后，独创性得分的可靠性。简短分析的结果表明，当应用最新的自动原创性评分方法建议时，通过GloVe 840B文本挖掘系统生成的原创性分数不会被Elaboration有意义地混淆。我们得出结论，即使部分归因于细化的方差，该评分方法也能够产生可靠的原创性评分。