Creativity research crucially relies on creativity evaluations by external raters, but it is not clear what properties characterize good raters. In the present study, we investigated whether rater personality and rater creativity are related to discernment (i.e., the ability to distinguish creative from uncreative responses) when evaluating others’ responses. To this end, 166 novice raters evaluated a pre-selected set of responses from the alternate uses task for creativity. We further assessed the rater’s creativity and personality and explored the role of task factors such as rating motivation and rating time. We found that creativity in terms of creative potential and creative achievement independently predicted rater discernment. At the level of personality, openness to experience was related to higher discernment, but not any other Big-Five factors or dark triad traits. Rater discernment also increased with rating time, rating motivation, and rater leniency (i.e., the tendency to appreciate creative responses). Finally, we seized the large rater sample to estimate to what extent inter-rater reliability depends on rater number and how dependable ratings are across the rating scale. Together, these findings highlight relevant factors at person, task, and method level predicting individual discernment, which should be considered to ensure good inter-rater reliability in creativity ratings.

中文翻译：

---

评估评估者：哪些因素可以预测新手创造力评估者的辨别力？

创造力研究关键依赖于外部评估者的创造力评估，但尚不清楚优秀评估者的特征。在本研究中，我们调查了评估者的个性和创造力是否与评估他人反应时的辨别力（即区分创造性和非创造性反应的能力）有关。为此，166 名新手评估者评估了从替代用途任务中预先选择的一组响应的创造力。我们进一步评估了评分者的创造力和个性，并探讨了评分动机和评分时间等任务因素的作用。我们发现，在创造潜力和创造成就方面的创造力独立地预测了评估者的辨别力。在人格层面上，对经验的开放性与更高的洞察力有关，但没有任何其他大五因素或黑暗三合会特征。评分者的辨别力也随着评分时间、评分动机和评分者的宽容度（即欣赏创造性反应的倾向）而增加。最后，我们抓住了较大的评分者样本来估计评分者间的可靠性在多大程度上取决于评分者的数量以及评分在整个评分量表中的可靠性。总之，这些发现突出了预测个人辨别力的人、任务和方法级别的相关因素，应考虑这些因素以确保在创造力评级中具有良好的评估者间可靠性。我们抓住了较大的评分者样本来估计评分者间的可靠性在多大程度上取决于评分者数量以及评分在整个评分量表中的可靠性。总之，这些发现突出了预测个人辨别力的人、任务和方法级别的相关因素，应考虑这些因素以确保在创造力评级中具有良好的评估者间可靠性。我们抓住了较大的评分者样本来估计评分者间的可靠性在多大程度上取决于评分者数量以及评分在整个评分量表中的可靠性。总之，这些发现突出了预测个人辨别力的人、任务和方法级别的相关因素，应考虑这些因素以确保在创造力评级中具有良好的评估者间可靠性。