当前位置: X-MOL 学术Behav. Res. Methods › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Can intelligent agents improve data quality in online questiosnnaires? A pilot study
Behavior Research Methods ( IF 5.953 ) Pub Date : 2021-04-05 , DOI: 10.3758/s13428-021-01574-w
Arne Söderström 1 , Adrian Shatte 2 , Matthew Fuller-Tyszkiewicz 1, 3
Affiliation  

We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality.



中文翻译:

智能代理能否提高在线问卷中的数据质量?一项试点研究

我们探索了聊天机器人在提高通过在线调查收集的数据质量方面的效用。通过 Prolific Academic 抽样的 300 名澳大利亚成年人在聊天机器人支持或无人协助的在线问卷条件中随机分配。调查问卷包括经过验证的措施,以及制定的挑战项目,但与经过验证的目标保持一致。聊天机器人条件通过虚拟支持代理提供了有关项目清晰度的可选帮助。聊天机器人的使用和用户满意度是通过会话日志和用户反馈来衡量的。数据质量作为验证和挑战措施之间关系的组间差异进行操作。调查结果广泛支持在线调查的聊天机器人实用程序,表明大多数具有聊天机器人访问权限的参与者都使用了它,发现它很有帮助,并展示了适度改善的数据质量(与对照相比)。一个挑战项目没有混淆被认为导致了低估的影响。调查结果表明,辅助聊天机器人可以提高数据质量,如果可用,将被许多参与者使用,并且被大多数用户认为是有益的。该试点研究的范围限制被认为导致了低估的影响。未来对包含扩展项目难度的较长形式的问卷进行的测试可能会进一步了解聊天机器人在调查完成和数据质量方面的实用性。如果可用,将被许多参与者使用,并且被大多数用户认为是有益的。该试点研究的范围限制被认为导致了低估的影响。未来对包含扩展项目难度的较长形式的问卷进行的测试可能会进一步了解聊天机器人在调查完成和数据质量方面的实用性。如果可用,将被许多参与者使用,并且被大多数用户认为是有益的。该试点研究的范围限制被认为导致了低估的影响。未来对包含扩展项目难度的较长形式的问卷进行的测试可能会进一步了解聊天机器人在调查完成和数据质量方面的实用性。

更新日期:2021-04-06
down
wechat
bug