The rise of increasingly more powerful chatbots offers a new way to collect information through conversational surveys, where a chatbot asks open-ended questions, interprets a user’s free-text responses, and probes answers whenever needed. To investigate the effectiveness and limitations of such a chatbot in conducting surveys, we conducted a field study involving about 600 participants. In this study with mostly open-ended questions, half of the participants took a typical online survey on Qualtrics and the other half interacted with an AI-powered chatbot to complete a conversational survey. Our detailed analysis of over 5,200 free-text responses revealed that the chatbot drove a significantly higher level of participant engagement and elicited significantly better quality responses measured by Gricean Maxims in terms of their informativeness, relevance, specificity, and clarity. Based on our results, we discuss design implications for creating AI-powered chatbots to conduct effective surveys and beyond.

中文翻译：

---

说说你自己

越来越强大的聊天机器人的兴起提供了一种通过对话调查收集信息的新方法，其中聊天机器人提出开放式问题，解释用户的自由文本回复，并在需要时探索答案。为了调查这种聊天机器人在进行调查时的有效性和局限性，我们进行了一项涉及约 600 名参与者的实地研究。在这项大多是开放式问题的研究中，一半的参与者参加了关于 Qualtrics 的典型在线调查，另一半与人工智能驱动的聊天机器人互动以完成对话调查。我们对 5,200 多个自由文本回复的详细分析表明，聊天机器人显着提高了参与者的参与度，并在 Gricean Maxims 衡量的信息量方面得到了明显更好的质量回复，相关性、特异性和清晰度。根据我们的结果，我们讨论了创建 AI 驱动的聊天机器人以进行有效调查及其他方面的设计含义。