Abstract

Answers from open-ended questions are a valuable part of journalism surveys. However, due to the expense and difficulties involved in manual coding, the current situation, in which open-ended questions are used and analyzed in large-scale web surveys, is not satisfactory. This article reviews the types, coding tasks, and automatic coding techniques of open-ended questions. We propose a five-step procedure on how to analyze open-ended questions through an automatic coding approach, the process of which is as follows: (a) locate the type of open-ended question, (b) choose the corresponding coding task, (c) adopt the appropriate automatic coding techniques, (d) perform the analysis, and (e) evaluate and interpret the results. We demonstrate this with survey data from Reuters Digital News Reports of 2019. Our proposed framework can serve as a practical guide for analyzing open-ended questions with automatic coding and also promote open science in journalism research. We conclude that although automatic coding cannot entirely replace human coding, the constant refinement of the statistical models and the promotion of open sharing of textual data will gradually render autocoding a standard tool for researchers in journalism and communication.

摘要

开放式问题的答案是新闻调查的重要组成部分。然而，由于手动编码所涉及的费用和困难，目前在大规模网络调查中使用和分析开放式问题的情况并不令人满意。本文回顾了开放式问题的类型、编码任务和自动编码技术。我们提出了一个关于如何通过自动编码方法分析开放式问题的五步程序，其过程如下：（a）定位开放式问题的类型，（b）选择相应的编码任务， (c) 采用适当的自动编码技术，(d) 进行分析，以及 (e) 评估和解释结果。我们通过 2019 年路透社数字新闻报道的调查数据证明了这一点。我们提出的框架可以作为使用自动编码分析开放式问题的实用指南，也可以促进新闻研究中的开放科学。我们的结论是，虽然自动编码不能完全取代人工编码，但统计模型的不断完善和文本数据开放共享的推广将逐渐使自动编码成为新闻和传播研究人员的标准工具。