Social sciences expose many cognitively complex, highly qualified, or fuzzy problems, whose resolution relies primarily on expert judgement rather than automated systems. One of such instances that we study in this work is a reflection analysis in the writings of student teachers. We share a hands-on experience on how these challenges can be successfully tackled in data collection for machine learning. Based on the novel deep learning architectures pre-trained for a general language understanding, we can reach an accuracy ranging from 76.56–79.37% on low-confidence samples to 97.56–100% on high confidence cases. We open-source all our resources and models, and use the models to analyse previously ungrounded hypotheses on reflection of university students. Our work provides a toolset for objective measurements of reflection in higher education writings, applicable in more than 100 other languages worldwide with a loss in accuracy measured between 0–4.2% Thanks to the outstanding accuracy of the deep models, the presented toolset allows for previously unavailable applications, such as providing semi-automated student feedback or measuring an effect of systematic changes in the educational process via the students’ response.

社会科学揭示了许多认知复杂、高度合格或模糊的问题，这些问题的解决主要依赖于专家判断，而不是自动化系统。我们在这项工作中研究的其中一个例子是学生教师作品中的反思分析。我们分享了如何在机器学习数据收集中成功应对这些挑战的实践经验。基于针对一般语言理解进行预训练的新型深度学习架构，我们可以在低置信度样本上达到 76.56-79.37% 到高置信度样本上的 97.56-100% 的准确度。我们开源了我们所有的资源和模型，并使用这些模型来分析以前没有根据的关于大学生反思的假设。