A distinctive feature of game-based learning environments is their capacity for enabling stealth assessment. Stealth assessment analyzes a stream of fine-grained student interaction data from a game-based learning environment to dynamically draw inferences about students’ competencies through evidence-centered design. In evidence-centered design, evidence models have been traditionally designed using statistical rules authored by domain experts that are encoded using Bayesian networks. This article presents DeepStealth , a deep learning-based stealth assessment framework, that yields significant reductions in the feature engineering labor that has previously been required to create stealth assessments. DeepStealth utilizes end-to-end trainable deep neural network-based evidence models. Using this framework, evidence models are devised using a set of predictive features captured from raw, low-level interaction data to infer evidence for competencies. We investigate two deep learning-based evidence models, long short-term memory networks (LSTMs) and n -gram encoded feedforward neural networks (FFNNs). We compare these models’ predictive performance for inferring students’ knowledge to linear-chain conditional random fields (CRFs) and naïve Bayes models. We perform feature set-level analyses of game trace logs and external pre-learning measures, and we examine the models’ early prediction capacity. The framework is evaluated using data collected from 182 middle school students interacting with a game-based learning environment for middle grade computational thinking. Results indicate that LSTM-based stealth assessors outperform competitive baseline approaches with respect to predictive accuracy and early prediction capacity. We find that LSTMs, FFNNs, and CRFs all benefit from combined feature sets derived from both game trace logs and external pre-learning measures.

中文翻译：

---

深潜：具有深度神经网络的基于游戏的学习潜行评估

基于游戏的学习环境的一个显着特征是其启用隐身评估的能力。隐身评估分析了来自基于游戏的学习环境中的一系列细粒度的学生互动数据，以通过以证据为中心的设计动态得出有关学生能力的推论。在以证据为中心的设计中，传统上使用领域专家编写的统计规则（使用贝叶斯网络编码）来设计证据模型。本文介绍深潜 ，这是一种基于深度学习的隐身评估框架，可显着减少以前创建隐身评估所需的要素工程工作。 深潜利用端到端可训练的基于深度神经网络的证据模型。使用此框架，可以使用从原始的低层交互数据中捕获的一组预测特征来设计证据模型，以推断出胜任力的证据。我们研究了两种基于深度学习的证据模型，长期短期记忆网络（LSTM）和ñ -gram编码的前馈神经网络（FFNN）。我们比较了这些模型在将学生的知识推断为线性链条件随机场（CRF）和朴素贝叶斯模型方面的预测性能。我们对游戏跟踪日志和外部预学习措施进行功能集级别的分析，并检查模型的早期预测能力。该框架是使用从182名中学生收集的数据进行评估的，这些数据与基于游戏的学习环境进行交互，用于中级计算思维。结果表明，基于LSTM的隐身评估师在预测准确性和早期预测能力方面优于竞争基准方法。我们发现LSTM，FFNN和CRF都受益于从游戏跟踪日志和外部预学习措施中获得的组合功能集。