Objective

To investigate speech features, human–machine alarms, and operator–system interaction for the estimation of cognitive workload in full-scale realistic simulated scenarios.

Background

Theories and models of cognitive workload are critical for the design and evaluation of human–machine systems. Unfortunately, there are very few nonintrusive cognitive workload measures available for realistic dynamic human–machine interaction.

Method

The study was conducted in a full-scope control room research simulator of an advanced nuclear reactor. Six crews, each consisting of three operators, participated in 12 scenarios. The operators rated their workload every second minute. Machine learning algorithms were trained to estimate operators’ workload based on crew communication, operator–system interaction, and system alarms.

Results

Random Forest (RF) utilizing speech and system features achieved an accuracy of 67% on test data. Utilizing speech features only, the accuracy achieved was 63%. The most important speech features were pitch, amplitude, and articulation rate. A 61% accuracy was achieved when alarms and operator–system interaction features were used. The most important features were the number of alarms and amount of operator–system interaction. Accuracy for algorithms trained for each operator ranged from 39% to 98%, with an average of 72%. For a majority of analyses performed, RF and extreme gradient boosting (XGB) outperformed other algorithms.

Conclusion

The results demonstrate that the features investigated and machine learning models developed provide a potential for the dynamic nonintrusive measurement of cognitive workload.

Application

The approach presented can be developed for nonintrusive workload measurement in real-world human–machine applications, simulator-based training, and research.

中文翻译：

---

动态工作中操作员认知工作量分类的语音特征、工厂系统警报和操作员-系统交互的调查

客观的

研究语音特征、人机警报和操作员系统交互，以估计全尺寸真实模拟场景中的认知工作量。

背景

认知工作量的理论和模型对于人机系统的设计和评估至关重要。不幸的是，几乎没有可用于现实动态人机交互的非侵入式认知工作量测量。

方法

该研究是在先进核反应堆的全范围控制室研究模拟器中进行的。六名机组人员，每组由三名操作员组成，参与了 12 个场景。操作员每隔一分钟评估一次他们的工作量。机器学习算法经过训练，可根据机组人员沟通、操作员与系统交互和系统警报来估计操作员的工作量。

结果

利用语音和系统特征的随机森林 (RF) 在测试数据上实现了 67% 的准确度。仅利用语音特征，达到的准确率为 63%。最重要的语音特征是音调、幅度和发音率。当使用警报和操作员系统交互功能时，准确率达到了 61%。最重要的特征是警报数量和操作员与系统交互的数量。为每个算子训练的算法的准确率在 39% 到 98% 之间，平均为 72%。对于执行的大多数分析，RF 和极端梯度提升 (XGB) 优于其他算法。

结论

结果表明，所研究的特征和开发的机器学习模型为认知工作量的动态非侵入式测量提供了潜力。

应用

所提出的方法可以开发用于现实世界人机应用程序、基于模拟器的培训和研究中的非侵入式工作负载测量。