当前位置: X-MOL 学术J. Multimodal User Interfaces › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces
Journal on Multimodal User Interfaces ( IF 2.2 ) Pub Date : 2020-06-02 , DOI: 10.1007/s12193-020-00325-z
Ruixue Liu , Erin Walker , Leah Friedman , Catherine M. Arrington , Erin T. Solovey

Automatic detection of an individual’s mind-wandering state has implications for designing and evaluating engaging and effective learning interfaces. While it is difficult to differentiate whether an individual is mind-wandering or focusing on the task only based on externally observable behavior, brain-based sensing offers unique insights to internal states. To explore the feasibility, we conducted a study using functional near-infrared spectroscopy (fNIRS) and investigated machine learning classifiers to detect mind-wandering episodes based on fNIRS data, both on an individual level and a group level, specifically focusing on automated window selection to improve classification results. For individual-level classification, by using a moving window method combined with a linear discriminant classifier, we found the best windows for classification and achieved a mean F1-score of 74.8%. For group-level classification, we proposed an individual-based time window selection (ITWS) algorithm to incorporate individual differences in window selection. The algorithm first finds the best window for each individual by using embedded individual-level classifiers and then uses these windows from all participants to build the final classifier. The performance of the ITWS algorithm is evaluated when used with eXtreme gradient boosting, convolutional neural networks, and deep neural networks. Our results show that the proposed algorithm achieved significant improvement compared to the previous state of the art in terms of brain-based classification of mind-wandering, with an average F1-score of 73.2%. This builds a foundation for mind-wandering detection for both the evaluation of multimodal learning interfaces and for future attention-aware systems.



中文翻译:

基于fNIRS的流浪分类和多模式学习界面的个性化窗口选择

自动检测一个人的思维游荡状态对设计和评估引人入胜的有效学习界面具有影响。虽然很难根据外部可观察到的行为来区分一个人是在游荡还是专注于任务,但基于大脑的感知却可以为内部状态提供独特的见解。为了探索可行性,我们使用功能近红外光谱(fNIRS)进行了一项研究,并研究了基于fNIRS数据的机器学习分类器,以检测在个人级别和小组级别上的心理徘徊事件,特别是针对自动窗口选择改善分类结果。对于个人级别的分类,通过结合使用移动窗口方法和线性判别式分类器,我们找到了最佳的分类窗口,平均F1得分为74.8%。对于组级别分类,我们提出了一种基于个体的时间窗口选择(ITWS)算法,以将个体差异纳入窗口选择中。该算法首先通过使用嵌入的个人级别分类器为每个个体找到最佳窗口,然后使用所有参与者的这些窗口来构建最终分类器。与eXtreme梯度增强,卷积神经网络和深度神经网络一起使用时,将评估ITWS算法的性能。我们的结果表明,与基于现有技术的现有技术相比,所提出的算法在基于大脑的心理游荡分类方面取得了显着改进,平均F1得分为73.2%。

更新日期:2020-06-02
down
wechat
bug