Abstract With the fastest growing popularity of gaming applications on android phone, analyzing emotion changes of steadfast android-gamers have become a study of utmost interest among most of the psychologists. Recently, some android games are producing negative impacts to the gamers; even in the worst cases the effect is becoming life-threatening too. Most of the existing research works are based on psychological view-point of exploring the impact (positive/negative) of playing android games for the child and adult age-group. However, the online recognition of emotional state changes of the android-gamers while playing video games may be relatively unexplored. To fill this void, the present study proposes a novel method of identifying the emotional state changes of android-gamers by decoding their brain signals and facial images simultaneously during playing video games. Besides above, the second novelty of the paper lies in designing a multimodal fusion method between brain signals and facial images for the said application. To address this challenge, the paper proposes a fused type-2 fuzzy deep neural network (FT2FDNN) which integrates the brain signal processing approach by a general type-2 fuzzy reasoning algorithm with the flavor of the image/video processing approach using a deep convolutional neural network. FT2FDNN uses multiple modalities to extract the similar information (here, emotional changes) simultaneously from the type-2 fuzzy and deep neural representations. The proposed fused type-2 fuzzy deep learning paradigm demonstrates promising results in classifying the emotional changes of gamers with high classification accuracy. Thus the proposed work explores a new era for future researchers.

中文翻译：

---

使用融合的 Type-2 模糊深度神经网络解码 Android 游戏玩家的情绪变化

摘要 随着安卓手机游戏应用的快速普及，分析稳定的安卓游戏玩家的情绪变化已成为大多数心理学家最感兴趣的研究。近期部分安卓游戏对游戏玩家产生负面影响；即使在最坏的情况下，这种影响也会危及生命。大多数现有的研究工作都是基于心理学的观点来探索玩安卓游戏对儿童和成人年龄组的影响（正面/负面）。然而，在线识别安卓游戏玩家在玩电子游戏时的情绪状态变化可能是相对未开发的。为了填补这个空白，本研究提出了一种通过在玩电子游戏期间同时解码他们的大脑信号和面部图像来识别机器人游戏玩家情绪状态变化的新方法。除此之外，本文的第二个新颖之处在于为上述应用设计了一种脑信号和面部图像之间的多模态融合方法。为了应对这一挑战，本文提出了一种融合的 2 类模糊深度神经网络 (FT2FDNN)，该网络通过通用的 2 类模糊推理算法将大脑信号处理方法与使用深度卷积的图像/视频处理方法的风味相结合。神经网络。FT2FDNN 使用多种模态从 2 类模糊和深度神经表征中同时提取相似信息（此处为情绪变化）。所提出的融合类型 2 模糊深度学习范式在以高分类精度对游戏玩家的情绪变化进行分类方面显示出有希望的结果。因此，拟议的工作为未来的研究人员探索了一个新时代。