Infrared-Thermal Imaging is a non-contact mechanism for psychophysiological research and application in Human–Computer Interaction (HCI). Real-time detection of the face and tracking the Regions of Interest (ROI) in the thermal video during HCI is challenging due to head motion artifacts. This paper proposes a three-stage HCI framework for computing the multivariate time-series thermal video sequences to recognize human emotion and provides distraction suggestions. The first stage comprises of face, eye, and nose detection using a Faster R-CNN (region-based convolutional neural network) architecture and used Multiple Instance Learning (MIL) algorithm for tracking the face ROIs across the thermal video. The mean intensity of ROIs is calculated which forms a multivariate time series (MTS) data. In the second stage, the smoothed MTS data are passed to the Dynamic Time Warping (DTW) algorithm to classify emotional states elicited by video stimulus. During HCI, our proposed framework provides relevant suggestions from a psychological and physical distraction perspective in the third stage. Our proposed approach signifies better accuracy in comparison with other classification methods and thermal data-sets.

红外热成像是一种非接触式机制，用于人机交互 (HCI) 中的心理生理学研究和应用。由于头部运动伪影，在 HCI 期间实时检测面部并跟踪热视频中的感兴趣区域 (ROI) 具有挑战性。本文提出了一个三阶段 HCI 框架，用于计算多元时间序列热视频序列以识别人类情感并提供分散注意力的建议。第一阶段包括使用 Faster R-CNN（基于区域的卷积神经网络）架构的面部、眼睛和鼻子检测，并使用多实例学习 (MIL) 算法跟踪热视频中的面部 ROI。计算 ROI 的平均强度，形成多元时间序列 (MTS) 数据。在第二阶段，平滑的 MTS 数据被传递到动态时间扭曲 (DTW) 算法，以对视频刺激引起的情绪状态进行分类。在 HCI 期间，我们提出的框架在第三阶段从心理和身体分心的角度提供相关建议。与其他分类方法和热数据集相比，我们提出的方法意味着更高的准确性。