We exploit a self-supervised deep multi-task learning framework for electrocardiogram (ECG) -based emotion recognition. The proposed solution consists of two stages of learning a) learning ECG representations and b) learning to classify emotions. ECG representations are learned by a signal transformation recognition network. The network learns high-level abstract representations from unlabeled ECG data. Six different signal transformations are applied to the ECG signals, and transformation recognition is performed as pretext tasks. Training the model on pretext tasks helps the network learn spatiotemporal representations that generalize well across different datasets and different emotion categories. We transfer the weights of the self-supervised network to an emotion recognition network, where the convolutional layers are kept frozen and the dense layers are trained with labelled ECG data. We show that the proposed solution considerably improves the performance compared to a network trained using fully-supervised learning. New state-of-the-art results are set in classification of arousal, valence, affective states, and stress for the four utilized datasets. Extensive experiments are performed, providing interesting insights into the impact of using a multi-task self-supervised structure instead of a single-task model, as well as the optimum level of difficulty required for the pretext self-supervised tasks.

中文翻译：

---

用于情绪识别的自监督心电图表征学习

我们利用自我监督的深度多任务学习框架来进行基于心电图 (ECG) 的情绪识别。提出的解决方案包括两个学习阶段a) 学习心电图表征和b) 学习对情绪进行分类。ECG 表示由信号转换识别网络学习。该网络从未标记的 ECG 数据中学习高级抽象表示。六种不同的信号变换应用于心电图信号，变换识别作为借口任务执行。在借口任务上训练模型有助于网络学习时空表示，这些表示可以很好地泛化不同的数据集和不同的情感类别。我们将自监督网络的权重转移到情绪识别网络，其中卷积层保持冻结状态，密集层使用标记的 ECG 数据进行训练。我们表明，与使用全监督学习训练的网络相比，所提出的解决方案显着提高了性能。在四个使用的数据集的唤醒、效价、情感状态和压力分类中设置了最新的最新结果。进行了广泛的实验，为使用多任务自监督结构而不是单任务模型的影响以及借口自监督任务所需的最佳难度水平提供了有趣的见解。