This article investigates the possibility to estimate the perceived Quality of Experience (QoE) automatically and unobtrusively by analyzing the face of the consumer of video streaming services, from which facial expression and gaze direction are extracted. If effective, this would be a valuable tool for the monitoring of personal QoE during video streaming services without asking the user to provide feedback, with great advantages for service management. Additionally, this would eliminate the bias of subjective tests and would avoid bothering the viewers with questions to collect opinions and feedback. The performed analysis relies on two different experiments: i) a crowdsourcing test, where the videos are subject to impairments caused by long initial delays and re-buffering events; ii) a laboratory test, where the videos are affected by blurring effects. The facial Action Units (AU) that represent the contractions of specific facial muscles together with the position of the eyes’ pupils are extracted to identify the correlation between perceived quality and facial expressions. An SVM with a quadratic kernel and a k-NN classifier have been tested to predict the QoE from these features. These have also been combined with measured application-level parameters to improve the quality prediction. From the performed experiments, it results that the best performance is obtained with the k-NN classifier by combining all the described features and after training it with both the datasets, with a prediction accuracy as high as 93.9% outperforming the state of the art achievements.

中文翻译：

---

从面部表情和注视方向估计视频流期间的体验质量

本文通过分析视频流服务消费者的面部，从中提取面部表情和凝视方向，研究了自动且不显眼地估计感知体验质量 (QoE) 的可能性。如果有效，这将是在视频流服务期间监控个人 QoE 的宝贵工具，而无需要求用户提供反馈，对服务管理具有很大优势。此外，这将消除主观测试的偏见，并避免用问题来打扰观众以收集意见和反馈。执行的分析依赖于两个不同的实验：i) 众包测试，其中视频会受到长时间初始延迟和重新缓冲事件造成的损害；ii) 实验室测试，视频受模糊效果影响的地方。提取代表特定面部肌肉收缩的面部动作单元 (AU) 以及眼睛瞳孔的位置，以识别感知质量与面部表情之间的相关性。已经测试了具有二次核和 k-NN 分类器的 SVM 以根据这些特征预测 QoE。这些还与测量的应用级参数相结合，以改进质量预测。从所进行的实验来看，k-NN 分类器通过结合所有描述的特征并在用两个数据集对其进行训练后获得了最佳性能，预测准确率高达 93.9%，优于最先进的成果. 提取代表特定面部肌肉收缩的面部动作单元 (AU) 以及眼睛瞳孔的位置，以识别感知质量与面部表情之间的相关性。已经测试了具有二次核和 k-NN 分类器的 SVM 以根据这些特征预测 QoE。这些还与测量的应用级参数相结合，以改进质量预测。从所进行的实验来看，k-NN 分类器通过结合所有描述的特征并在用两个数据集对其进行训练后获得了最佳性能，预测准确率高达 93.9%，优于最先进的成果. 提取代表特定面部肌肉收缩的面部动作单元 (AU) 以及眼睛瞳孔的位置，以识别感知质量与面部表情之间的相关性。已经测试了具有二次核和 k-NN 分类器的 SVM 以根据这些特征预测 QoE。这些还与测量的应用级参数相结合，以改进质量预测。从所进行的实验来看，k-NN 分类器通过结合所有描述的特征并在用两个数据集对其进行训练后获得了最佳性能，预测准确率高达 93.9%，优于最先进的成果.