当前位置: X-MOL 学术IEEE Trans. Vis. Comput. Graph. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
DGaze: CNN-Based Gaze Prediction in Dynamic Scenes.
IEEE Transactions on Visualization and Computer Graphics ( IF 5.2 ) Pub Date : 2020-02-13 , DOI: 10.1109/tvcg.2020.2973473
Zhiming Hu , Sheng Li , Congyi Zhang , Kangrui Yi , Guoping Wang , Dinesh Manocha

We conduct novel analyses of users' gaze behaviors in dynamic virtual scenes and, based on our analyses, we present a novel CNN-based model called DGaze for gaze prediction in HMD-based applications. We first collect 43 users' eye tracking data in 5 dynamic scenes under free-viewing conditions. Next, we perform statistical analysis of our data and observe that dynamic object positions, head rotation velocities, and salient regions are correlated with users' gaze positions. Based on our analysis, we present a CNN-based model (DGaze) that combines object position sequence, head velocity sequence, and saliency features to predict users' gaze positions. Our model can be applied to predict not only realtime gaze positions but also gaze positions in the near future and can achieve better performance than prior method. In terms of realtime prediction, DGaze achieves a 22.0% improvement over prior method in dynamic scenes and obtains an improvement of 9.5% in static scenes, based on using the angular distance as the evaluation metric. We also propose a variant of our model called DGaze ET that can be used to predict future gaze positions with higher precision by combining accurate past gaze data gathered using an eye tracker. We further analyze our CNN architecture and verify the effectiveness of each component in our model. We apply DGaze to gaze-contingent rendering and a game, and also present the evaluation results from a user study.

中文翻译:

DGaze:动态场景中基于CNN的注视预测。

我们对动态虚拟场景中用户的注视行为进行了新颖的分析,并基于我们的分析,提出了一种基于CNN的新颖模型DGaze,用于基于HMD的应用中的注视预测。我们首先在自由观看条件下的5个动态场景中收集了43个用户的眼动跟踪数据。接下来,我们对数据进行统计分析,并观察到动态对象位置,头部旋转速度和显着区域与用户的注视位置相关。根据我们的分析,我们提出了一个基于CNN的模型(DGaze),该模型结合了对象位置序列,头部速度序列和显着性特征来预测用户的凝视位置。我们的模型不仅可以用于预测实时注视位置,而且可以在不久的将来预测注视位置,并且可以实现比现有方法更好的性能。在实时预测方面,基于角度距离作为评估指标,DGaze在动态场景中比以前的方法提高了22.0%,在静态场景中则提高了9.5%。我们还提出了一种称为DGaze ET的模型变体,该模型可通过结合使用眼动仪收集的准确的过去凝视数据来以更高的精度预测未来的凝视位置。我们将进一步分析CNN架构,并验证模型中每个组件的有效性。我们将DGaze应用到视线渲染和游戏中,并提供用户研究的评估结果。我们还提出了一种称为DGaze ET的模型变体,该模型可通过结合使用眼动仪收集的准确的过去注视数据来以更高的精度预测未来的注视位置。我们将进一步分析CNN架构,并验证模型中每个组件的有效性。我们将DGaze应用到视线渲染和游戏中,并提供用户研究的评估结果。我们还提出了一种称为DGaze ET的模型变体,该模型可通过结合使用眼动仪收集的准确的过去注视数据来以更高的精度预测未来的注视位置。我们将进一步分析CNN架构,并验证模型中每个组件的有效性。我们将DGaze应用到视线渲染和游戏中,并提供用户研究的评估结果。
更新日期:2020-04-22
down
wechat
bug