The accurate detection and tracking of pupil is important to many applications such as human–computer interaction, driver’s fatigue detection and diagnosis of brain diseases. Existing approaches however face challenges in handing low quality of pupil images. In this paper, we propose an integrated pupil tracking framework, namely LVCF, based on deep learning. LVCF consists of the pupil detection model VCF which is an end-to-end network, and the LSTM pupil motion prediction model which applies LSTM to track pupil’s position. The proposed network was trained and evaluated on 10600 images and 75 videos taken from 3 realistic datasets. Within an error threshold of 5 pixels, VCF achieves an accuracy of more than 81%, and LVCF outperforms the state of arts by 9% in terms of percentage of pupils tracked. The project of LCVF is available at https://github.com/UnderTheMangoTree/LVCF.

瞳孔的准确检测和跟踪对于人机交互、驾驶员疲劳检测和脑部疾病诊断等许多应用具有重要意义。然而，现有方法在处理低质量的瞳孔图像方面面临挑战。在本文中，我们提出了一个基于深度学习的集成瞳孔跟踪框架，即 LVCF。LVCF 由端到端网络的瞳孔检测模型 VCF 和应用 LSTM 跟踪瞳孔位置的 LSTM 瞳孔运动预测模型组成。提议的网络在 10600 张图像和 75 个视频上进行了训练和评估，这些视频来自 3 个真实数据集。在 5 个像素的误差阈值内，VCF 实现了超过 81% 的准确度，并且 LVCF 在跟踪的瞳孔百分比方面比现有技术高出 9%。LCVF 的项目可在 https://github 获得。