The objective of this work is automatic labelling of characters in TV video and movies, given weak supervisory information provided by an aligned transcript. We make five contributions: (i) a new strategy for obtaining stronger supervisory information from aligned transcripts; (ii) an explicit model for classifying background characters, based on their face-tracks; (iii) employing new ConvNet based face features, and (iv) a novel approach for labelling all face tracks jointly using linear programming. Each of these contributions delivers a boost in performance, and we demonstrate this on standard benchmarks using tracks provided by authors of prior work. As a fifth contribution, we also investigate the generalisation and strength of the features and classifiers by applying them “in the raw” on new video material where no supervisory information is used. In particular, to provide high quality tracks on those material, we propose efficient track classifiers to remove false positive tracks by the face tracker. Overall we achieve a dramatic improvement over the state of the art on both TV series and film datasets, and almost saturate performance on some benchmarks.

中文翻译：

---

电影和电视材料的自动视频面部标签。

这项工作的目标是，在对齐的笔录提供的监管信息薄弱的情况下，自动标记电视视频和电影中的字符。我们做出了五点贡献：（i）一种新策略，可从对齐的笔录中获得更强大的监管信息；（ii）根据背景角色的面部表情对背景角色进行分类的显式模型；（iii）采用新的基于ConvNet的面部特征，以及（iv）一种使用线性编程共同标记所有面部轨迹的新颖方法。所有这些贡献都可以提高性能，我们将使用以前工作的作者提供的方法在标准基准上对此进行演示。作为第五项贡献，我们还通过将特征和分类器“原始”应用到不使用任何监督信息的新视频资料中，来研究特征和分类器的概括性和强度。特别是，为了在这些材料上提供高质量的轨道，我们提出了有效的轨道分类器，以通过面部跟踪器消除误报。总体而言，我们在电视连续剧和电影数据集方面都比现有技术有了显着改善，并且在某些基准上的表现几乎达到饱和。