Abstract

Video has rapidly become one of the most common sources of visual information transfer. The number of videos uploaded to YouTube in a single day is estimated to take over 82 years to watch. Automated tools and techniques for analyzing and understanding video content, thus, have become an essential requirement. This paper addresses the problem of video highlight generation for large video files. We propose a novel skimming-based unsupervised video highlight generation method utilizing statistical image processing and data clustering, which process frame-level static and dynamic features of input video in two streams. The dynamic feature stream is represented by computing a dense optical flow for each consecutive frame, providing instantaneous velocity information for every pixel, which is then characterized by a per-frame orientation histogram, weighted by the norm, with orientations quantized. To process multi-scene videos, we utilize the divisive hierarchical clustering capability of growing self-organizing map (GSOM) using a dual-step top-down hierarchical approach in which the first level consists of clustering of spatial and temporal features of the video and in the second level, each parent cluster is hierarchically subdivided into child clusters using GSOM. The video highlight generation process is conducted real time by evaluating segments of video snippets based on a pre-defined time interval. We demonstrate the accuracy, robustness and the quality of highlights generated using a qualitative analysis conducted using 1625 human experts on highlights generated from two datasets. Further, we conduct a runtime analysis to demonstrate the efficient processing capability of the proposed method, to be used in real-time settings.

中文翻译：

---

利用双流分层增长的自组织地图实时自动生成视频集锦

摘要

视频已迅速成为视觉信息传输的最常见来源之一。据估计，一天之内上传到YouTube的视频数量超过82年。因此，用于分析和理解视频内容的自动化工具和技术已经成为基本要求。本文解决了大型视频文件的视频精彩片段生成问题。我们提出了一种利用统计图像处理和数据聚类的基于撇号的新型无监督视频亮点生成方法，该方法可在两个流中处理输入视频的帧级静态和动态特征。动态特征流通过为每个连续帧计算密集的光流来表示，为每个像素提供瞬时速度信息，然后以每帧方向直方图为特征，由规范加权，方向量化。为了处理多场景视频，我们利用自上而下的双步分层方法，利用不断增长的自组织地图（GSOM）的分割分层聚类能力，其中第一级包括视频和视频的时空特征聚类。在第二级中，使用GSOM将每个父群集细分为子群集。通过基于预定义的时间间隔评估视频片段的片段，实时进行视频精彩片段生成过程。我们展示了使用1625位人类专家对从两个数据集生成的高光进行定性分析所生成的高光的准确性，鲁棒性和质量。进一步，