Procedural knowledge is crucial for understanding and performing concrete real-world tasks. Yet, despite the importance of procedural knowledge, research into procedural knowledge understanding is still under-developed. In particular, videos contain rich semantics that are important for understanding procedural knowledge, but have traditionally been less explored than natural language texts for understanding procedural knowledge. Motivated by harnessing procedural knowledge from videos for task assistance (i.e., assisting people in performing procedural tasks), we present the first comprehensive survey of procedural video datasets. Through systematically surveying 23 procedural video datasets, including both instructional and non-instructional videos, in a conceptual framework for task assistance, we seek to understand the trends and gaps in existing datasets, as well as to gain insights into the future of such datasets. This survey examines the current state of procedural video datasets, in terms of their data, content and annotation characteristics, as well as processing function and evaluation. The survey also identifies and suggests a number of possible directions to bring this area to the next level.

程序知识对于理解和执行具体的实际任务至关重要。然而，尽管程序知识很重要，但对程序知识理解的研究仍不完善。尤其是，视频包含丰富的语义，这对于理解过程知识很重要，但是传统上，与自然语言文本相比，视频对于理解过程知识的探索较少。通过利用视频中的过程知识来进行任务辅助（例如，协助人们执行过程任务），我们提出了对过程视频数据集的首次全面调查。通过在任务协助的概念框架中系统地调查23个程序视频数据集，包括教学视频和非教学视频，我们力求了解现有数据集的趋势和差距，以及对此类数据集的未来的见解。这项调查从过程视频数据集的数据，内容和注释特征以及处理功能和评估的角度检查了它们的当前状态。该调查还确定并提出了许多可能的方向，以使这一领域更上一层楼。