Temporal Action Detection is an important yet challenging task, in which temporal action proposal generation plays an important part. Since the temporal boundaries of action instances in videos are often ambiguous, it’s difficult to locate them precisely. Boundary Sensitive Network (BSN) (Lin et al. in ECCV, 2018) is a state-of-the-art corner-based method that can generate high-quality proposals with high recall rate. It contains a temporal evaluation network and a proposal evaluation network to generate and evaluate proposals separately, which can find the temporal boundaries of action instances directly to produce proposals with flexible temporal intervals and evaluate the quality of proposals. But BSN still has some issues: (1) Due to the small reception field of temporal evaluation network, it often generates many false temporal boundaries. (2) Evaluating the quality of proposals is a difficult task and not well solved in the paper. To address these issues, we propose Complementary Boundary Estimation Network (CBEN), an improved approach to temporal action proposal generation based on the framework of BSN. Specifically, we improve BSN in two aspects: Firstly, considering the temporal evaluation network of BSN can only capture local information and tends to have high response at background segments, we combine it with a new network with larger reception field to better identify false temporal action boundaries. Secondly, to evaluate the quality of temporal action proposals more accurately, we propose a class-based proposal evaluation network and combine it with a tIoU-based proposal evaluation network to filter out low-quality proposals. Extensive experiments on THUMOS14 and ActivityNet-1.3 datasets indicate that CBEN can achieve better performance than current mainstream methods on temporal action proposal generation. We further combine CBEN with an off-the-shelf action classifier, and show consistent performance improvements on THUMOS14 dataset.

时间行为检测是一项重要而又具有挑战性的任务，其中时间行为建议的生成起着重要的作用。由于视频中动作实例的时间边界通常是模糊的，因此很难精确定位它们。边界敏感网络（BSN）（Lin等人，在ECCV中，2018年）是一种基于角点的最新方法，可以生成具有高召回率的高质量建议。它包含一个时间评估网络和一个提案评估网络，以分别生成和评估提案，它们可以直接找到动作实例的时间边界，以生成具有灵活时间间隔的提案并评估提案的质量。但是BSN仍然存在一些问题：（1）由于时间评估网络的接收领域较小，它经常会产生许多错误的时间边界。（2）评价建议书的质量是一项艰巨的任务，本文没有很好地解决。为了解决这些问题，我们提出了互补边界估计网络（CBEN），这是一种基于BSN框架的时间行动建议生成的改进方法。具体来说，我们从两个方面改进了BSN：首先，考虑到BSN的时间评估网络只能捕获本地信息，并且往往在背景部分具有较高的响应速度，因此我们将其与具有较大接收范围的新网络相结合，以更好地识别错误的时间行为边界。其次，为了更准确地评估时间行动提案的质量，我们提出了一个基于类别的提案评估网络，并将其与基于tIoU的提案评估网络相结合，以过滤出低质量的提案。在THUMOS14和ActivityNet-1.3数据集上的大量实验表明，CBEN可以在时间行动建议生成方面比当前的主流方法获得更好的性能。我们进一步将CBEN与现成的动作分类器结合在一起，并在THUMOS14数据集上显示出一致的性能改进。