Video frame interpolation is a challenging problem that involves various scenarios depending on the variety of foreground and background motions, frame rate, and occlusion. Therefore, generalizing across different scenes is difficult for a single network with fixed parameters. Ideally, one could have a different network for each scenario, but this will be computationally infeasible for practical applications. In this work, we propose MetaVFI, an adaptive video frame interpolation algorithm that uses additional information readily available at test time but has not been exploited in previous works. We initially show the benefits of test-time adaptation through simple fine-tuning of a network and then greatly improve its efficiency by incorporating meta-learning. Thus, we obtain significant performance gains with only a single gradient update without introducing any additional parameters. Moreover, the proposed MetaVFI algorithm is model-agnostic which can be easily combined with any video frame interpolation network. We show that our adaptive framework greatly improves the performance of baseline video frame interpolation networks on multiple benchmark datasets.

中文翻译：

---

通过元学习进行视频帧插值的测试时间适应


视频帧插值是一个具有挑战性的问题，涉及各种场景，具体取决于前景和背景运动、帧速率和遮挡的变化。因此，对于具有固定参数的单个网络来说，跨不同场景进行泛化是很困难的。理想情况下，每个场景都可以有不同的网络，但这对于实际应用来说在计算上是不可行的。在这项工作中，我们提出了 MetaVFI，一种自适应视频帧插值算法，该算法使用在测试时容易获得但在之前的工作中尚未利用的附加信息。我们最初通过简单的网络微调来展示测试时间适应的好处，然后通过结合元学习大大提高其效率。因此，我们仅通过一次梯度更新即可获得显着的性能提升，而无需引入任何额外的参数。此外，所提出的 MetaVFI 算法与模型无关，可以轻松与任何视频帧插值网络相结合。我们表明，我们的自适应框架极大地提高了多个基准数据集上的基线视频帧插值网络的性能。