Video frame interpolation is a challenging problem that involves various scenarios depending on the variety of foreground and background motions, frame rate, and occlusion. Therefore, generalizing across different scenes is difficult for a single network with fixed parameters. Ideally, one could have a different network for each scenario, but this will be computationally infeasible for practical applications. In this work, we propose MetaVFI, an adaptive video frame interpolation algorithm that uses additional information readily available at test time but has not been exploited in previous works. We initially show the benefits of test-time adaptation through simple fine-tuning of a network and then greatly improve its efficiency by incorporating meta-learning. Thus, we obtain significant performance gains with only a single gradient update without introducing any additional parameters. Moreover, the proposed MetaVFI algorithm is model-agnostic which can be easily combined with any video frame interpolation network. We show that our adaptive framework greatly improves the performance of baseline video frame interpolation networks on multiple benchmark datasets.

中文翻译：

---

通过元学习对视频帧插值进行测试时间自适应。

视频帧插值是一个具有挑战性的问题，它涉及各种场景，具体取决于各种前景和背景运动、帧速率和遮挡。因此，对于具有固定参数的单个网络，跨不同场景的泛化是困难的。理想情况下，每个场景都可以有不同的网络，但这对于实际应用来说在计算上是不可行的。在这项工作中，我们提出了 MetaVFI，这是一种自适应视频帧插值算法，它使用在测试时容易获得的附加信息，但在以前的工作中没有被利用。我们最初通过简单的网络微调展示了测试时间适应的好处，然后通过结合元学习大大提高了它的效率。因此，我们仅通过一次梯度更新而无需引入任何额外参数即可获得显着的性能提升。此外，所提出的 MetaVFI 算法与模型无关，可以很容易地与任何视频帧插值网络相结合。我们展示了我们的自适应框架极大地提高了基线视频帧插值网络在多个基准数据集上的性能。