当前位置: X-MOL 学术IEEE Access › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Video Frame Interpolation Based on Symmetric and Asymmetric Motions
IEEE Access ( IF 3.9 ) Pub Date : 2023-03-06 , DOI: 10.1109/access.2023.3252911
Whan Choi, Yeong Jun Koh, Chang-Su Kim

Video frame interpolation is the task to synthesize intermediate frames between consecutive frames to increase the frame rate. Recently, various deep-learning techniques have been proposed to interpolate intermediate frames more reliably. However, many existing methods use either symmetric (linear) or asymmetric (non-linear) schemes only to estimate motions for the warping process, resulting in unreliable interpolation results. In this paper, we propose a novel video frame interpolation network based on both symmetric and asymmetric motion-based warping modules, which can deal with linear and non-linear motions, as well as occlusions, effectively. The symmetric warping module estimates symmetric motions to generate intermediate frames, while the asymmetric one predicts asymmetric motions to address non-linear motions and occlusion problems. We combine symmetric and asymmetric warping results to reconstruct intermediate frames more reliably. We also develop the frame synthesis network to refine the combined warping results. Experimental results demonstrate that the proposed network outperforms state-of-the-art video interpolation algorithms and that the two types of warping modules work effectively in a complementary manner on various benchmark datasets.

中文翻译:

基于对称和非对称运动的视频帧插值

视频帧插值是在连续帧之间合成中间帧以提高帧速率的任务。最近,已经提出了各种深度学习技术来更可靠地插入中间帧。然而,许多现有方法仅使用对称(线性)或非对称(非线性)方案来估计翘曲过程的运动,从而导致不可靠的插值结果。在本文中,我们提出了一种基于对称和非对称运动变形模块的新型视频帧插值网络,可以有效地处理线性和非线性运动以及遮挡。对称变形模块估计对称运动以生成中间帧,而不对称变形模块预测不对称运动以解决非线性运动和遮挡问题。我们结合对称和非对称变形结果来更可靠地重建中间帧。我们还开发了帧合成网络来改进组合变形结果。实验结果表明,所提出的网络优于最先进的视频插值算法,并且两种类型的变形模块在各种基准数据集上以互补的方式有效地工作。
更新日期:2023-03-10
down
wechat
bug