当前位置: X-MOL 学术ACM Trans. Graph. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Mononizing binocular videos
ACM Transactions on Graphics  ( IF 7.8 ) Pub Date : 2020-11-27 , DOI: 10.1145/3414685.3417764
Wenbo Hu 1 , Menghan Xia 1 , Chi-Wing Fu 1 , Tien-Tsin Wong 1
Affiliation  

This paper presents the idea of mono-nizing binocular videos and a framework to effectively realize it. Mono-nize means we purposely convert a binocular video into a regular monocular video with the stereo information implicitly encoded in a visual but nearly-imperceptible form. Hence, we can impartially distribute and show the mononized video as an ordinary monocular video. Unlike ordinary monocular videos, we can restore from it the original binocular video and show it on a stereoscopic display. To start, we formulate an encoding-and-decoding framework with the pyramidal deformable fusion module to exploit long-range correspondences between the left and right views, a quantization layer to suppress the restoring artifacts, and the compression noise simulation module to resist the compression noise introduced by modern video codecs. Our framework is self-supervised, as we articulate our objective function with loss terms defined on the input: a monocular term for creating the mononized video, an invertibility term for restoring the original video, and a temporal term for frame-to-frame coherence. Further, we conducted extensive experiments to evaluate our generated mononized videos and restored binocular videos for diverse types of images and 3D movies. Quantitative results on both standard metrics and user perception studies show the effectiveness of our method.

中文翻译:

单目化双目视频

本文提出的想法单一化双目视频和有效实现它的框架。Mono-nize 意味着我们故意将双目视频转换为常规单目视频,其中立体信息以视觉但几乎不可察觉的形式隐式编码。因此,我们可以将单声道化的视频作为普通的单目视频公正地分发和显示。与普通的单目视频不同,我们可以从中恢复原始的双目视频,并在立体显示器上显示。首先,我们用金字塔形可变形融合模块制定了一个编码和解码框架,以利用左右视图之间的远程对应,一个量化层来抑制恢复伪影,以及压缩噪声模拟模块来抵抗压缩现代视频编解码器引入的噪声。我们的框架是自我监督的,当我们用在输入上定义的损失项来表达我们的目标函数时:一个用于创建单声道视频的单目项,一个用于恢复原始视频的可逆项,以及一个用于帧到帧相干性的时间项。此外,我们进行了广泛的实验来评估我们为不同类型的图像和 3D 电影生成的单声道视频和恢复的双目视频。标准指标和用户感知研究的定量结果显示了我们方法的有效性。我们进行了广泛的实验来评估我们为不同类型的图像和 3D 电影生成的单目视频和恢复的双目视频。标准指标和用户感知研究的定量结果显示了我们方法的有效性。我们进行了广泛的实验来评估我们为不同类型的图像和 3D 电影生成的单目视频和恢复的双目视频。标准指标和用户感知研究的定量结果显示了我们方法的有效性。
更新日期:2020-11-27
down
wechat
bug