当前位置: X-MOL 学术Vis. Comput. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Feature-attention module for context-aware image-to-image translation
The Visual Computer ( IF 3.0 ) Pub Date : 2020-09-23 , DOI: 10.1007/s00371-020-01943-0
Jing Bai , Ran Chen , Min Liu

In a summer2winter image-to-image translation, trees should be transformed from green to gray, but the colors of houses or girls should not be changed. However, current unsupervised one-to-one image translation techniques failed to focus the translation on individual objects. To tackle this issue, we propose a novel feature-attention module for capturing the mutual influences of various features, so as to automatically attend only to specific scene objects in unsupervised image-to-image translation. The proposed module can be integrated into different image translation networks and improve their context-aware translation ability. The qualitative and quantitative experiments on horse2zebra, apple2orange and summer2winter datasets based on DualGAN, CycleGAN and UNIT demonstrate a significant improvement in our proposed module over the state-of-the-art methods. In addition, the experiments on apple2orange dataset based on MUNIT and DRIT further indicate the effectiveness of FA module in multimodal translation tasks. We also show that the computation complexity of the proposed module is linear to the image size; moreover, the experiments on the day2night dataset prove that the proposed module is insensitive to the growth of image resolution. The source code and trained models are available at https://github.com/gaoyuainshuyi/fa .

中文翻译:

用于上下文感知图像到图像转换的特征注意模块

在 Summer2winter 图像到图像的转换中,树木应从绿色变为灰色,但不应更改房屋或女孩的颜色。然而,当前的无监督一对一图像翻译技术未能将翻译重点放在单个对象上。为了解决这个问题,我们提出了一种新颖的特征注意模块,用于捕捉各种特征的相互影响,以便在无监督的图像到图像转换中自动只关注特定的场景对象。所提出的模块可以集成到不同的图像翻译网络中,并提高它们的上下文感知翻译能力。基于DualGAN的horse2zebra、apple2orange和summer2winter数据集的定性和定量实验,CycleGAN 和 UNIT 证明我们提出的模块比最先进的方法有了显着的改进。此外,在基于 MUNIT 和 DRIT 的 apple2orange 数据集上的实验进一步表明了 FA 模块在多模态翻译任务中的有效性。我们还表明,所提出模块的计算复杂度与图像大小呈线性关系;此外,在 day2night 数据集上的实验证明,所提出的模块对图像分辨率的增长不敏感。源代码和训练模型可在 https://github.com/gaoyuainshuyi/fa 获得。此外,在 day2night 数据集上的实验证明,所提出的模块对图像分辨率的增长不敏感。源代码和训练模型可在 https://github.com/gaoyuainshuyi/fa 获得。此外,在 day2night 数据集上的实验证明,所提出的模块对图像分辨率的增长不敏感。源代码和训练模型可在 https://github.com/gaoyuainshuyi/fa 获得。
更新日期:2020-09-23
down
wechat
bug