当前位置: X-MOL 学术Signal Process. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Detail-enhanced image inpainting based on discrete wavelet transforms
Signal Processing ( IF 4.4 ) Pub Date : 2021-08-04 , DOI: 10.1016/j.sigpro.2021.108278
Bin Li 1, 2 , Bowei Zheng 1 , Haodong Li 1 , Yanran Li 2
Affiliation  

Deep-learning-based method has made great breakthroughs in image inpainting by generating visually plausible contents with reasonable semantic meaning. However, existing deep learning methods still suffer from distorted structures or blurry textures. To mitigate this problem, completing semantic structure and enhancing textural details should be considered simultaneously. To this end, we propose a two-parallel-branch completion network, where the first branch fills semantic content in spatial domain, and the second branch helps to generate high-frequency details in wavelet domain. To reconstruct an inpainted image, the output of the first branch is also decomposed by discrete wavelet transform, and the resulting low-frequency wavelet subband is used jointly with the output of the second branch. In addition, for improving the network capability in semantic understanding, a multi-level fusion module (MLFM) is designed in the first branch to enlarge the receptive field. Furthermore, drawing lessons from some traditional exemplar-based inpainting methods, we develop a free-form spatially discounted mask (SD-mask) to assign different importance priorities for the missing pixels based on their positions, enabling our method to handle missing regions with arbitrary shapes. Extensive experiments on several public datasets demonstrate that the proposed approach outperforms current state-of-the-art ones. The codes are public available at https://github.com/media-sec-lab/DWT_Inpainting.



中文翻译:

基于离散小波变换的细节增强图像修复

基于深度学习的方法通过生成具有合理语义的视觉上似是而非的内容,在图像修复方面取得了重大突破。然而,现有的深度学习方法仍然存在结构扭曲或纹理模糊的问题。为了缓解这个问题,应该同时考虑完成语义结构和增强纹理细节。为此,我们提出了一个双并行分支完成网络,其中第一个分支填充空间域中的语义内容,第二个分支帮助生成小波域中的高频细节。为了重建修复图像,第一个分支的输出也被离散小波变换分解,产生的低频小波子带与第二个分支的输出一起使用。此外,为了提高网络在语义理解方面的能力,在第一分支中设计了多级融合模块(MLFM)以扩大感受野。此外,从一些传统的基于样本的修复方法中吸取教训,我们开发了一种自由形式的空间折扣掩码(SD-掩码),根据它们的位置为丢失的像素分配不同的重要性优先级,使我们的方法能够处理任意丢失的区域形状。对几个公共数据集的大量实验表明,所提出的方法优于当前最先进的方法。这些代码可在 https://github.com/media-sec-lab/DWT_Inpainting 上公开获得。从一些传统的基于样本的修复方法中汲取教训,我们开发了一种自由形式的空间折扣掩码 (SD-mask),根据它们的位置为缺失像素分配不同的重要性优先级,使我们的方法能够处理具有任意形状的缺失区域。对几个公共数据集的大量实验表明,所提出的方法优于当前最先进的方法。这些代码可在 https://github.com/media-sec-lab/DWT_Inpainting 上公开获得。从一些传统的基于样本的修复方法中汲取教训,我们开发了一种自由形式的空间折扣掩码 (SD-mask),根据它们的位置为缺失像素分配不同的重要性优先级,使我们的方法能够处理具有任意形状的缺失区域。对几个公共数据集的大量实验表明,所提出的方法优于当前最先进的方法。这些代码可在 https://github.com/media-sec-lab/DWT_Inpainting 上公开获得。对几个公共数据集的大量实验表明,所提出的方法优于当前最先进的方法。这些代码可在 https://github.com/media-sec-lab/DWT_Inpainting 上公开获得。对几个公共数据集的大量实验表明,所提出的方法优于当前最先进的方法。这些代码可在 https://github.com/media-sec-lab/DWT_Inpainting 上公开获得。

更新日期:2021-08-09
down
wechat
bug