当前位置: X-MOL 学术IEEE Trans. Pattern Anal. Mach. Intell. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Interactive Multi-Dimension Modulation for Image Restoration.
IEEE Transactions on Pattern Analysis and Machine Intelligence ( IF 23.6 ) Pub Date : 2022-11-07 , DOI: 10.1109/tpami.2021.3129345
Jingwen He , Chao Dong , Liu Yihao , Yu Qiao

Interactive image restoration aims to generate restored images by adjusting a controlling coefficient which determines the restoration level. Previous works are restricted in modulating image with a single coefficient. However, real images always contain multiple types of degradation, which cannot be well determined by one coefficient. To make a step forward, this paper presents a new problem setup, called multi-dimension (MD) modulation, which aims at modulating output effects across multiple degradation types and levels. Compared with the previous single-dimension (SD) modulation, the MD setup to handle multiple degradations adaptively and relief data unbalancing problem in different degradation types. We also propose a deep architecture - CResMD with newly introduced controllable residual connections for multi-dimension modulation. Specifically, we add a controlling variable on the conventional residual connection to allow a weighted summation of input and residual. The values of these weights are generated by another condition network. We further propose a new data sampling strategy based on beta distribution together with a simple loss reweighting approach to balance different degradation types and levels. With corrupted image and degradation information as inputs, the network can output the corresponding restored image. By tweaking the condition vector, users can control the output effects in MD space at test time. Moreover, we also provide an estimation network to predict the condition vector, thus the base network could directly output the restored image without modulation from users. Extensive experiments demonstrate that the proposed CResMD achieves excellent performance on both SD and MD modulation tasks.

中文翻译:

用于图像恢复的交互式多维调制。

交互式图像恢复旨在通过调整确定恢复级别的控制系数来生成恢复的图像。以前的工作仅限于使用单个系数调制图像。然而,真实图像总是包含多种类型的退化,这些退化不能由一个系数很好地确定。为了向前迈出一步,本文提出了一种新的问题设置,称为多维 (MD) 调制,旨在跨多个退化类型和级别调制输出效果。与之前的单维 (SD) 调制相比,MD 设置可以自适应地处理多个退化并缓解不同退化类型中的数据不平衡问题。我们还提出了一种深度架构——CResMD,它具有新引入的用于多维调制的可控残差连接。具体来说,我们在传统的残差连接上添加了一个控制变量,以允许输入和残差的加权求和。这些权重的值由另一个条件网络生成。我们进一步提出了一种基于 beta 分布的新数据采样策略以及一种简单的损失重新加权方法来平衡不同的退化类型和水平。以损坏的图像和退化信息作为输入,网络可以输出相应的恢复图像。通过调整条件向量,用户可以在测试时控制 MD 空间中的输出效果。此外,我们还提供了一个估计网络来预测条件向量,因此基础网络可以直接输出恢复的图像,而无需用户调制。
更新日期:2021-11-19
down
wechat
bug