Existing neural networks proposed for low-level image processing tasks are usually implemented by stacking convolution layers with limited kernel size. Every convolution layer merely involves in context information from a small local neighborhood. More contextual features can be explored as more convolution layers are adopted. However it is difficult and costly to take full advantage of long-range dependencies. We propose a novel non-local module, Pyramid Non-local Block, to build up connection between every pixel and all remain pixels. The proposed module is capable of efficiently exploiting pairwise dependencies between different scales of low-level structures. The target is fulfilled through first learning a query feature map with full resolution and a pyramid of reference feature maps with downscaled resolutions. Then correlations with multi-scale reference features are exploited for enhancing pixel-level feature representation. The calculation procedure is economical considering memory consumption and computational cost. Based on the proposed module, we devise a Pyramid Non-local Enhanced Networks for edge-preserving image smoothing which achieves state-of-the-art performance in imitating three classical image smoothing algorithms. Additionally, the pyramid non-local block can be directly incorporated into convolution neural networks for other image restoration tasks. We integrate it into two existing methods for image denoising and single image super-resolution, achieving consistently improved performance.

中文翻译：

---

PNEN：金字塔非本地增强网络。


现有的用于低级图像处理任务的神经网络通常是通过堆叠内核大小有限的卷积层来实现的。每个卷积层仅涉及来自小型局部邻域的上下文信息。随着更多卷积层的采用，可以探索更多上下文特征。然而，充分利用远程依赖性是困难且昂贵的。我们提出了一个新颖的非本地模块，金字塔非局部块，在每个像素和所有剩余像素之间建立连接。所提出的模块能够有效地利用不同规模的低级结构之间的成对依赖关系。通过首先学习具有全分辨率的查询特征图和具有缩小分辨率的参考特征图金字塔来实现该目标。然后利用与多尺度参考特征的相关性来增强像素级特征表示。考虑到内存消耗和计算成本，计算过程是经济的。基于所提出的模块，我们设计了一个金字塔非局部用于边缘保留图像平滑的增强网络，在模仿三种经典图像平滑算法方面实现了最先进的性能。此外，金字塔非局部块可以直接合并到卷积神经网络中用于其他图像恢复任务。我们将其集成到两种现有的图像去噪和单图像超分辨率方法中，实现了性能的持续改进。