Non-local patch-based methods were until recently the state of the art for image denoising but are now outperformed by CNNs. In video denoising, however, they are still competitive with CNNs, as they can effectively exploit the video temporal redundancy, which is a key factor to attain high denoising performance. The problem is that CNN architectures are not compatible with the search for self-similarities. In this work, we propose a simple, yet efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed method to image and video denoising. In the case of video, the patches are searched for in a 3D spatiotemporal volume. The proposed method achieves state-of-the-art results.

直到最近，基于非局部补丁的方法仍是图像去噪的最新技术，但现在已被CNN超越。但是，在视频降噪中，它们仍然可以与CNN竞争，因为它们可以有效利用视频时间冗余，这是获得高降噪性能的关键因素。问题在于CNN架构与自相似性搜索不兼容。在这项工作中，我们提出了一种简单而有效的方式来将视频自相似性馈入CNN。经由第一不可训练层将非局部性并入网络中，该第一不可训练层针对输入图像中的每个补丁在搜索区域中找到其最相似的补丁。然后，将这些色块的中心值收集到分配给每个图像像素的特征向量中。该信息被提供给CNN，该CNN经过训练可以预测干净的图像。我们将提出的方法应用于图像和视频去噪。在视频的情况下，以3D时空体积搜索补丁。所提出的方法获得了最新的结果。