Data-driven rain streak removal methods, most of which rely on synthesized paired data, usually come across the generalization problem when being applied in real scenarios. In this paper, we propose a novel deep-learning based rain streak removal method injected with self-supervision to obtain the capacity of removing more varied-scale rain streaks in practical applications. To this end, in this work, efforts are made from two perspectives. First, considering that rain streak removal is highly correlated with texture characteristics, we create a fractal band learning (FBL) network based on frequency band recovery. It integrates commonly seen band feature operations as neural forms and effectively improves the capacity to capture discriminative features for deraining. Second, to further improve the generalization ability of FBL to remove rain streaks of varied scales, we incorporate scale-robust self-supervision to regularize the network training. The constraint forces the extracted features of an input rain image at different scales to be equivalent after rescaling operations. Therefore, our method can offer similar responses based on solely image content without the interference of scale change and is capable to remove varied-scale rain streaks. Extensive experiments in quantitative and qualitative evaluations demonstrate the superiority of our method for rain streak removal, especially for the real cases where very large rain streaks exist, and prove the effectiveness of each component.

中文翻译：

---

通过分形带学习和自我监督消除任意规模的降雨条纹

数据驱动的去除雨水条痕的方法大多依赖于合成的配对数据，在实际应用中通常会遇到泛化问题。在本文中，我们提出了一种新颖的基于深度学习的雨水去除方法，该方法具有自我监督功能，以在实际应用中获得去除更多不同尺度雨水条纹的能力。为此，在这项工作中，从两个角度进行了努力。首先，考虑到去除雨水条纹与纹理特征高度相关，我们基于频带恢复创建了分形频带学习（FBL）网络。它以神经形式集成了常见的带特征操作，并有效地提高了捕获有区别特征以进行排空的能力。第二，为了进一步提高FBL的泛化能力，以消除各种规模的雨水条纹，我们采用了规模健壮的自我监督机制来规范化网络训练。约束迫使重新缩放操作后，不同尺度的输入降雨图像的提取特征相等。因此，我们的方法可以仅基于图像内容提供相似的响应，而不会受到水垢变化的干扰，并且能够消除水垢变化的雨条纹。在定量和定性评估中进行的大量实验证明了我们的雨纹去除方法的优越性，特别是对于存在很大雨纹的实际情况，并证明了每个组件的有效性。约束迫使重新缩放操作后，不同尺度的输入降雨图像的提取特征相等。因此，我们的方法可以仅基于图像内容提供类似的响应，而不会受到水垢变化的干扰，并且能够消除水垢变化的雨条纹。在定量和定性评估中进行的大量实验证明了我们的雨纹去除方法的优越性，特别是对于存在很大雨纹的实际情况，并证明了每个组件的有效性。约束迫使重新缩放操作后，不同尺度的输入降雨图像的提取特征相等。因此，我们的方法可以仅基于图像内容提供相似的响应，而不会受到水垢变化的干扰，并且能够消除水垢变化的雨条纹。在定量和定性评估中进行的大量实验证明了我们的雨纹去除方法的优越性，特别是对于存在很大雨纹的实际情况，并证明了每个组件的有效性。