Perception of content and structure in images with rainstreaks or raindrops is challenging, and it often calls for robust deraining algorithms to remove the diversified rainy effects. Much progress has been made on the design of advanced encoder–decoder single image deraining networks. However, most of the existing networks are built in a blind manner and often produce over/under-deraining artefacts. In this paper, we point out, for the first time, that the unsatisfactory results are caused by the highly imbalanced distribution between rainy effects and varied background scenes. Ignoring this phenomenon results in the representation learned by the encoder being biased towards rainy regions, while paying less attention to the valuable contextual regions. To resolve this, a context-enhanced representation learning and deraining network is proposed with a novel two-branch encoder design. Specifically, one branch takes the rainy image directly as input for learning a mixed representation depicting the variation of both rainy regions and contextual regions, and another branch is guided by a carefully learned soft attention mask to learn an embedding only depicting the contextual regions. By combining the embeddings from these two branches with a carefully designed co-occurrence modelling module, and then improving the semantic property of the co-occurrence features via a bi-directional attention layer, the underlying imbalanced learning problem is resolved. Extensive experiments are carried out for removing rainstreaks and raindrops from both synthetic and real rainy images, and the proposed model is demonstrated to produce significantly better results than state-of-the-art models. In addition, comprehensive ablation studies are also performed to analyze the contributions of different designs. Code and pre-trained models will be publicly available at https://github.com/RobinCSIRO/CERLD-Net.git.

感知带有雨滴或雨滴的图像中的内容和结构具有挑战性，这通常需要强大的排水算法来消除各种雨天效应。在高级编码器-解码器单图像消除网络的设计方面已经取得了很大进展。但是，大多数现有网络都是以盲目方式构建的，通常会产生过度/欠消耗的人工制品。在本文中，我们首次指出，令人不满意的结果是由于雨天效果和变化的背景场景之间的高度不平衡造成的。忽略这种现象会导致编码器学习到的表示偏向多雨区域，而很少注意有价值的上下文区域。为了解决这个问题，提出了一种具有新颖的两分支编码器设计的上下文增强表示学习和排水网络。具体而言，一个分支直接将多雨图像作为输入，以学习描述雨域和上下文区域变化的混合表示，而另一分支则由精心学习的软注意蒙版引导，以学习仅描述上下文区域的嵌入。通过将来自这两个分支的嵌入与精心设计的共现建模模块进行组合，然后通过双向关注层改善共现特征的语义特性，可以解决潜在的不平衡学习问题。为了从合成和真实的多雨图像中去除雨滴和雨滴，进行了广泛的实验，并证明所提出的模型比最先进的模型产生明显更好的结果。此外，还进行了全面的消融研究，以分析不同设计的贡献。代码和预先训练的模型将在https://github.com/RobinCSIRO/CERLD-Net.git上公开提供。