Rain fills the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, quantifying its impact is difficult. In this context, we present a rain rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain. We present three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. We validate our rendering methods with a user study, demonstrating our rain is judged as much as 73% more realistic than the state-of-theart. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection, 60% for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of 21% on object detection, 37% on semantic segmentation, and 8% on depth estimation.

中文翻译：

---

用于评估和提高对恶劣天气的鲁棒性的雨水渲染

雨水使大气中充满了水粒子，这打破了光从场景到相机原封不动地传播的普遍假设。虽然众所周知下雨会影响计算机视觉算法，但很难量化其影响。在这种情况下，我们提出了一个雨水渲染管道，可以对常见的计算机视觉算法进行系统评估，以控制降雨量。我们提出了三种不同的方法来将合成雨添加到现有的图像数据集中：完全基于物理；完全由数据驱动；以及两者的结合。基于物理的降雨增强结合了物理粒子模拟器和精确的降雨光度建模。我们通过用户研究验证了我们的渲染方法，证明我们判断的降雨比最先进的方法逼真度高出 73%。使用我们生成的降雨增强的 KITTI、Cityscapes 和 nuScenes 数据集，我们对对象检测、语义分割和深度估计算法进行了全面评估，并表明它们在退化天气中的性能下降，对象检测的性能下降了 15%，语义分割增加 60%，深度估计误差增加 6 倍。对我们增强的合成数据进行微调后，对象检测提高了 21%，语义分割提高了 37%，深度估计提高了 8%。