Rain streaks attached to a camera may seriously affect the visibility of the background and considerably degrade image quality. We handle this problem by removing rain streaks to convert a rainy image into a clean one. This creates a problem in that the information with respect to the background of the occluded parts is close to being lost for the most part. To remove rain streaks from a single image, we employ a multi-scale aggregated generator network. Differing from previous generative adversarial networks, an enhanced generator block (EGB) is applied in the generator; this broadens the receptive field of the input features and enhances the attention to rain streaks details. In addition, both the generator and the discriminator networks contain the visual mask that learns the rain streak regions and their surroundings during training. By putting the mask forward, the generator network focuses on rain streaks and the surrounding regions and the discriminator network is capable of evaluating the local consistency of the restored regions.