Image captioning has made substantial progress with huge supporting image collections sourced from the web. However, recent studies have pointed out that captioning datasets, such as COCO, contain gender bias found in web corpora. As a result, learning models could heavily rely on the learned priors and image context for gender identification, leading to incorrect or even offensive errors. To encourage models to learn correct gender features, we reorganize the COCO dataset and present two new splits COCO-GB V1 and V2 datasets where the train and test sets have different gender-context joint distribution. Models relying on contextual cues will suffer from huge gender prediction errors on the anti-stereotypical test data. Benchmarking experiments reveal that most captioning models learn gender bias, leading to high gender prediction errors, especially for women. To alleviate the unwanted bias, we propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence. Experimental results validate that GAIC can significantly reduce gender prediction errors with a competitive caption quality. Our codes and the designed benchmark datasets are available at https://github.com/CaptionGenderBias2020.

中文翻译：

---

减轻字幕系统中的性别偏见

图像字幕已经取得了实质性的进展，大量支持来自网络的图像集。然而，最近的研究指出，字幕数据集，如 COCO，包含在网络语料库中发现的性别偏见。因此，学习模型可能在很大程度上依赖于学习到的先验和图像上下文进行性别识别，从而导致不正确甚至令人反感的错误。为了鼓励模型学习正确的性别特征，我们重新组织了 COCO 数据集并展示了两个新的拆分 COCO-GB V1 和 V2 数据集，其中训练集和测试集具有不同的性别上下文联合分布。依赖上下文线索的模型将在反刻板印象测试数据上遭受巨大的性别预测错误。基准实验表明，大多数字幕模型学习性别偏见，导致较高的性别预测错误，特别是对于女性。为了减轻不必要的偏见，我们提出了一种新的引导注意图像字幕模型（GAIC），该模型提供视觉注意的自我指导，以鼓励模型捕获正确的性别视觉证据。实验结果验证了 GAIC 可以显着减少性别预测错误，并具有具有竞争力的字幕质量。我们的代码和设计的基准数据集可在 https://github.com/CaptionGenderBias2020 获得。