Image-to-Image translation aims to learn an image from a source domain to a target domain. However, there are three main challenges, such as lack of paired datasets, multimodality, and diversity, that are associated with these problems and need to be dealt with. Convolutional neural networks (CNNs), despite of having great performance in many computer vision tasks, they fail to detect the hierarchy of spatial relationships between different parts of an object and thus do not form the ideal representative model we look for. This article presents a new variation of generative models that aims to remedy this problem. We use a trainable transformer, which explicitly allows the spatial manipulation of data within training. This differentiable module can be augmented into the convolutional layers in the generative model, and it allows to freely alter the generated distributions for image-to-image translation. To reap the benefits of proposed module into generative model, our architecture incorporates a new loss function to facilitate an effective end-to-end generative learning for image-to-image translation. The proposed model is evaluated through comprehensive experiments on image synthesizing and image-to-image translation, along with comparisons with several state-of-the-art algorithms.

中文翻译：

---

用于图像到图像转换的等变对抗网络

图像到图像的翻译旨在将图像从源域学习到目标域。然而，存在与这些问题相关并需要解决的三个主要挑战，例如缺乏配对数据集、多模态和多样性。卷积神经网络 (CNN) 尽管在许多计算机视觉任务中具有出色的性能，但它们无法检测对象不同部分之间的空间关系层次结构，因此无法形成我们寻找的理想代表模型。本文介绍了一种旨在解决此问题的生成模型的新变体。我们使用可训练的转换器，它明确允许在训练中对数据进行空间操作。这个可微分模块可以扩充到生成模型的卷积层中，它允许自由更改生成的图像到图像转换的分布。为了在生成模型中获得提议模块的好处，我们的架构结合了一个新的损失函数，以促进图像到图像转换的有效端到端生成学习。通过对图像合成和图像到图像转换的综合实验以及与几种最先进算法的比较，对所提出的模型进行了评估。