We tackle the problem of estimating dense semantic correspondence between two images depicting different instances of the same category. In this paper, we consider semantic context and correspondence information from the neighborhood in order to overcome the drawback of previous works that estimate the correspondence of each pixel or patch independently. To this end, a novel network, called SANet, with a trainable spatial aggregation module is proposed, which is trained in an end-to-end manner and outputs semantic flow. We train this SANet by adopting two complementary loss terms: landmark loss, focusing on keypoints with ground truth, and consistency loss, applicable to all pixels without ground truth. Qualitative and quantitative experimental results demonstrate the improved network achieves a better balance between accuracy and smoothness comparing with the baseline and warps images with better visual quality.

我们解决了估计描述同一类别不同实例的两幅图像之间的密集语义对应的问题。在本文中，我们考虑了来自邻域的语义上下文和对应信息，以克服以前的工作中独立估计每个像素或补丁对应关系的缺点。为此，提出了一种具有可训练的空间聚合模块的新型网络，称为SANet，该网络以端到端的方式进行训练并输出语义流。我们通过采用两个互补的损耗项来训练此SANet：地标损耗（重点在于具有真实面的关键点）和一致性损耗（适用于所有没有真实面的像素）。