We propose an image alignment algorithm based on weak supervision, which aims to identify the correspondence between a pair of reference and target images with no supervision of individual pixels. Since most existing methods have relied on a predefined geometric model such as homography, they often suffer from a lack of model flexibility and generalizability. To tackle the challenge, we propose a novel nonparametric transformation model based on graph convolutional networks without an explicit geometric constraint. The proposed method is generic and flexible in the sense that it is applicable to the image pairs undergoing diverse local and/or global transformations. To make the algorithm more suitable for real-world scenarios having potential noises from moving objects, we disregard those objects with an off-the-shelf semantic segmentation model. The proposed algorithm is evaluated on the Cityscapes dataset with annotated pixel-level correspondences and outperforms baseline methods relying on global parametric transformations.

我们提出了一种基于弱监督的图像对齐算法，旨在识别一对参考图像和目标图像之间的对应关系，而无需对单个像素进行监督。由于大多数现有方法都依赖于预定义的几何模型（例如单应性），因此它们通常缺乏模型的灵活性和通用性。为了应对这一挑战，我们提出了一种基于图卷积网络的新型非参数变换模型，没有明确的几何约束。所提出的方法是通用和灵活的，因为它适用于经历各种局部和/或全局变换的图像对。为了使算法更适合具有来自移动对象的潜在噪声的现实场景，我们使用现成的语义分割模型忽略这些对象。