Feature based image matching has been a research focus in photogrammetry and computer vision for decades, as it is the basis for many applications where multi-view geometry is needed. A typical feature based image matching algorithm contains five steps: feature detection, affine shape estimation, orientation assignment, description and descriptor matching. This paper contains innovative work in different steps of feature matching based on convolutional neural networks (CNN). For the affine shape estimation and orientation assignment, the main contribution of this paper is twofold. First, we define a canonical shape and orientation for each feature. As a consequence, instead of the usual Siamese CNN, only single branch CNNs needs to be employed to learn the affine shape and orientation parameters, which turns the related tasks from supervised to self supervised learning problems, removing the need for known matching relationships between features. Second, the affine shape and orientation are solved simultaneously. To the best of our knowledge, this is the first time these two modules are reported to have been successfully trained together. In addition, for the descriptor learning part, a new weak match finder is suggested to better explore the intra-variance of the appearance of matched features. For any input feature patch, a transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features; they are subsequently used in the standard descriptor learning framework. The proposed modules are integrated into an inference pipeline to form the proposed feature matching algorithm. The algorithm is evaluated on standard benchmarks and is used to solve for the parameters of image orientation of aerial oblique images. It is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block geometry than conventional methods. The code is available at https://github.com/Childhoo/Chen_Matcher.git.

几十年来，基于特征的图像匹配一直是摄影测量和计算机视觉的研究重点，因为它是许多需要多视图几何的应用的基础。典型的基于特征的图像匹配算法包含五个步骤：特征检测、仿射形状估计、方向分配、描述和描述符匹配。本文包含基于卷积神经网络 (CNN) 的特征匹配不同步骤的创新工作。对于仿射形状估计和方向分配，本文的主要贡献是双重的。首先，我们为每个特征定义一个规范的形状和方向。因此，不需要使用通常的 Siamese CNN，只需要使用单分支 CNN 来学习仿射形状和方向参数，这将相关任务从监督学习问题转变为自我监督学习问题，从而消除了对特征之间已知匹配关系的需求。其次，同时求解仿射形状和方向。据我们所知，这是第一次有报道称这两个模块一起成功训练。此外，对于描述符学习部分，提出了一种新的弱匹配查找器，以更好地探索匹配特征外观的内部方差。对于任何输入特征补丁，在描述符空间中远离输入特征补丁的转换补丁被定义为弱匹配特征。提出了一个弱匹配查找器网络来主动发现这些弱匹配特征；它们随后被用于标准描述符学习框架中。将所提出的模块集成到推理管道中，以形成所提出的特征匹配算法。该算法在标准基准上进行评估，用于求解航拍倾斜图像的图像方位参数。结果表明，与传统方法相比，基于深度学习特征的图像匹配导致更多的配准图像、更多的重建 3D 点和更稳定的块几何形状。该代码可在 https://github.com/Childhoo/Chen_Matcher.git 获得。比传统方法更多的重建 3D 点和更稳定的块几何形状。该代码可在 https://github.com/Childhoo/Chen_Matcher.git 获得。比传统方法更多的重建 3D 点和更稳定的块几何形状。该代码可在 https://github.com/Childhoo/Chen_Matcher.git 获得。