Graph matching involves combinatorial optimization based on edge-to-edge affinity matrix, which can be generally formulated as Lawler's quadratic assignment problem (QAP). This paper presents a QAP network directly learning with the affinity matrix (equivalently the association graph) whereby the matching problem is translated into a constrained vertex classification task. The association graph is learned by an embedding network for vertex classification, followed by Sinkhorn normalization and a cross-entropy loss for end-to-end learning. We further improve the embedding model on association graph by introducing Sinkhorn based matching-aware constraint, as well as dummy nodes to deal with unequal sizes of graphs. To our best knowledge, this is one of the first network to directly learn with the general Lawler's QAP. In contrast, recent deep matching methods focus on the learning of node/edge features in two graphs respectively. We also show how to extend our network to hypergraph matching, and matching of multiple graphs. Experimental results on both synthetic graphs and real-world images show its effectiveness. For pure QAP tasks on synthetic data and QAPLIB benchmark, our method can perform competitively and even surpass state-of-the-art graph matching and QAP solvers with notable less time cost. We provide a project homepage at http://thinklab.sjtu.edu.cn/project/NGM/index.html.

中文翻译：

---

神经图匹配网络：学习 Lawler 的二次分配问题并扩展到超图和多图匹配

图匹配涉及基于边到边亲和矩阵的组合优化，通常可以表述为劳勒的二次分配问题（QAP）。本文提出了一个直接使用亲和矩阵（相当于关联图）学习的 QAP 网络，从而将匹配问题转化为受约束的顶点分类任务。关联图由嵌入网络学习用于顶点分类，然后是 Sinkhorn 归一化和用于端到端学习的交叉熵损失。我们通过引入基于 Sinkhorn 的匹配感知约束以及虚拟节点来处理不等大小的图，进一步改进了关联图上的嵌入模型。据我们所知，这是第一个直接使用通用 Lawler 的 QAP 学习的网络之一。相比之下，最近的深度匹配方法分别侧重于两个图中节点/边缘特征的学习。我们还展示了如何将我们的网络扩展到超图匹配和多个图的匹配。合成图和真实世界图像的实验结果表明了它的有效性。对于合成数据和 QAPLIB 基准上的纯 QAP 任务，我们的方法可以具有竞争力，甚至超越最先进的图匹配和 QAP 求解器，时间成本显着降低。我们在以下位置提供项目主页 我们的方法可以具有竞争力，甚至超越最先进的图匹配和 QAP 求解器，时间成本显着降低。我们在以下位置提供项目主页 我们的方法可以具有竞争力，甚至超越最先进的图匹配和 QAP 求解器，时间成本显着降低。我们在以下位置提供项目主页http://thinklab.sjtu.edu.cn/project/NGM/index.html.