Spectral Graph Matching and Regularized Quadratic Relaxations I Algorithm and Gaussian Analysis

Abstract

Graph matching aims at finding the vertex correspondence between two unlabeled graphs that maximizes the total edge weight correlation. This amounts to solving a computationally intractable quadratic assignment problem. In this paper, we propose a new spectral method, graph matching by pairwise eigen-alignments (GRAMPA). Departing from prior spectral approaches that only compare top eigenvectors, or eigenvectors of the same order, GRAMPA first constructs a similarity matrix as a weighted sum of outer products between all pairs of eigenvectors of the two graphs, with weights given by a Cauchy kernel applied to the separation of the corresponding eigenvalues, then outputs a matching by a simple rounding procedure. The similarity matrix can also be interpreted as the solution to a regularized quadratic programming relaxation of the quadratic assignment problem. For the Gaussian Wigner model in which two complete graphs on n vertices have Gaussian edge weights with correlation coefficient $1-\sigma ^2$, we show that GRAMPA exactly recovers the correct vertex correspondence with high probability when $\sigma = O(\frac{1}{\log n})$. This matches the state of the art of polynomial-time algorithms and significantly improves over existing spectral methods which require $\sigma$ to be polynomially small in n. The superiority of GRAMPA is also demonstrated on a variety of synthetic and real datasets, in terms of both statistical accuracy and computational efficiency. Universality results, including similar guarantees for dense and sparse Erdős–Rényi graphs, are deferred to a companion paper.

Appendices

Concentration Inequalities for Gaussians

We collect auxiliary results on concentration of polynomials of Gaussian variables.

Lemma 13

Let z be a standard Gaussian vector in $\mathbb {R}^n$. For any fixed $v \in \mathbb {R}^n$ and $\delta > 0$, it holds with probability at least $1 - \delta$ that

$$\begin{aligned} |v^\top z| \le \Vert v\Vert _2 \sqrt{2 \log (1/\delta )} . \end{aligned}$$

Lemma 14

(Hanson–Wright inequality) Let z be a sub-Gaussian vector in $\mathbb {R}^n$, and let M be a fixed matrix in ${\mathbb {C}}^{n \times n}$. Then, we have with probability at least $1 - \delta$ that

$$\begin{aligned} |z^\top M z - {\text {Tr}}M|&\le C \Vert z\Vert _{\psi _2}^2 \Big ( \Vert M\Vert _F \sqrt{ \log (1/\delta ) } + \Vert M \Vert \log (1/\delta ) \Big ) \nonumber \\&\le 2C \Vert z\Vert _{\psi _2}^2 \Vert M\Vert _F \log (1/\delta ) , \end{aligned}$$

(75)

where C is a universal constant and $\Vert z\Vert _{\psi _2}$ is the sub-Gaussian norm of z.

See [59, Section 3.1] for the complex-valued version of the Hanson–Wright inequality. The following lemma is a direct consequence of (75), by taking M to be a diagonal matrix.

Lemma 15

Let z be a standard Gaussian vector in $\mathbb {R}^n$. For an entrywise nonnegative vector $v \in \mathbb {R}^n$, it holds with probability at least $1 - \delta$ that

$$\begin{aligned} \Big | \sum _{i=1}^n v_i z_i^2 - \sum _{i=1}^n v_i \Big | \le C\Big (\Vert v\Vert _2 \sqrt{ \log (1/\delta ) } + \Vert v\Vert _\infty \log (1/\delta )\Big ). \end{aligned}$$

In particular, it holds with probability at least $1 - \delta$ that

$$\begin{aligned} \big | \Vert z\Vert _2^2 - n \big | \le C\Big (\sqrt{n \log (1/\delta ) } + \log (1/\delta )\Big ) . \end{aligned}$$

Theorem 4

(Hypercontractivity concentration [61, Theorem 1.9]) Let z be a standard Gaussian vector in $\mathbb {R}^n$, and let $f(z_1, \dots , z_n)$ be a degree-d polynomial of z. Then, it holds that

$$\begin{aligned} \mathbb {P}\Big \{ \big | f(z) - \mathbb {E}[ f(z) ] \big | > t \Big \} \le e^2 \exp \bigg [ - \Big ( \frac{ t^2 }{ C \, \mathsf {Var}[ f(z) ] } \Big )^{1/d} \bigg ] , \end{aligned}$$

where $\mathsf {Var}[ f(z) ]$ denotes the variance of f(z) and $C > 0$ is a universal constant.

Finally, the following result gives a concentration inequality in terms of the restricted Lipschitz constants, obtained from the usual Gaussian concentration of measure plus a Lipschitz extension argument.

Lemma 16

Let $B\subset {\mathbb {R}}^n$ be an arbitrary measurable subset. Let $F: {\mathbb {R}}^n \rightarrow {\mathbb {R}}$ such that F is L-Lipschitz on B. Let $X \sim N(0,{\mathbf {I}}_n)$. Then, for any $t>0$,

$$\begin{aligned} \mathbb {P}\left\{ |F(X)- \mathbb {E}F(X)| \ge t + \delta \right\} \le \exp \left( -\frac{ct^2}{L^2} \right) + \epsilon , \end{aligned}$$

where c is a universal constant, and $\delta = 2 \sqrt{\epsilon ( nL^2+F(0)^2 + \mathbb {E}[F(X)^2] )}$.

Proof

Let ${{\widetilde{F}}}: {\mathbb {R}}^n\rightarrow {\mathbb {R}}$ be an L-Lipschitz extension of F, e.g., ${{\widetilde{F}}}(x) = \inf _{y\in B} F(y)+L\Vert x-y\Vert$. Then, by the Gaussian concentration inequality (cf., e.g., [66, Theorem 5.2.2]), we have

$$\begin{aligned} \mathbb {P}\left\{ |{{\widetilde{F}}}(X)- \mathbb {E}{{\widetilde{F}}}(X)| \ge t \right\} \le \exp \left( -\frac{ct^2}{L^2} \right) . \end{aligned}$$

It remains to show that $|\mathbb {E}F(X)-\mathbb {E}{{\widetilde{F}}}(X)| \le \delta$. Indeed, by Cauchy–Schwarz, $|\mathbb {E}F(X)-\mathbb {E}{\widetilde{F}}(X)| \le \mathbb {E}|F(X-{{\widetilde{F}}}(X)|{{\mathbf {1}}_{\left\{ {X \notin B}\right\} }} \le \sqrt{\epsilon \mathbb {E}[|F(X-{{\widetilde{F}}}(X)|^2]}$. Finally, noting that $|{{\widetilde{F}}}(X)| \le F(0)+L\Vert X\Vert _2$ and $\mathbb {E}\Vert X\Vert _2^2 =n$ completes the proof. $\square$

Kronecker Gymnastics

Given $A,B \in {\mathbb {C}}^{n\times n}$, the Kronecker product $A \otimes B \in {\mathbb {C}}^{n^2\times n^2}$ is defined as $\left[ {\begin{matrix} a_{11} B&{}\ldots &{}a_{1n} B\\ \vdots &{}\vdots &{}\vdots \\ a_{n1} B&{}\ldots &{}a_{nn} B \end{matrix}} \right]$. The vectorized form of $A=[a_1,\ldots ,a_n]$ is $\,\textsf {vec} (A)=[a_1^\top ,\ldots ,a_n^\top ]^\top \in {\mathbb {C}}^{n\otimes n}$. It is convenient to identify $[n^2]$ with by $[n]^2$ ordered as $\{(1,1),\ldots ,(1,n),\ldots ,(n,n)\}$, in which case we have $(A\otimes B)_{ij,k\ell }=A_{ik}B_{j\ell }$ and $\,\textsf {vec} (A)_{ij}=A_{ij}$.

We collect some identities for Kronecker products and vectorizations of matrices used throughout this paper:

$$\begin{aligned} \langle A \otimes A, B \otimes B \rangle&= \langle A, B \rangle ^2 , \\ (A \otimes B) ( U \otimes V)&= AU \otimes BV , \\ \,\textsf {vec} (A U B)&= (B^\top \otimes A) \,\textsf {vec} (U) , \\ (X \otimes Y) \,\textsf {vec} (U)&= \,\textsf {vec} (Y U X^\top ) , \\ (A \otimes B)^\top&= A^\top \otimes B^\top . \end{aligned}$$

The third equality implies that

$$\begin{aligned} \langle A \otimes B, \,\textsf {vec} (U) \,\textsf {vec} (V)^\top \rangle&= \langle \,\textsf {vec} (U), (A \otimes B) \,\textsf {vec} (V) \rangle \\&= \langle \,\textsf {vec} (U), \,\textsf {vec} (B V A^\top ) \rangle \\&=\langle U, BVA^\top \rangle = \langle B, U A V^\top \rangle \end{aligned}$$

and hence

$$\begin{aligned} \,\textsf {vec} (U)^\top ( A \otimes B)\,\textsf {vec} (V)&= \langle A \otimes B, \,\textsf {vec} (U) \,\textsf {vec} (V)^\top \rangle = \langle B, U A V^\top \rangle \\&= \langle A, U^\top B V\rangle = \langle UA,B V\rangle . \end{aligned}$$

Applying the third equality to column vector z and noting that $\,\textsf {vec} (z^\top )=\,\textsf {vec} (z)=z$, we have

$$\begin{aligned} (X \otimes y) z =&~ \,\textsf {vec} (yz^\top X^\top ) , \\ (y \otimes X) z =&~ \,\textsf {vec} (X z y^\top ) . \end{aligned}$$

In particular, it holds that

$$\begin{aligned} ({\mathbf {I}}\otimes y) z =&~ \,\textsf {vec} (yz^\top ) = z \otimes y , \\ (y \otimes {\mathbf {I}}) z =&~ \,\textsf {vec} (z y^\top )= y \otimes z . \end{aligned}$$

Signal-to-Noise Heuristics

We justify the choice of the Cauchy weight kernel in (4) by a heuristic signal-to-noise calculation for ${\widehat{X}}$. We assume without loss of generality that $\pi ^*$ is the identity, so that diagonal entries of ${\widehat{X}}$ indicate similarity between matching vertices of A and B. Then, for the rounding procedure in (5), we may interpret $n^{-1}{\text {Tr}}{\widehat{X}}$ and $(n^{-2}\sum _{i,j:\,i \ne j} {\widehat{X}}_{ij}^2)^{1/2} \approx n^{-1}\Vert {\widehat{X}}\Vert _F$ as the average signal strength and noise level in ${\widehat{X}}$. Let us define a corresponding signal-to-noise ratio as

$$\begin{aligned} \mathrm {SNR}=\frac{\mathbb {E}[{\text {Tr}}{\widehat{X}}]}{\mathbb {E}[\Vert {\widehat{X}}\Vert _F^2]^{1/2}} \end{aligned}$$

and compute this quantity in the Gaussian Wigner model.

We abbreviate the spectral weights $w(\lambda _i,\mu _j)$ as $w_{ij}$. For ${\widehat{X}}$ defined by (3) with any weight kernel w(x, y), we have

$$\begin{aligned} {\text {Tr}}{\widehat{X}}=\sum _{ij} w_{ij} \cdot u_i^\top {\mathbf {J}}v_j \cdot u_i^\top v_j. \end{aligned}$$

Applying that (A, B) is equal in law to $(OAO^\top ,OBO^\top )$ for a rotation O such that $O{\mathbf {1}}=\sqrt{n}{\mathbf {e}}_k$, we obtain for every k that

$$\begin{aligned} \mathbb {E}[{\text {Tr}}{\widehat{X}}]=\sum _{ij} n \cdot \mathbb {E}[w_{ij} \cdot u_i^\top ({\mathbf {e}}_k{\mathbf {e}}_k^\top ) v_j \cdot u_i^\top v_j]. \end{aligned}$$

Then, averaging over $k=1,\ldots ,n$ and applying $\sum _k {\mathbf {e}}_k {\mathbf {e}}_k^\top ={\mathbf {I}}$ yield that

$$\begin{aligned} \mathbb {E}[{\text {Tr}}{\widehat{X}}]=\sum _{ij} \mathbb {E}[w_{ij}(u_i^\top v_j)^2]. \end{aligned}$$

For the noise, we have

$$\begin{aligned} \Vert {\widehat{X}}\Vert _F^2= & {} {\text {Tr}}{\widehat{X}}{\widehat{X}}^\top =\sum _{i,j,k,l} w_{ij}w_{kl} (u_i^\top {\mathbf {J}}v_j) (u_k^\top {\mathbf {J}}v_\ell ) {\text {Tr}}(u_iv_j^\top \cdot v_\ell u_k^\top )\\= & {} \sum _{ij} w_{ij}^2 (u_i^\top {\mathbf {J}}v_j)^2. \end{aligned}$$

Applying the equality in law of (A, B) and $(OAO^\top ,OBO^\top )$ for a uniform random orthogonal matrix O, and writing $r=O{\mathbf {1}}/\sqrt{n}$, we get

$$\begin{aligned} \mathbb {E}[\Vert {\widehat{X}}\Vert _F^2]=\sum _{ij}n^2 \cdot \mathbb {E}[w_{ij}^2(u_i^\top r)^2(v_j^\top r)^2]. \end{aligned}$$

Here, $r=(r_1,\ldots ,r_n)$ is a uniform random vector on the unit sphere, independent of (A, B). For any deterministic unit vectors u, v with $u^\top v=\alpha$, we may rotate to $u={\mathbf {e}}_1$ and $v=\alpha {\mathbf {e}}_1+\sqrt{1-\alpha ^2}{\mathbf {e}}_2$ to get

$$\begin{aligned} \mathbb {E}[(u^\top r)^2(v^\top r)^2]= & {} \mathbb {E}[r_1^2 \cdot (\alpha r_1+\sqrt{1-\alpha ^2}r_2)^2]\\= & {} \alpha ^2\mathbb {E}[r_1^4]+(1-\alpha ^2)\mathbb {E}[r_1^2r_2^2] =\frac{1+2\alpha ^2}{n(n+2)}, \end{aligned}$$

where the last equality applies an elementary computation. Bounding $1+2\alpha ^2 \in [1,3]$ and applying this conditional on (A, B) above, we obtain

$$\begin{aligned} \mathbb {E}[\Vert {\widehat{X}}\Vert _F^2]=\frac{cn}{n+2} \sum _{ij} \mathbb {E}[w_{ij}^2] \end{aligned}$$

for some value $c \in [1,3]$.

To summarize,

$$\begin{aligned} \mathrm {SNR} \asymp \frac{\sum _{ij} \mathbb {E}[w(\lambda _i,\mu _j) (u_i^\top v_j)^2]}{\sqrt{\sum _{ij} \mathbb {E}[w(\lambda _i,\mu _j)^2]}}. \end{aligned}$$

The choice of weights which maximizes this SNR would satisfy $w(\lambda _i,\mu _j) \propto (u_i^\top v_j)^2$. Recall that for $n^{-1+\varepsilon } \ll \sigma ^2 \ll n^{-\varepsilon }$ and i, j in the bulk of the spectrum, we have the approximation (11). Thus, this optimal choice of weights takes a Cauchy form, which motivates our choice in (4).

We note that this discussion is only heuristic, and maximizing this definition of SNR does not automatically imply any rigorous guarantee for exact recovery of $\pi ^*$. Our proposal in (4) is a bit simpler than the optimal choice suggested by (11): The constant C in (11) depends on the semicircle density near $\lambda _i$, but we do not incorporate this dependence in our definition. Also, while (11) depends on the noise level $\sigma$, our main result in Theorem 1 shows that $\eta$ need not be set based on $\sigma$, which is usually unknown in practice. Instead, our result shows that the simpler choice $\eta =c/\log n$ is sufficient for exact recovery of $\pi ^*$ over a range of noise levels $\sigma \lesssim \eta$.