The Asymptotics of the Clustering Transition for Random Constraint Satisfaction Problems

Abstract

Random constraint satisfaction problems exhibit several phase transitions when their density of constraints is varied. One of these threshold phenomena, known as the clustering or dynamic transition, corresponds to a transition for an information theoretic problem called tree reconstruction. In this article we study this threshold for two CSPs, namely the bicoloring of k-uniform hypergraphs with a density $\alpha$ of constraints, and the q-coloring of random graphs with average degree c. We show that in the large k, q limit the clustering transition occurs for $\alpha = \frac{2^{k-1}}{k} (\ln k + \ln \ln k + \gamma _{\mathrm{d}} + o(1))$, $c= q (\ln q + \ln \ln q + \gamma _{\mathrm{d}}+ o(1))$, where $\gamma _{\mathrm{d}}$ is the same constant for both models. We characterize $\gamma _{\mathrm{d}}$ via a functional equation, solve the latter numerically to estimate $\gamma _{\mathrm{d}} \approx 0.871$, and obtain an analytic lowerbound $\gamma _{\mathrm{d}} \ge 1 + \ln (2 (\sqrt{2}-1)) \approx 0.812$. Our analysis unveils a subtle interplay of the clustering transition with the rigidity (naive reconstruction) threshold that occurs on the same asymptotic scale at $\gamma _{\mathrm{r}}=1$.

A The Graph Coloring Case

This Appendix is devoted to the graph coloring problem. We shall present computations that are the counterparts of the ones explained in Sect. 2 and 3 of the main text for the hypergraph bicoloring problem, namely the definition of the reconstruction problem and its study in the limit where the number of colors q and the degree c diverge simultaneously according to (1), and show that the very same equations summarized at the beginning of Sect. 4 arise in this limit. As the computations are quite similar we will be more succinct than in the main text and concentrate on the specificities of the coloring problem.

Let us consider a rooted tree with spins $\sigma _i$ placed on its vertices. These spins can take q values, interpreted as colors, $\sigma _i\in \{1,\dots ,q \}$. A proper coloring of the tree is a configuration of the spins such that no edge is monochromatic (i.e. no pair of adjacent vertices are given the same color). A uniform proper coloring can be drawn in a broadcast fashion, by choosing the color $\sigma$ of the root uniformly at random among the q possible ones, then each descendent of the root is assigned a color uniformly at random among the $q-1$ colors distinct from $\sigma$, and this is repeated recursively down to the n-th generation of the tree. In the reconstruction problem an observer is then provided with the colors on the vertices of the n-th generation of the tree only, and asked to guess the color of the root. The optimal strategy is to compute $\eta$, the posterior probability of the root given the observations, which is a distribution over $\{1,\dots ,q\}$. A moment of thought reveals that the probability distribution of $\eta$, with respect to a broadcast process conditioned on the root value $\sigma$, and with respect to a random choice of the tree as a Galton-Watson branching process with Poisson offspring distribution of mean c, is a measure $P_{\sigma ,n}(\eta )$ that can be determined recursively through the induction relation:

$$\begin{aligned} P_{\sigma ,n+1}(\eta ) = \sum _{l=0}^\infty e^{-c} \frac{c^l}{l!} \frac{1}{(q-1)^l} \sum _{\sigma _1,\dots ,\sigma _l \ne \sigma } \int \prod _{i=1}^l \mathrm{d}P_{\sigma _i,n}(\eta _i) \, \delta (\eta -f_\mathrm{c}(\eta _1,\dots ,\eta _l)) \ , \end{aligned}$$

(86)

where the Belief Propagation recursion function $f_{\mathrm{c}}$ is defined here in such a way that $\eta =f_\mathrm{c}(\eta _1,\dots ,\eta _l)$ means

$$\begin{aligned} \eta (\tau ) = \frac{\underset{i=1}{\overset{l}{\prod }}(1-\eta _i(\tau ))}{\underset{\tau '}{\sum } \underset{i=1}{\overset{l}{\prod }}(1-\eta _i(\tau '))} \ . \end{aligned}$$

(87)

The initial condition of this recursive computation is given by $P_{\sigma ,0}=\delta (\eta - \delta _\sigma )$, where $\delta _\sigma$ is the measure concentrated on the color $\sigma$, i.e. $\delta _\sigma (\tau ) = {\mathbb {I}}(\sigma =\tau )$, which corresponds to the colors being revealed on the leaves of the tree. These equations correspond to (9,12,13) for the hypergraph bicoloring problem.

We separate now the contribution of “hard fields”, or frozen variables, namely the configurations of boundary variables that determine unambiguously the root in the naive reconstruction procedure. This forced value can only be the correct one the root had in the broadcast process, hence we shall write :

$$\begin{aligned} P_{\sigma ,n}(\eta ) = w_n \, \delta (\eta -\delta _\sigma ) + (1-w_n) \, Q_{\sigma ,n}(\eta ) \ , \end{aligned}$$

(88)

where $Q_{\sigma ,n}$ has no atom on $\delta _\sigma$; this mimicks the decomposition (22) of the main text. Plugging this decomposition in (86) yields the evolution equation for the weight of the hard fields:

$$\begin{aligned} w_{n+1} = \left( 1-e^{-\frac{c w_n}{q-1}} \right) ^{q-1} \ , \qquad \text {with} \ \ w_0=1 \ . \end{aligned}$$

(89)

Indeed $f_{\mathrm{c}}(\eta _1,\dots ,\eta _l) = \delta _\sigma$ if and only if for each color $\sigma '\ne \sigma$ at least one of the arguments $\eta _i$ is equal to $\delta _{\sigma '}$, thus forbidding all colors except $\sigma$. The number of hard fields of the color $\sigma ' \ne \sigma$ is easily seen from (86) to be Poisson distributed with average $\frac{c w_n}{q-1}$, independently from one color $\sigma '$ to another, from which (89) follows.

The recursion (89) has a bifurcation at $c_{\mathrm{r}}(q)$, in the sense that $w_n \rightarrow 0$ as $n \rightarrow \infty$ if and only if $c<c_{\mathrm{r}}(q)$. Writing down the equations fixing $c_{\mathrm{r}}$ and $w_{\mathrm{r}}$ at the bifurcation, which are similar to (25), and then expanding them for large q one finds the asymptotic expansion $c_{\mathrm{r}}=q (\ln q + \ln \ln q + 1 + o(1))$. We shall thus study the large q limit with c getting also large, on the scale $c=q (\ln q + \ln \ln q + \gamma )$ with $\gamma$ finite. In this limit one finds that for n finite

$$\begin{aligned} w_n = 1 - \frac{x_n}{\ln q} + o\left( \frac{1}{\ln q} \right) \ , \end{aligned}$$

(90)

where $x_n$ is of order 1 and obeys exactly the same recursion as in the bicoloring case, namely $x_0=0$ and $x_{n+1}=e^{-\gamma + x_n}$.

Let us now simplify the evolution equation for the distributions $Q_{\sigma ,n}$ of the soft fields. First of all we insert the decomposition (88) in the right hand side of (86) and obtain, without any approximation,

$$\begin{aligned}&P_{\sigma ,n+1}(\eta ) = \sum _{l=0}^\infty e^{-c(1-w_n)} \frac{(c(1-w_n))^l}{l!} \frac{1}{(q-1)^l} \sum _{\sigma _1,\dots ,\sigma _l \ne \sigma } \sum _{\{p_{\sigma '}=0,1\}_{\sigma ' \ne \sigma }} \nonumber \\&\quad \prod _{\sigma ' \ne \sigma } \left( e^{-\frac{c w_n}{q-1}}\right) ^{p_{\sigma '}} \left( 1-e^{-\frac{c w_n}{q-1}}\right) ^{1-p_{\sigma '}} \int \prod _{i=1}^l \mathrm{d}Q_{\sigma _i,n}(\eta _i) \, \delta (\eta -{\widetilde{f}}_\mathrm{c}(\sigma ,\{p_{\sigma '}\}_{\sigma ' \ne \sigma };\eta _1,\dots ,\eta _l)) \ , \end{aligned}$$

(91)

where l is the number of neighbors of the root that receive a soft field, $\sigma _1,\dots ,\sigma _l$ the colors these vertices have in the broadcast, and the indicator variables $p_{\sigma '}$ are equal to 1 if and only if no neighbor of the root assigned the color $\sigma '$ in the broadcast is perfectly recovered (i.e. receives a hard field). Hence the colors $\sigma '\ne \sigma$ with $p_{\sigma '}=0$ are precisely the ones forbidden for the root, as at least one of its neighbors is forced to this value. The relation $\eta ={\widetilde{f}}_\mathrm{c}(\sigma ,\{p_{\sigma '}\}_{\sigma ' \ne \sigma };\eta _1,\dots ,\eta _l)$ is obtained by specializing $f_{\mathrm{c}}$ of (87) to this pattern for the presence of hard fields in its arguments, and thus reads

$$\begin{aligned} \eta (\tau ) = \frac{p_\tau \underset{i=1}{\overset{l}{\prod }} (1-\eta _i(\tau ))}{\underset{\tau '}{\sum } p_{\tau '}\underset{i=1}{\overset{l}{\prod }}(1-\eta _i(\tau '))} \ , \end{aligned}$$

(92)

with the convention $p_{\sigma }=1$.

Let us call $p =\underset{\sigma ' \ne \sigma }{\sum } p_{\sigma '}$ the number of colors that satisfy the condition explained above; in the equation (91) it corresponds to a random variable with a binomial distribution of parameters $(q-1,e^{-\frac{c w_n}{q-1}})$. According to (90) the product of these parameters go to zero as $1/\ln q$ in the limit we are considering, we shall thus truncate (91) on the smallest possible values of p. As $p=0$ yields a hard field in the left hand side of (91), the distribution of the soft fields is dominated in this limit by the case $p=1$. As a consequence the fields $\eta$ in the support of $Q_{\sigma ,n}$ have non-zero values on two colors only, $\sigma$ and another one $\sigma '$ uniformly distributed on the $q-1$ possibilities. Let us parametrize this type of distributions via a distribution $Q_n(h)$ on real random variables $h\in [-1,1]$,

$$\begin{aligned} Q_{\sigma ,n}(\eta ) = \int \mathrm{d}Q_n(h) \frac{1}{q-1} \sum _{\sigma ' \ne \sigma } \delta (\eta -s(\sigma ,\sigma ';h)) \ , \end{aligned}$$

(93)

with

$$\begin{aligned} s(\sigma ,\sigma ';h)(\tau ) = {\left\{ \begin{array}{ll} \frac{1+h}{2} &{} \text {if} \ \ \tau = \sigma \\ \frac{1-h}{2} &{} \text {if} \ \ \tau = \sigma ' \\ 0 &{} \text {otherwise} \end{array}\right. } \ . \end{aligned}$$

(94)

Let us also denote ${\widehat{f}}_{\mathrm{c}}(\sigma ,\sigma ';\eta _1,\dots ,\eta _l)$ the function ${\widetilde{f}}_{\mathrm{c}}(\sigma ,\{p_{\sigma ''}\}_{\sigma '' \ne \sigma };\eta _1,\dots ,\eta _l)$ with $p_{\sigma ''}=\delta _{\sigma '',\sigma '}$, in such a way that the only two non-zero components of ${\widehat{f}}_\mathrm{c}(\sigma ,\sigma ';\eta _1,\dots ,\eta _l)$ correspond to the colors $\sigma$ and $\sigma '$. Using this notation, and the parametrization (93), one can deduce from (91) the evolution equation for $Q_{\sigma ,n}(\eta )$ at lowest order:

$$\begin{aligned} Q_{\sigma ,n+1}(\eta ) =&\frac{1}{q-1}\sum _{\sigma ' \ne \sigma } \sum _{l=0}^\infty e^{-c(1-w_n)} \frac{(c(1-w_n))^l}{l!} \prod _{i=1}^l \left( \frac{1}{(q-1)^2} \sum _{\begin{array}{c} \sigma _i\ne \sigma \\ \sigma '_i \ne \sigma _i \end{array}} \int \mathrm{d}Q_n(h_i) \right) \nonumber \\&\delta (\eta -{\widehat{f}}_\mathrm{c}(\sigma ,\sigma ';s(\sigma _1,\sigma '_1;h_1),\dots ,s(\sigma _l,\sigma '_l;h_l))) \end{aligned}$$

(95)

To put this expression under the form (93), and hence close the recursion on $Q_n(h)$, it remains to notice that

$$\begin{aligned} {\widehat{f}}_{\mathrm{c}}(\sigma ,\sigma ';\eta _1,\dots ,\eta _l) = s(\sigma ,\sigma ',h) \quad \text {with} \ \ h=f(u_1,\dots ,u_l) \ , \quad u_i = {\widehat{u}}(\sigma ,\sigma ';\eta _i) \ , \end{aligned}$$

(96)

where f is the function defined for Ising spins in (9), and

$$\begin{aligned} {\widehat{u}}(\sigma ,\sigma ';\eta ) = \frac{\eta (\sigma ')-\eta (\sigma )}{2-\eta (\sigma )-\eta (\sigma ')} \ . \end{aligned}$$

(97)

For a fixed choice of $\sigma$ and $\sigma ' \ne \sigma$, one can see that ${\widehat{u}}(\sigma ,\sigma ';s(\sigma _i,\sigma '_i;h_i))$ is a random variable with respect to the uniform choices of $\sigma _i \ne \sigma$ and $\sigma '_i \ne \sigma _i$, that takes the following values (recall the definition of the function $g_1$ from (39)):

$$\begin{aligned} {\left\{ \begin{array}{ll} g_1(h_i) &{} \text {with probability} \ \frac{q-2}{(q-1)^2} \ , \text {when} \ \sigma _i =\sigma ' \ , \ \sigma '_i \ne \sigma \ , \\ g_1(-h_i) &{} \text {with probability} \ \frac{q-2}{(q-1)^2} \ , \text {when} \ \sigma '_i =\sigma ' \ , \ \sigma _i \ne \sigma \ , \\ -g_1(-h_i) &{} \text {with probability} \ \frac{q-2}{(q-1)^2} \ , \text {when} \ \sigma '_i =\sigma \ , \ \sigma _i \ne \sigma ' \ , \\ h_i &{} \text {with probability} \ \frac{1}{(q-1)^2} \ , \text {when} \ \sigma '_i =\sigma \ , \ \sigma _i = \sigma ' \ , \\ 0 &{} \text {otherwise} \ . \end{array}\right. } \end{aligned}$$

(98)

As $c(1-w_n)\frac{q-2}{(q-1)^2} \rightarrow x_n$ in the regime we are considering, the number of occurrences of the first three cases in (95) is Poissonian of mean $x_n$; on the other hand the number of times the fourth case happens vanishes when q diverges (as 1/q), while the fifth does not contribute to (95), because $f(u_1,\dots ,u_l,0,\dots ,0)=f(u_1,\dots ,u_l)$. We thus see by comparison with (42,43) that the probability distribution $Q_n(h)$ defined in (93) obeys exactly the same recursion equations as the one derived in the main text for the hypergraph bicoloring model, the initial condition $Q_1(h)=\delta (h)$ being also valid here.