Phase detection with neural networks: interpreting the black box

We consider supervised learning problems with labeled training data $\mathcal{D}={\left\{{z}_{i}\right\}}_{i=0}^{n}$, where z_i = (x_i, y_i). The input data is coming from some input space ${x}_{i}\in \mathcal{X}$, and the model predicts the outputs coming from some output space ${y}_{i}\in \mathcal{Y}$. In our setup, the inputs x_i are the state vectors for a given physical system, and y_i are the corresponding phase labels. The model is determined by the set of parameters θ. In the training process, the parameters' space is being searched for the final parameters ${\hat{\theta }}_{\mathcal{D}}\equiv \hat{\theta }$ of the ML model, which minimize the training loss function $\mathfrak{L}\left(\mathcal{D},\theta \right)=\frac{1}{n}{\sum }_{z\in \mathcal{D}}\mathcal{L}\left(z,\theta \right)$. The training data set size, n, tends to be of the order of thousands. After training, a model can make a prediction for an unseen test point, z_test, with the test loss function value, $\mathcal{L}\left({z}_{\text{test}},\hat{\theta }\right)$, related to the model certainty of this prediction.

An intuitive way of unraveling the logic learned by the machine is retraining the model after removing a single training point z_r (starting from the same minimum, if a non-convex problem is analyzed), and checking how it changes the prediction of a specific test point z_test. Such a leave-one-out training (LOO) [45] studies the change of the parameters θ, now shifted to a new minimum ${\hat{\theta }}_{\mathcal{D}{\backslash}\left\{{z}_{\text{r}}\right\}}$ of the loss function, as depicted in figure 1(a). The largest test loss changes, ${\Delta}\mathcal{L}\equiv \mathcal{L}\left({z}_{\text{test}},\hat{\theta }\right)-\mathcal{L}\left({z}_{\text{test}},{\hat{\theta }}_{\mathcal{D}{\backslash}\left\{{z}_{\text{r}}\right\}}\right)$, indicate the most influential training points for a given test point z_test being the ones whose removal causes the largest change. Influential examples can be both helpful (${\Delta}\mathcal{L}{ >}0$) and harmful (${\Delta}\mathcal{L}{< }0$). Such an analysis gives the notion of a similarity used by the machine in a given problem, as training points being the closest in the ${\Delta}\mathcal{L}$ space can be understood as the most similar. Once the most influential points are indicated, we can decode what characteristics are looked at by comparing 'similar' points in the machine's 'understanding'. It can be especially useful in phase classification problems where the analysis of ${\Delta}\mathcal{L}$ enables the recovery of patterns being crucial for distinguishing the phases. However, this technique is prohibitively expensive, as the model needs retraining for each removed z.

Zoom In Zoom Out Reset image size

To circumvent this problem, one can make a Taylor expansion of the loss function $\mathcal{L}$ w.r.t. the parameters around the minimum $\hat{\theta }$ and approximate ${\Delta}\mathcal{L}$ resulting from the LOO training, as presented in figure 1(a). This method was proposed for regression problems already forty years ago [45–47] and named influence functions. This interpretability method is not only computationally feasible but also correctly treats a model as a function of training data. The influence function reads

$$\begin{equation*}\mathcal{I}\left({z}_{\text{r}},{z}_{\text{test}}\right)=\frac{1}{n}{\nabla }_{\theta }\mathcal{L}{\left({z}_{\text{test}},\hat{\theta }\right)}^{T}{H}_{\theta }^{-1}\left(\hat{\theta }\right){\nabla }_{\theta }\mathcal{L}\left({z}_{\text{r}},\hat{\theta }\right),\end{equation*}$$

and it estimates ${\Delta}\mathcal{L}$ for a chosen test point z_test after the removal of a chosen training point z_r. ${\nabla }_{\theta }\mathcal{L}\left({z}_{\text{test}},\hat{\theta }\right)$ is the gradient of the loss function of the single test point, ${\nabla }_{\theta }\mathcal{L}\left({z}_{\text{r}},\hat{\theta }\right)$ is the gradient of the loss function of the single training point whose removal's impact is being approximated, and ${H}_{\theta }^{-1}\left(\hat{\theta }\right)$ is the inverse of Hessian, ${H}_{i,j}\left(\hat{\theta }\right)=\frac{{\partial }^{2}}{{\partial }_{{\theta }_{i}}{\partial }_{{\theta }_{j}}}\mathfrak{L}\left(\mathcal{D},\theta \right){\vert }_{\theta =\hat{\theta }}$. All derivatives are calculated w.r.t. the model parameters θ, evaluated at $\hat{\theta }$ corresponding to the minimum of the loss, $\mathfrak{L}\left(\mathcal{D},\hat{\theta }\right)$. We can only ensure the existence of the inverse of the Hessian if it is positive-definite. It is rarely the case with more sophisticated ML models such as NNs, whose loss landscape is highly non-convex and whose local minima are dominantly flat [48]. However, Koh et al showed that this method can be generalized to such minima and therefore applied to ML [49, 50]. The example code can be found in Ref. [51].

We apply influence functions to a small CNN (see appendix B for the architecture) trained to recognize phases in the extended Hubbard model, namely a 1D system consisting of spinless fermions at half-filling. The Hubbard models are of fundamental importance to the condensed-matter physics, with the two-dimensional Fermi–Hubbard model believed to describe the high-temperature superconductivity of cuprates [52]. The chosen 1D system has the advantage of being within the power of efficient numerical simulations. As a result, it has a rich and well-studied phase diagram [53, 54] and is a promising candidate to be simulated in quantum simulator [52]. As such, it is suitable to benchmark the influence functions (or any interpretability method) in phase classification problems. In this model, fermions hop between neighboring sites with amplitudes J and interact with nearest neighbors with strength V₁ and next-nearest neighbors with strength V₂

$$\begin{equation}\hat{H}=-J\sum _{\langle i,j\rangle }{c}_{i}^{{\dagger}}{c}_{j}+{V}_{1}\sum _{\langle i,j\rangle }{n}_{i}{n}_{j}+{V}_{2}\sum _{\langle \langle i,j\rangle \rangle }{n}_{i}{n}_{j}.\end{equation} \tag{ 1 }$$

The competition between the system parameters J, V₁, and V₂ leads to four different phases: gapless Luttinger liquid (LL), two gapped charge-density-wave phases with density patterns 1010 (CDW-I) and 11001100 (CDW-II), and bond-order (BO) phase, as seen in figure 1(b). The order parameter describing the transition to the CDW-I (-II) phase is the average difference between (next-)nearest-site densities. Staggered effective hopping amplitudes characterize the BO phase. We feed the CNN with ground states expressed in the Fock basis, labeled with their appropriate phases, calculated for a 12-site system (see appendix A for the details). The hopping amplitude, J, is set to 1 throughout the paper.

We train a CNN to classify ground states into two phases: LL and CDW-I based on the transition line marked with the arrow (1) in figure 1(b) for V₂ = 0. We plot the influence functions of all training examples for a chosen test point (marked with orange line) in figure 2. The order parameter describing the transition here is the average difference between nearest-site densities, which is zero in the LL phase and non-zero (growing to one) in the CDW-I phase.

Zoom In Zoom Out Reset image size

The panels (a) and (b) present how influential training points are for test points from the LL phase. The test state (a) is the ground state located deeply in the LL phase, while (b) is closer to the transition. If the CNN learns an order parameter, all training points, i.e., ground states from the LL phase exhibiting a zero order parameter, should be similarly positively influential, and that is precisely what we observe. They form an almost flat line in panels (a) and (b). For both test points (a) and (b) from the LL phase, the most harmful training points are the ones closest to the transition, but on the CDW-I side. These states are the most similar (with the smallest order parameter value), but already labeled differently.

A careful reader can notice that if the CNN learns an order parameter, the training points from the LL phase, all exhibiting a zero order parameter, should be similarly influential and form a flat line in all the panels of figure 2. However, we see that in reality, their influence changes linearly, what panel (c) shows especially well. This divergence from expected behavior is mostly due to numerical reasons, and we discuss it in appendix A.

On the side of the CDW-I phase, the influence pattern is significantly different. The curvature of influential points corresponds to the growth of the order parameter, and the most influential helpful points are the ones closest to the test point in the order parameter space, slightly shifted towards the transition point, as they provide more information. Panel (c) shows the influence functions of training points for the test states on the CDW-I side, close to the transition. The most harmful examples are, as in the previous test points, the ones closest to the transition, but on its other side. However, panel (d) presents a distinct behavior of the most harmful examples being in the same phase. All the training points are similarly influential with small values of influence functions resulting in the almost flat line. It is a signature of the CNN's high certainty regarding the prediction made in panel (d) manifesting with a small test loss function $\mathcal{L}\left({z}_{\text{test}},\hat{\theta }\right)$. Also, the analyzed test point is deeply in the CDW-I phase, with all neighboring states being almost identical with the order parameter close to 1. The most harmful examples are the ones we label as the CDW-I phase, but very different, so the ones closest to the transition.

While analyzing the figures, it is vital to keep in mind that we do not explicitly provide any information on the nearest-neighbor interaction, V₁/J, present on the x-axis (or any physical parameters, in general). We provide the input states in the random order. Therefore, the smooth patterns created by the influence functions and resulting ordering of training points, especially on the CDW-I phase's side, is the sole consequence of the internal analysis of the states by the machine.

With a similar approach, we validate the transfer learning to another transition line. We take the trained CNN from figure 2, and in figure 3 we apply it to test states coming from the transition line for V₂ = 0.25V₁, where the next-nearest-neighbor interaction shifts phase transition to higher values of V₁/J. Therefore the training and test states come from different transition lines, V₂ = 0 and 0.25V₁, marked in figure 1(b) with the arrows (1) and (2), respectively. Notice the shift of the panels' backgrounds as compared to figure 2. They mark two phases of the test transition line, having a different transition point (V₁/J = 1.85) than the training transition line (V₁/J = 1).

Zoom In Zoom Out Reset image size

Panels (a) and (b) of figure 3 show the influence function values of training data for test states from the LL phase, while (c) and (d)—from the CDW-I phase. Panel (a) is identical to the panel (a) of figure 2, but already panel (b) shows an interesting divergence from the figure 2(b), being a result of a shifted transition point of the test line compared to the training line. No longer the same value of V₁/J, for which test and training states were calculated, yields the same order parameter for both of them. For example, the test state being in the LL phase, close to the transition point for V₂ = 0.25V₁ should be the most similar to the training points from the LL phase, close to the transition point V₂ = 0, and have the most similar order parameter. The ML algorithm follows this similarity with regards to the order parameter, what implies a successful transfer learning scheme. We see similar behavior in panel (c), where the most helpful points are also shifted as compared to figure 2(c).

This time we analyze the transition line crossing three phases, LL, BO, and CDW-II, which is indicated by the arrow (3) in figure 1(b). Two order parameters describe this transition. One is the average difference of the next-nearest neighbor density, which equals zero in the LL and BO phases, and grows to 1 in the CDW-II phase. The other is the staggering of effective nearest-neighbor hoppings, being 0 in the LL phase, non-zero in the BO phase, and slowly decaying to 0 in the CDW-II phase. In the studied range of parameters, two phases (BO and CDW-II) co-exist (see appendix A for the details). It is crucial to note that in this section, we train on the mentioned transition line crossing three phases, but we label ground states only as belonging to one out of two phases.

In the first set-up, with results presented in the panels (a) and (b) of figure 4, we label ground states as belonging to the LL (blue dots, label 0) or the BO and CDW-II phases (purple dots, label 1). Independently on the test point location, notice two similarity regions within purple training points. Apparently the NN learns two different patterns (order parameters) to classify the data correctly. Therefore, it notices the existence of the third phase within the incorrectly labeled data. Inferring the third phase would be impossible without interpretability methods, which in this sense pave the way towards unknown phases detection.

Zoom In Zoom Out Reset image size

The second set-up consists of labeling the same data as belonging to the LL and BO phases (blue dots, label 0) or the CDW-II phase (purple dots, label 1). The influence functions' values, resulting from this classification, are in the panels (c) and (d) of figure 4. The pattern they form is starkly different. First of all, there is no additional similarity region within training points from the LL and BO. The behavior is then more similar to the one seen in figure 2 with the transition between LL and CDW-I. It is not identical, though, as in the phase LL + BO the most helpful training points are always distributed randomly, but deep in the LL phase, avoiding the BO phase. The most helpful points on the CDW-II side are deep in the CDW-II phase in contrast to figure 2, where they mostly follow the test point. Consider, that the deeper the CDW-II phase, the smaller the BO order parameter, what makes CDW-II predictions easier. The observed pattern is the example of NN not learning correctly the order parameter and potentially overfitting.

Finally, we trained a CNN on the same data, but with three labels correctly corresponding to all three phases. The influence patterns resemble those seen in figure 2 and panels (c) and (d) of figure 4, indicating that CNN correctly learns both appropriate order parameters.