On active learning methods for manifold data

Abstract

Active learning is a major area of interest within the field of machine learning, especially when the labeled instances are very difficult, time-consuming or expensive to obtain. In this paper, we review various active learning methods for manifold data, where the intrinsic manifold structure of data is also incorporated into the active learning query strategies. In addition, we present a new manifold-based active learning algorithm for Gaussian process classification. This new method uses a data-dependent kernel derived from a semi-supervised model that considers both labeled and unlabeled data. The method performs a regularization on the smoothness of the fitted function with respect to both the ambient space and the manifold where the data lie. The regularization parameter is treated as an additional kernel (covariance) parameter and estimated from the data, permitting adaptation of the kernel to the given dataset manifold geometry. Comparisons with other AL methods for manifold data show faster learning performance in our empirical experiments. MATLAB code that reproduces all examples is provided as supplementary materials.

Appendices

Appendix A: Expectation propagation approximation

As discussed before, for GP Classification (GPC), we need to approximate the posterior distribution of latent variables. Nickisch and Rasmussen (2008) provide a comprehensive overview of different methods for approximate inference in GPC model for binary classification. The algorithms they evaluated includes Laplace Approximation, Expectation Propagation, KL-Divergence Minimization, Variational Bounds, Factorial Variational and Markov Chain Monte Carlo. They draw the conclusion that “Expectation Propagation algorithm is almost always the method of choice unless the computational budget is very tight.” Thus, in the SSGP-AL algorithm, Expectation Propagation (EP) is chosen as the approximation method for GPC. In order to simplify notational complexities, we present the standard EP algorithm for supervised GPC. In our paper, we use a data-dependent kernel $\tilde{\mathcal {K}}$ that also utilizes the information in the unlabeled instances, so the posterior distribution will also condition on the graph $\mathcal {G}$.

In EP, the problem of non-Gaussian likelihood function is solved by a local likelihood approximation in the form of an un-normalized Gaussian function in the latent variable $f_{\mathbf{x}_i}$, i.e.,

$$\begin{aligned} P(y_i|f_{x_i}) \simeq t_i (f_{\mathbf{x}_i}|\tilde{Z}_i, \tilde{\mu }_i,\tilde{ \sigma }_i^2)\triangleq \tilde{Z}_i N(f_{\mathbf{x}_i}|\tilde{\mu }_i,\tilde{ \sigma }_i^2) \end{aligned}$$

(37)

where $\tilde{\mu }_i, \tilde{ \sigma }_i^2$ and $\tilde{Z}_i$ are local approximation parameters. After we have the local approximations, the posterior distribution $P(\mathbf{f}_L|Y_L)$ can be approximated by $q(\mathbf{f}_L|Y_L)$, defined as

$$\begin{aligned} q(\mathbf{f}_L|Y_L) \triangleq \frac{\prod _{i=1}^l t_i (f_{\mathbf{x}_i}|\tilde{Z}_i, \tilde{\mu }_i,\tilde{ \sigma }_i^2) N(\mathbf{0}, \mathbf{K}_{LL})}{Z_{EP}} \end{aligned}$$

(38)

Note that $Z_{EP}=q(Y_L)$ is the marginal likelihood (also known as evidence or normalization constant) associated with the approximate posterior distribution $q(\mathbf{f}_L|Y_L)$. Let $\tilde{{\varvec{\mu }}}$ be a vector of $\tilde{\mu }_i$ and ${\tilde{\varvec{\Sigma }}}$ be a diagonal matrix with elements ${\tilde{\Sigma }}_{ii}=\tilde{ \sigma }_i^2$, we can rewrite the product of the independent local approximations $t_i$ as

$$\begin{aligned} \prod _{i=1}^l t_i (f_{\mathbf{x}_i}|\tilde{Z}_i, \tilde{\mu }_i,\tilde{ \sigma }_i^2)=N(\tilde{\mu }, {\tilde{\Sigma }}) \prod _{i=1}^l \tilde{Z}_i \end{aligned}$$

(39)

By the property of product of two Gaussians, it can be easily shown that

$$\begin{aligned} q(\mathbf f_L|Y_L)\sim N(\mathbf \mu , \Sigma ) \end{aligned}$$

(40)

where

$$\begin{aligned} {\varvec{\mu }}=\Sigma \tilde{\Sigma }\tilde{\mu } \text { and } {\varvec{\Sigma }}=(\mathbf{K}_{LL}^{-1}+{\tilde{\varvec{\Sigma }}}^{-1})^{-1}. \end{aligned}$$

(41)

Given the approximate posterior distribution $q(f_L|Y_L)$, one can compute the approximate posterior distribution of latent variables $f_{\mathbf{x}_t}$ at test point $\mathbf{x}_t$, given by

$$\begin{aligned} P(f_{\mathbf{x}_t}|Y_L)= \int P(f_{\mathbf{x}_t}|f_L) P(f_L|Y_L) df_L \approx N(\mu _t, \sigma ^2_t), \end{aligned}$$

(42)

where

$$\begin{aligned} \mu _t= & {} \mathbf{K}_{Lt}^T \mathbf{K}_{LL}^{-1}{\varvec{\mu }} \end{aligned}$$

(43)

$$\begin{aligned} \sigma _t^2= & {} \mathcal {K}(\mathbf{x}_t,\mathbf{x}_t)-\mathbf{K}_{Lt}^T(\mathbf{K}_{LL}^{-1}+{\tilde{\varvec{\Sigma }}})\mathbf{K}_{Lt}\end{aligned}$$

(44)

$$\begin{aligned} \mathbf{K}_{Lt}= & {} [\mathcal {K}(\mathbf{x}_t,\mathbf{x}_1),\ldots ,\mathcal {K}(\mathbf{x}_t,\mathbf{x}_l)]^T \end{aligned}$$

(45)

It can be shown that the predictive probability at test point $\mathbf{x}_t$ is given by

$$\begin{aligned} q(y_t=1|Y_L)= & {} \Phi \left( \frac{\mu _t}{\sqrt{1+\sigma _t^2}}\right) \end{aligned}$$

(46)

$$\begin{aligned}= & {} \Phi \left( \frac{\mathbf{K}_{Lt}^T (\mathbf{K}_{LL}+{\tilde{\varvec{\Sigma }}})^{-1}{{\tilde{\varvec{\mu }}}}}{\sqrt{1+\mathcal {K}(\mathbf{x}_t,\mathbf{x}_t)-\mathbf{K}_{Lt}^T(\mathbf{K}_{LL}+{\tilde{\varvec{\Sigma }}})^{-1}\mathbf{K}_{Lt}}}\right) \end{aligned}$$

(47)

In order to get the posterior distribution $q(\mathbf{f}_L|Y_L)$, we first need to compute the local approximation $t_i$ and its corresponding parameters $\tilde{\mu }_i, \tilde{ \sigma }_i^2, \tilde{Z}_i$. The key idea behind the EP algorithm is to update the individual $t_i$ approximations sequentially. We start with some current posterior approximation $q(f_{\mathbf{x}_i}|Y_L)\sim N(\mu _i, \sigma _i^2)$, and after leaving out the local approximation $t_i$, one obtains the so-called cavity distribution

$$\begin{aligned} q_{-i}(f_{\mathbf{x}_i})\triangleq N(f_{\mathbf{x}_i}|\mu _{-i}, \sigma _{-i}^2) \end{aligned}$$

(48)

where

$$\begin{aligned} \mu _{-i}= & {} \sigma _{-i}^2 \mu _i - \tilde{\sigma }_i^{-2}\tilde{\mu }_i ) \end{aligned}$$

(49)

$$\begin{aligned} \sigma _{-i}^2= & {} (\sigma _i^{-2}-\tilde{\sigma }_i^{-2})^{-1} \end{aligned}$$

(50)

This result can be easily checked by multiplying Eqs. (37) and (48) to recover $q(f_{\mathbf{x}_i}|Y_L)$. Next combine the cavity distribution $q_{-i}(f_{\mathbf{x}_i})$ with the exact likelihood function $P(y_i|f_{\mathbf{x}_i})$ to obtain a non-Gaussian distribution which can then be approximated by a desired Gaussian distribution $\hat{q}(f_{\mathbf{x}_i})$

$$\begin{aligned} q_{-i}(f_{\mathbf{x}_i})P(y_i|f_{\mathbf{x}_i})\simeq \hat{Z}_i N(\hat{\mu }_i,\hat{\sigma }_i)\triangleq \hat{q}(f_{\mathbf{x}_i}). \end{aligned}$$

(51)

Finally, we compute the local approximation parameters $\tilde{\mu }_i, \tilde{ \sigma }_i^2, \tilde{Z}_i$ of the approximation $t_i$ by minimizing the Kullback–Leibler (KL) divergence between $\hat{q}(f_{\mathbf{x}_i})$ and $q_{-i}(f_{\mathbf{x}_i})t_i$. It can be shown this minimization is equivalent to matching the first and second moments of these two distributions. The zeroth-order (normalization constant), first-order and second-order moments of the distribution $\hat{q}(f_{\mathbf{x}_i})$ can be shown to be

$$\begin{aligned} \hat{Z}_i= & {} \Phi (z_i) \end{aligned}$$

(52)

$$\begin{aligned} \hat{\mu }_i= & {} \mu _{-i}+\frac{y_i \sigma _{-i}^2 N(z_i)}{\Phi (z_i) \sqrt{1+\sigma _{-i}^2}} \end{aligned}$$

(53)

$$\begin{aligned} \hat{\sigma }_i^2= & {} \sigma _{-i}^2 - \frac{\sigma _{-i}^4 N(z_i)}{(1+\sigma _{-i}^2)\Phi (z_i)}\left( z_i+\frac{N(z_i)}{\Phi (z_i)}\right) \end{aligned}$$

(54)

where $z_i=y_i \mu _{-i}/\sqrt{(1+\sigma _{-i}^2)}$. From matching the above moments, the local approximation parameters of $t_i$ are:

$$\begin{aligned} \tilde{\mu }_i= & {} \tilde{\sigma }_i^2 (\hat{\sigma }_i^{-2}\hat{\mu }_i - \sigma _{-i}^{-2} \mu _{-i}) \end{aligned}$$

(55)

$$\begin{aligned} \tilde{\sigma }_i^2= & {} (\hat{\sigma }_i^{-2}-\sigma _{-i}^{-2})^{-1} \end{aligned}$$

(56)

$$\begin{aligned} \tilde{Z}_i= & {} \Phi (z_i) \sqrt{2\pi } \sqrt{\sigma _{-i}^2+\tilde{\sigma }_i^2}\mathrm {exp}\left( \frac{(\mu _{-i}-\tilde{\mu }_i)^2}{2(\sigma _{-i}^2+\tilde{\sigma }_i^2)}\right) \end{aligned}$$

(57)

After updating the site parameters, the approximate posterior distribution $q(\mathbf{f}_L|Y_L)$ is updated using Eq. (41). The pseudo-code for the EP algorithm is provided in Algorithm 2.

Appendix B: Estimation of the GP hyperparameters

In the new approach presented in Sect. 4, we estimate the GP hyperparameters in the data-based kernel $\tilde{\mathcal {K}}$ by maximizing the logarithm of $\tilde{Z}_{EP}$ in

$$\begin{aligned} q(\mathbf{f}_L|Y_L,\mathcal {G}) \triangleq \frac{\prod _{i=1}^l t_i (f_{\mathbf{x}_i}|\tilde{Z}_i, \tilde{\mu }_i,\tilde{ \sigma }_i^2) N(\mathbf{0}, \tilde{\mathbf{K}}_{LL})}{\tilde{Z}_{EP}} \end{aligned}$$

(58)

where $\tilde{Z}_{EP}$ is an approximation for marginal likelihood $P(Y_L|\mathcal {G})$ in (31) by using Expectation Propagation. Note that $\tilde{Z}_{EP}=q(Y_L|\mathcal {G})$ is the marginal likelihood associated with the approximate posterior distribution $q(\mathbf{f}_L|Y_L,\mathcal {G})$. Let $\tilde{{\varvec{\mu }}}$ be a vector containing the $\tilde{\mu }_i$ and $\tilde{\varvec{\Sigma }}$ be a diagonal matrix with elements $\tilde{\varvec{\Sigma }}(i,i)=\tilde{ \sigma }_i^2$. We can rewrite the product of the independent local approximations $t_i$ as

$$\begin{aligned} \prod _{i=1}^l t_i (f_{\mathbf{x}_i}|\tilde{Z}_i, \tilde{\mu }_i,\tilde{ \sigma }_i^2)=N(\tilde{\mu }, \tilde{\varvec{\Sigma }}) \prod _{i=1}^l \tilde{Z}_i \end{aligned}$$

(59)

Based on the Expectation Propagation algorithm, the approximate marginal likelihood $\tilde{Z}_{EP}$ is given by

$$\begin{aligned} \tilde{Z}_{EP} = \int q(Y_L,\mathbf{f}_L |\mathcal {G} ) d\mathbf{f}_L = \int P(\mathbf{f}_L | \mathcal {G}) \prod _{i=1}^l t_i(f_{\mathbf{x}_i}|\tilde{Z_i}, \tilde{\mu }_i, \tilde{\sigma }_i^2) d\mathbf{f}_L \end{aligned}$$

(60)

Given $P(\mathbf {f_L}|\mathcal {G}) \sim N(\mathbf {0}, {\tilde{\varvec{K}}}_{LL})$ and Eq. (59), and using the property of the product of two multivariate Gaussians, it can be shown that:

$$\begin{aligned} \tilde{Z}_{EP}=Z_0^{-1} \prod _{i=1}^l \tilde{Z}_i \int N(\mathbf{f}_L | {\varvec{\omega }} , {\varvec{\Omega }} ) \mathbf{f}_L =Z_0^{-1} \prod _{i=1}^l \tilde{Z}_i \end{aligned}$$

(61)

where

$$\begin{aligned} Z_0^{-1}= & {} (2\pi )^{l/2} |{\tilde{\mathbf {K}}}_{LL}+{\tilde{\varvec{\Sigma }}}|^{-1/2} \mathrm {exp}\left( -\frac{1}{2}{\tilde{\mu }}^\top ({\tilde{\mathbf {K}}}_{LL}+{\tilde{\varvec{\Sigma }}})^{-1}{\tilde{\mu }}\right) \end{aligned}$$

(62)

$$\begin{aligned} {\varvec{\omega }}= & {} {\varvec{\Omega }}{\tilde{\varvec{\Sigma }}}^{-1} {\tilde{\mu }} \end{aligned}$$

(63)

$$\begin{aligned} {\varvec{\Omega }}= & {} ({\tilde{\mathbf {K}}}_{LL}^{-1}+{\tilde{\varvec{\Sigma }}}^{-1})^{-1} \end{aligned}$$

(64)

and $\int N(\mathbf{f}_L | {\varvec{\omega }} , {\varvec{\Omega }} ) \mathbf{f}_L =1$ by the definition of multivariate Gaussian PDF. Then the log-likelihood yields

$$\begin{aligned} \log \tilde{Z}_{EP}= & {} \mathrm {log} (Z_0^{-1})+\sum _{i=1}^l \mathrm {log }(\tilde{Z}_i) \end{aligned}$$

(65)

$$\begin{aligned}= & {} -\frac{1}{2} \mathrm {log} |\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }}|-\frac{1}{2}\tilde{\mu }^\top (\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }})^{-1}\tilde{\mu } +\sum _{i=1}^l \mathrm {log} \; \Phi \left( \frac{y_i \mu _{-i}}{\sqrt{1+\sigma ^2_{-i}}}\right) \nonumber \\&+\frac{1}{2}\sum _{i=1}^l \mathrm {log}\left( \sigma ^2_{-i}+\tilde{\sigma }^2_i\right) +\sum _{i=1}^l \frac{(\mu _{-i}-\tilde{\mu }_i)^2}{2(\sigma _{-i}^2+\tilde{\sigma }_i^2)} \end{aligned}$$

(66)

where the explicit form of $\tilde{Z}_i$ is derived in the EP process, $\mu _{-i}$ and $\sigma _{-i}$ are the parameters in the cavity distribution.

Optimization of $\log \tilde{Z}_{EP}$ with respect to the hyperparameters of a data-based kernel function $\tilde{\mathcal {K}}$ requires evaluation of the partial derivatives from Eq. (66). Seeger (2005) demonstrated that the derivatives of the last three terms in Eq. (66) with respect to the hyperparameters vanish. As a result, only the derivatives of the first two terms need to be considered.

Let ${\varvec{\theta }}=\{\theta _j\}_{j=1}^m$ be the collection of hyperparameters in the data-based kernel $\tilde{\mathcal {K}}$. Given Eq. (66), the derivative of log-likelihood $\log \tilde{Z}_{EP}$ with respect to hyperparameter $\theta _j$ can be reduced to

$$\begin{aligned} \frac{\partial \log \tilde{Z}_{EP}}{\partial \theta _j}= & {} \frac{\partial }{\partial \theta _j}\left( -\frac{1}{2} \mathrm {log} |\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }}|-\frac{1}{2}\tilde{\mu }^\top (\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }})^{-1}\tilde{\mu }\right) \end{aligned}$$

(67)

$$\begin{aligned}= & {} -\frac{1}{2} {{\,\mathrm{tr}\,}}\left( (\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }})^{-1}\frac{\partial \tilde{\mathbf {K}}_{LL}}{\partial \theta _j}\right) \nonumber \\&+\frac{1}{2}\tilde{\mu }^\top (\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }})^{-1}\frac{\partial \tilde{\mathbf {K}}_{LL}}{\partial \theta _j}(\tilde{\mathbf {K}}_{LL}+\tilde{\varvec{\Sigma }})^{-1}\tilde{\mu } \end{aligned}$$

(68)

In this paper, we choose a radial basis function (RBF) kernel as the base kernel $\mathcal {K}$ in Eq. (29), i.e.,

$$\begin{aligned} \mathcal {K} (\mathbf{x}_i, \mathbf{x}_j)=\exp \left( -\frac{(\mathbf{x}_i-\mathbf{x}_j)^\top (\mathbf{x}_i - \mathbf{x}_j)}{2\sigma ^2_{\mathrm {rbf}}}\right) \end{aligned}$$

(69)

Thus, note there are two hyperparameters in the data-based kernel $\tilde{\mathcal {K}}$, which include the regularization parameter $\lambda _R$ and the range (or length-scale) parameter $\sigma _{\mathrm {rbf}}$ in the RBF kernel $\mathcal {K}$. Both of the hyperparameters are optimized under the log-scale. The derivatives of Gram matrix $\tilde{\mathbf {K}}$ with respect to $\log \sigma _{\mathrm {rbf}}$ and $\log \lambda _R$ are derived as

$$\begin{aligned} \frac{\partial \tilde{\mathbf {K}}}{\partial (\log \sigma _{\mathrm {rbf}})}= & {} (\mathbf {I}+\lambda _R \mathbf {K} L)^{-1} \frac{\partial \mathbf {K}}{\partial (\log \sigma _{\mathrm {rbf}})} (\mathbf {I}+\lambda _R L \mathbf {K} )^{-1} \end{aligned}$$

(70)

$$\begin{aligned} \frac{\partial \tilde{\mathbf {K}}}{\partial (\log \lambda _R)}= & {} -(\mathbf {K}^{-1}+\lambda _R L)^{-1} \lambda _R L (\mathbf {K}^{-1}+\lambda _R L)^{-1} \end{aligned}$$

(71)

and for the RBF kernel $\mathcal {K}$, one can easily get

$$\begin{aligned} \frac{\partial \mathbf {K}}{\partial (\log \sigma _{\mathrm {rbf}})} =-2\mathbf {K} \circ \log \mathbf {K} \end{aligned}$$

(72)

where $\circ$ is the Hadamard product between two matrices, and $\log \mathbf {K}$ is a notation that represents taking logarithm of each element in matrix $\mathbf {K}$ (not the logarithm of matrix $\mathbf {K}$). Note that in (68) $\partial \tilde{\mathbf {K}}_{LL} / \partial \theta _j$ is just a submatrix, associated with the labeled data $\mathbf {X}_L$, of $\partial \tilde{\mathbf {K}} / \partial \theta _j$, since the marginal likelihood $\tilde{Z}_{EP}$ can only be computed for the labeled data.

In summary, the hyperparameters $\lambda _R$ and $\sigma _{\mathrm {rbf}}$ in the data-based kernel $\tilde{\mathcal {K}}$ can both be estimated by maximizing the log-likelihood (66). This optimization problem can be solved by using a gradient-based algorithm with the derivatives provided in Eqs. (70–72).