Sampling properties of color Independent Component Analysis

Abstract

Independent Component Analysis (ICA) offers an effective data-driven approach for blind source extraction encountered in many signal and image processing problems. Although many ICA methods have been developed, they have received relatively little attention in the statistics literature, especially in terms of rigorous theoretical investigation for statistical inference. The current paper aims at narrowing this gap and investigates the statistical sampling properties of the colorICA (cICA) method. The cICA incorporates the correlation structure within sources through parametric time series models in the frequency domain and outperforms several existing ICA alternatives numerically. We establish the consistency and asymptotic normality of the cICA estimates, which then enables statistical inference based on the estimates. These asymptotic properties are further validated using simulation studies.

Introduction

Independent component analysis (ICA) offers an effective data-driven approach for blind source extraction encountered in many signal and image processing problems. The ICA problem can be formally described by viewing the observed signal matrix $\mathbf{X}$ as a linear combination (mixture) of independent latent random variables (sources) $\mathbf{S}$ so that

$$\underset{M\times T}{\mathbf{X}}=\underset{M\times M}{{\mathbf{W}}^{-1}}\underset{M\times T}{\mathbf{S}},$$

where each column of the mixture $\mathbf{X}$, $\mathbf{X}\left(t\right),t\in \{0,1,\dots ,T-1\}$, is observed at $M$ channels for $T$ time points, the matrix $\mathbf{W}$ is non-random and known as the unmixing matrix, and each column $\mathbf{S}\left(t\right)$ of the source matrix $\mathbf{S}$ contains $M$ unknown independent sources. The objective of ICA is to estimate the unmixing matrix $\mathbf{W}$ to recover the hidden sources through $\mathbf{S}=\mathbf{W}\mathbf{X}$.

There exist many ICA methods including Infomax [5], [27], maximum likelihood estimation [12], JADE [8], fastICA [20], probabilistic ICA [1], estimating score functions [40], kernel [3], [9], smoothing splines [18], logsplines [26], and log-concave projection [38]. In addition, ICA based on two scatter matrices have been developed [23], [33], [35]. For additional extensive literature review, see [13] and [20].

The theoretical properties of the methods mentioned above have been studied. Examples include methods for the fourth moment based ICA algorithms, such as fast ICA and JADE, [32], [34], [39], [42] and semiparametric approaches including [10], [11], [16], [22], [24], [37], [38]. They study $\sqrt{T}$-consistency (in some cases, asymptotic normality) of the unmixing matrix $\mathbf{W}$ under some smooth density assumptions on the underlying sources $\mathbf{S}$. Recently [29] proposed an ICA method based on distance covariance and showed its consistency.

The above papers intrinsically assume that each source is independent and identically distributed. However, it may not be so desirable to apply these instantaneous ICA methods to many practical applications where each independent source is known to possess specific correlation structures. For example, in brain imaging studies, the independent signals usually are auto-correlated. [28] proposed a color ICA (cICA) procedure that makes uses of the spectral density functions of the source signals instead of their marginal density functions. The cICA models the sources using parametric time series models such as autoregressive moving average (ARMA) linear processes and estimates the model parameters as well as the unmixing matrix $\mathbf{W}$ through maximizing the Whittle likelihood [43], obtained in the spectral domain. The cICA numerically outperformed other ICA methods for the auto-correlated sources.

The purpose of this paper is to study the theoretical properties of the cICA to strengthen the cICA application to areas such as statistical inferential issues arising in medical brain imaging studies and signal processing problems. The main obstacle here is the identifiability of the mixing matrix, which has been examined in the i.i.d. [10], [11] and autocorrelated [6] settings using the equivalent class concept. In particular, a second-order blind identification (SOBI) algorithm based on the sample autocovariance matrices was introduced along with some sampling properties [6], [30]. In the context of auto-correlated sources, there are several differences in cICA and SOBI. First, cICA is formulated in the frequency domain while SOBI is in the time domain in which the estimation of the sample autocovariance is not efficient [7]. Second, cICA estimates both the mixing matrix and the color source spectral parameters. In contrast, SOBI used asymptotic properties of the sample auto-covariance matrix to approximate the interference to signal ratio (ISR) [6]. Third, cICA exploits the sampling properties of FFT leading to the Whittle likelihood based estimation of the latent parameters and the associated results will be exact under the Gaussian assumption [14]; while this assumption is a leading case in cICA (that is, for motivation only) but it plays a critical role in developing the SOBI asymptotic properties. Lastly, the Whittle likelihood approach allows one to formulate various statistical inferential procedures using the standard asymptotic theory.

In this paper, we continue to exploit the ideas of cICA and extend the earlier work of [28] with several aims. The first aim is to establish the sampling properties by proving the consistency and asymptotic normality of the cICA estimates, for both the linear process parameters and the entries of the unmixing matrix $\mathbf{W}$. The second aim is to study the theoretical properties of cICA when combined with prewhitening (or sphering), which is routinely performed prior to ICA to reduce computational intensity. This extension over [28] enables one to deal with arbitrary non-orthogonal mixing matrices, while the earlier work only studies scenarios with orthogonal mixing matrices. Finally, we perform extensive simulation studies to provide empirical finite-sample validation for our theoretical results. The desirable numerical properties presented here further support the practical applicability of cICA.

The remainder of the paper is organized as follows. Section 2 provides a brief background on multivariate spectral density estimation, the Whittle likelihood approach for parameter estimation, and the cICA method. Section 3 presents the main theorems that state the $\sqrt{T}$-consistency and asymptotic normality of the cICA estimates. Section 4 reports the simulation studies. All the proofs are provided in Section 5. Some theoretical derivations and computational details can be found in the Supplementary document.