Feature screening based on distance correlation for ultrahigh-dimensional censored data with covariate measurement error

Abstract

Feature screening is an important method to reduce the dimension and capture informative variables in ultrahigh-dimensional data analysis. Its key idea is to select informative variables using correlations between the response and the covariates. Many methods have been developed for feature screening. These methods, however, are challenged by complex features pertinent to the data collection as well as the nature of the data themselves. Typically, incomplete response caused by right-censoring and covariate measurement error are often accompanying with survival analysis. Even though many methods have been proposed for censored data, little work has been available when both incomplete response and measurement error occur simultaneously. In addition, the conventional feature screening methods may fail to detect the truly important covariates that are marginally independent of the response variable due to correlations among covariates. In this paper, we explore this important problem and propose the model-free feature screening method in the presence of the censored response and error-prone covariates. In addition, we also develop the iteration method to improve the accuracy of selecting all important covariates. Numerical studies are reported to assess the performance of the proposed method. Finally, we implement the proposed method to a real dataset.

Appendices

Appendix A Proof of Theorem 3.1

We first consider $\text {dcov} \left( Y^*, X_k \right)$ and $\text {dcov}^*\left( Y^*, X_k^*\right)$ for the kth component of X with $k=1,\ldots ,p$. Note that the former formulation is based on the true covariates $X_k$, while the latter formulation is based on the surrogate covariates $X_k^*$.

Since the error term $\epsilon _k$, the kth component of $\epsilon$, follows a normal distribution $N(0,\sigma _{\epsilon ,kk})$, then its characteristic function is given by

$$\begin{aligned} E\left\{ \exp \left( \mathbf {i} s \epsilon _k \right) \right\} = \exp \left( -\frac{1}{2} s^2 \sigma _{\epsilon ,kk} \right) . \end{aligned}$$

(A.1)

By the direct derivations, we have

$$\begin{aligned} \phi ^*_{X_k^*}(s)= & {} E\left\{ \exp \left( \mathbf {i}s X_k^*\right) \right\} \exp \left( \frac{1}{2} s^2 \sigma _{\epsilon ,kk} \right) \nonumber \\= & {} E\left\{ \exp \left( \mathbf {i}s X_k \right) \right\} E\left\{ \exp \left( \mathbf {i}s \epsilon _k \right) \right\} \exp \left( \frac{1}{2} s^2 \sigma _{\epsilon ,kk} \right) \nonumber \\= & {} E\left\{ \exp \left( \mathbf {i}s X_k \right) \right\} , \end{aligned}$$

(A.2)

where the second equality is due to the independence of $X_k$ and $\epsilon _k$, and the last equality is due to (A.1).

In addition, we can also derive

$$\begin{aligned} \phi ^*_{Y^*,X_k^*}(r,s)= & {} E\left\{ \exp \left( \mathbf {i}rY^*+ \mathbf {i}s X_k^*\right) \right\} \exp \left( \frac{1}{2} s^2 \sigma _{\epsilon ,kk} \right) \nonumber \\= & {} E\left\{ \exp \left( \mathbf {i}rY^*+ \mathbf {i}s X_k \right) \right\} E\left\{ \exp \left( \mathbf {i}s \epsilon _k \right) \right\} \exp \left( \frac{1}{2} s^2 \sigma _{\epsilon ,kk} \right) \nonumber \\= & {} E\left\{ \exp \left( \mathbf {i}rY^*+ \mathbf {i}s X_k \right) \right\} , \end{aligned}$$

(A.3)

where the second equality is due to the independence of $\epsilon _k$ and $(X_k, Y^*)$, and the last equality again comes from (A.1). As a result, combining (A.2) and (A.3) with $\text {dcov}^*\left( Y^*, X_k^*\right)$ gives the same expression as $\text {dcov} \left( Y^*, X_k \right)$.

The equivalence of $\text {dcov}^*\left( X_k^*, X_k^*\right)$ and $\text {dcov} \left( X_k, X_k \right)$ holds by the similar derivations. Therefore, we conclude that $\text {dcorr} \left( Y^*, X_k \right)$ and $\text {dcorr}^*\left( Y^*, X_k^*\right)$ are equivalent in the sense that $\text {dcorr} \left( Y^*, X_k \right) > 0$ if and only if $\text {dcorr}^*\left( Y^*, X_k^*\right) > 0$. Consequently, the same active features can be determined for $X^*$ and X. $\square$

Appendix B Overview of error correction in the Cox model

In this appendix, we outline the strategy of correcting covariate measurement error when fitting the Cox model. This idea comes from Chen and Yi (2020), and this method can be used to fit the Cox model with covariate measurement error in Sect. 4.3.

For $i=1,\ldots ,n$, let $Y_i$ and $\delta _i$ denote the survival time and the censoring indicator defined in Sect. 2.1. Let $X_i$ denote q-dimensional vector of unobserved covariates with $q<n$ after feature screening procedure, and let $X_i^*$ be the surrogate version of $X_i$. Based on the Cox model (17) and the unobserved covariates $X_i$, the likelihood function is given by (e.g., Lawless 2003)

$$\begin{aligned} L(\gamma ) = \prod \limits _{i=1}^{n} \left\{ \lambda _0(Y_i) \exp \left( X_i^\top \gamma \right) \right\} ^{\delta _i} \exp \left\{ - \Lambda _0(Y_i) \exp \left( X_i^\top \gamma \right) \right\} , \end{aligned}$$

(B.1)

where $\Lambda _0(t) = \int _0^t \lambda _0(u)du$ is called the cumulative baseline hazards function.

Let $\ell (\gamma ) = \log L(\gamma )$. Since $\ell (\gamma )$ contains the $X_i$ whose measurements are unavailable, we want to modify $\ell (\gamma )$ to be a new function, say $\ell ^*(\gamma )$, of the observed measurements and the model parameters so that its conditional expectation equals to $\ell (\gamma )$:

$$\begin{aligned} E\left\{ \ell ^*(\gamma )|\mathbb {X},\mathbb {C},\mathbb {T}\right\} = \ell (\gamma ), \end{aligned}$$

(B.2)

where the expectation is taken with respect to the conditional distribution of $\mathbb {X}^*$ given $\left\{ \mathbb {X}, \mathbb {C}, \mathbb {T} \right\}$, where $\mathbb {X}^*= \{X_1^*,\ldots ,X_n^*\}$, $\mathbb {X} = \{X_1,\ldots ,X_n\}$, $\mathbb {C} = \{C_1,\ldots ,C_n\}$, and $\mathbb {T} = \{T_1,\ldots ,T_n\}$. Such a strategy is useful in yielding an unbiased estimating function and is sometimes called the “corrected” likelihood method or the insertion correction approach (e.g., Carroll et al. 2006, Section 7.4).

Noticing that the $X_i$ appear in $\ell (\gamma )$ in linear and exponential forms, we define

$$\begin{aligned} \ell ^*(\gamma )= & {} \sum _{i=1}^{n} \bigg [ \delta _i \log \lambda _0 (Y_i) + \delta _i (X_i^{*\top } \gamma ) - \Lambda _0 (Y_i) \exp (X_i^{*\top } \gamma ) \left\{ m(\gamma )\right\} ^{-1} \bigg ],\qquad \end{aligned}$$

(B.3)

where $m(z) = \exp (\frac{1}{2} z^\top \Sigma _{\epsilon } z)$ and $\Sigma _{\epsilon }$ is defined in Sect. 2.2. It is easily seen that $\ell ^*(\gamma )$ satisfies (B.2).

To use (B.3) to derive an estimator of $\gamma$, we need to deal with the baseline hazard function $\lambda _0 \left( \cdot \right)$ and its cumulative function $\Lambda _0 \left( \cdot \right)$. First, we discretize $\Lambda _0 \left( \cdot \right)$ so that $\lambda _0 \left( \cdot \right)$ has a nonzero value if $t = Y_i$ for $i = 1,\ldots ,n$; otherwise, $\lambda _0 (t) =0$. Let $\lambda _i$ denote $\lambda _0 (Y_i)$ for $i = 1,\ldots ,n$. Then $\Lambda _0 (t)$ is taken as $\sum \nolimits _{i=1}^{n} \mathbb {I}(Y_i \leqslant t) \lambda _i$. Next, given $\gamma$, we solve $\frac{\partial \ell ^*(\gamma )}{\partial \lambda _i} = 0$ for $i = 1,\ldots ,n$, which leads to an estimator of $\lambda _i$, given by

$$\begin{aligned} \widehat{\lambda }_i = \frac{\delta _i}{\sum \limits _{k=1}^{n} \mathbb {I}(Y_i \le Y_k) \exp { \left( X_k^{*\top } \gamma \right) } \left\{ m(\gamma ) \right\} ^{-1}}\ \text{ for } i=1,...,n; \end{aligned}$$

(B.4)

and the corresponding estimate of the cumulative baseline hazards function:

$$\begin{aligned} \widehat{\Lambda }_0 (t) = \sum _{i=1}^{n} \mathbb {I}(Y_i \le t) \widehat{\lambda }_i \ . \end{aligned}$$

(B.5)

Finally, plugging (B.4) and (B.5) into (B.3) gives the function

$$\begin{aligned} \widehat{\ell }^*(\gamma )= & {} \sum \limits _{i=1}^{n} \left[ \delta _i \log \widehat{\lambda }_i + \delta _i (X_i^{*\top } \gamma ) - \widehat{\Lambda }_0 (Y_i) \exp (X_i^{*\top } \gamma ) \left\{ m(\gamma )\right\} ^{-1} \right] . \end{aligned}$$

An estimator of $\gamma$ is then obtained by maximizing $\widehat{\ell }^*(\gamma )$:

$$\begin{aligned} \widehat{\gamma } = {\mathop {\mathrm{argmax}}\limits _{\gamma }} \widehat{\ell }^*(\gamma ). \end{aligned}$$