Uniform joint screening for ultra-high dimensional graphical models
Introduction
Rapidly developed technologies have brought in large amounts of data with numerous variables in many research areas including genomics, social networks, computer visions and so on. It is of great practical interest to explore how these variables are related to each other. Under such circumstances, graphical models provide a general framework to characterize the conditional dependence structures among these nodes and are widely used in many applications owing to their descriptive simplicity [17], [21], [30]. Therefore, recovering the graphical structure in a scalable way is of urgent need in the big data era.
A large amount of literature has been devoted to recover the conditional dependence structures in graphical models. When the conditional independence can be characterized by zero entries in the precision matrix, such as in Gaussian graphical models, there are mainly two categories of methods for estimating the graphical structures. The first class optimizes the penalized likelihood or the penalized empirical risk with -penalty or some non-convex penalty to estimate the precision matrix. See, for instance, [8], [13], [14], [25], [34], [35], among many others. The second class of methods decompose the estimation of graphical structures into a series of nodewise regressions. That is, fitting sparse regressions for each node by treating the others as predictors [2], [3], [23], [24], [27], [28], [33]. Although these methods provide accurate estimation for graphical structures, they can be infeasible when the dimensionality is ultra-high due to the tremendous computation and memory costs.
To alleviate the computational costs caused by high dimensionality, conducting a screening procedure before the second stage analysis is an efficient and straightforward strategy. Under the ultra-high dimensional settings, screening methods were shown to enjoy great advantages in linear models [9], [31] and have been generalized to different models owing to the computation convenience [7], [11], [12], [18], [19], [22], [36]. Specifically, for ultra-high graphical models, [20] proposed a screening approach called the graphical sure screening (GRASS) to recover the graphical structures. Despite the significant computational advantages, GRASS is not that suitable to identify the conditional dependence structure in graphical models since the statistics are constructed based on the marginal correlations so that two variables adopting joint correlations instead of the marginal ones would not be connected. In this paper, we suggest a new method called the graphical uniform joint screening (GUS) to overcome such issues for edge identification in ultra-high graphical models.
The major contributions of this paper are threefold. First, a uniform joint screening procedure is developed to identify the ultra-high dimensional graphical structures. The statistics of GUS are constructed based on the joint correlations instead of the marginal ones to evaluate the significance of edges. To the best of our knowledge, this is the first joint screening procedure proposed in graphical models. Moreover, a uniform threshold is suggested to screen all significant edges in the graph at one time so that GUS can be adaptive to various kinds of graphical structures. Second, we provide theoretical guarantees for the proposed procedure by establishing the sure screening property as well as the screening consistency without marginal correlation assumptions. Last but not the least, a scalable implementation of GUS is proposed so that all statistics can be generated via a single estimation of the ultra-high dimensional precision matrix, making it as fast as GRASS in terms of computational complexity. The numerical studies show that the proposed procedure enjoys both appealing statistical accuracy and computational efficiency.
The rest of this paper is organized as follows. Section 2 presents the problem setup and the proposed method. Theoretical properties including sure screening property and screening consistency are established in Section 3. We provide simulation and real data studies in Section 4. Section 5 concludes with discussion and possible future work. All technical details are relegated to the Section 6.
Section snippets
Model setting
To illustrate the idea of uniform joint screening, we adopt the Gaussian graphical model for simplicity. In such model, a graph can be characterized by a -variate random vector which follows multivariate normal distribution where is the mean vector, is the covariance matrix and is an undirected graph with the vertex (or node) set and the edge set between vertices. Specifically, the edge between and is absent if and only if
Theoretical properties
In this section, we study the theoretical properties of GUS. To begin with, we introduce the following propositions of and which are defined as and respectively.
Proposition 1 The random vector has a spherically symmetric distribution, that is, has the same probability distribution as for every orthonormal matrix .
Proposition 2 For the random matrix , there are some and such that the following inequality holds for any submatrix
Numerical studies
In this section, we use numerical data to investigate the finite sample performance of the proposed GUS procedure, compared to another approach of recovering graphical model, graphical sure screening (GRASS) [20]. We also report the performance of innovated scalable efficient estimation (ISEE) [10] as a reference since ISEE enjoys appealing estimation accuracy and computational efficiency. In two simulation examples, the performance measure we consider to study screening accuracy is positive
Conclusion
In this paper, we have proposed a new procedure called as the graphical uniform joint screening to discover the conditional dependence in graphical models. Instead of screening out edges nodewisely, a uniform threshold is utilized for all statistics indicating the significance of different edges to adapt to various kinds of graphical structures. Moreover, a scalable approach is proposed for simplifying the calculations to adapt to large-scale applications. Both the established theoretical
Proofs
Proof of Theorem 1 We will finish the proof by studying the uniform property of associated with both significant and insignificant edges. Recall the statistic defined in (6) Based on the regression model of on
it is sufficient that can be rewritten as where is the th natural base in the -dimension space. For convenience of representation, we have the following notations
CRediT authorship contribution statement
Zemin Zheng: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Haiyu Shi: Methodology, Writing - original draft, Writing - review & editing, Software. Yang Li: Methodology, Writing - original draft, Writing - review & editing, Project administration. Hui Yuan: Methodology, Writing - original draft, Software.
Acknowledgments
This work was supported by National Natural Science Foundation of China Grants 11601501, 11671374, 71731010, and 71921001, Anhui Provincial Natural Science Foundation, China Grant 1708085QA02, and Fundamental Research Funds for the Central Universities, China Grant WK2040160028. The authors also sincerely thank the Editor, Associate Editor and the referees for their helpful comments and suggestions that led to substantial improvement of the paper.
References (36)
- et al.
Network exploration via the adaptive LASSO and SCAD penalties
Ann. Appl. Stat.
(2009) - et al.
Principled sure independence screening for Cox models with ultra-high-dimensional covariates
J. Multivariate Anal.
(2012) - et al.
Gene expression during the life cycle of drosophila melanogaster
Science
(2002) - et al.
A constrained minimization approach to sparse precision matrix estimation
J. Amer. Statist. Assoc.
(2011) - et al.
Estimating sparse precision matrix: Optimal rates of convergence and adaptive estimation
Ann. Statist.
(2016) - et al.
Activation of the easter zymogen is regulated by five other genes to define dorsal-ventral polarity in the Drosophila embryo
Development
(1992) - et al.
The strawberry notch gene functions with Notch in common developmental pathways
Development
(1993) - et al.
An Introduction to the Bootstrap
(1994) - et al.
Nonparametric independence screening in sparse ultra-high-dimensional additive models
J. Amer. Statist. Assoc.
(2011) - et al.
Sure independence screening for ultrahigh dimensional feature space
J. R. Stat. Soc. Ser. B. Stat. Methodol.
(2008)
Innovated scalable efficient estimation in ultra-large Gaussian graphical models
Ann. Statist.
Sure independence screening in generalized linear models with NP-dimensionality
Ann. Statist.
Joint variable screening in the censored accelerated failure time model
Statist. Sinica
Sparse inverse covariance estimation with the graphical lasso
Biostatistics
Joint estimation of multiple graphical models
Biometrika
Down-regulation of NF-B target genes by the AP-1 and STAT complex during the innate immune response in drosophila
PLoS Biol.
Estimating time-varying networks
Ann. Appl. Stat.
Graphical Models
Cited by (1)
Reproducible learning in large-scale graphical models
2022, Journal of Multivariate Analysis