This paper investigates the problem of testing independence of two random vectors of general dimensions. For this, we give for the first time a distribution-free consistent test. Our approach combines distance covariance with the center-outward ranks and signs developed in Hallin (2017). In technical terms, the proposed test is consistent and distribution-free in the family of multivariate distributions

This paper proposes a model-free and data-adaptive feature screening method for ultra-high dimensional data. The proposed method is based on the projection correlation which measures the dependence between two random vectors. This projection correlation based method does not require specifying a regression model, and applies to data in the presence of heavy tails and multivariate responses. It enjoys

In many social science experiments, subjects often interact with each other and as a result one unit’s treatment influences the outcome of another unit. Over the last decade, a significant progress has been made towards causal inference in the presence of such interference between units. Researchers have shown that the two-stage randomization of treatment assignment enables the identification of average

Causal inference on a population of units connected through a network often presents technical challenges, including how to account for interference. In the presence of interference, for instance, potential outcomes of a unit depend on its treatment as well as on the treatments of other units, such as its neighbors in the network. In observational studies, a further complication is that the typical

Traditional MCMC algorithms are computationally intensive and do not scale well to large data. In particular, the Metropolis-Hastings (MH) algorithm requires passing over the entire dataset to evaluate the likelihood ratio in each iteration. We propose a general framework for performing MH-MCMC using mini-batches of the whole dataset and show that this gives rise to approximately a tempered stationary

Space-filling designs are widely used in both computer and physical experiments. Column-orthogonality, maximin distance and projection uniformity are three basic and popular space-filling criteria proposed from different perspectives, but their relationships have been rarely investigated. We show that the average squared correlation metric is a function of the pairwise L2-distances between the rows

Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes, which show marked patterns of conditional heteroscedasticity. Significantly compounding the challenges, the recalls

Researchers across the health and social sciences generally assume that observations are independent, even while relying on convenience samples that draw subjects from one or a small number of communities, schools, hospitals, etc. A paradigmatic example of this is the Framingham Heart Study (FHS). Many of the limitations of such samples are well-known, but the issue of statistical dependence due to

Abstract–In nonexperimental settings, the regression discontinuity (RD) design is one of the most credible identification strategies for program evaluation and causal inference. However, RD treatment effect estimands are necessarily local, making statistical methods for the extrapolation of these effects a key area for development. We introduce a new method for extrapolation of RD effects that relies

While it is common practice in applied network analysis to report various standard network summary statistics, these numbers are rarely accompanied by uncertainty quantification. Yet any error inherent in the measurements underlying the construction of the network, or in the network construction procedure itself, necessarily must propagate to any summary statistics reported. Here we study the problem

Phylogenetic tree inference using deep DNA sequencing is reshaping our understanding of rapidly evolving systems, such as the within-host battle between viruses and the immune system. Densely sampled phylogenetic trees can contain special features, including sampled ancestors in which we sequence a genotype along with its direct descendants, and polytomies in which multiple descendants arise simultaneously

Abstract The scientific needs and computational limitations of the twentieth century fashioned classical statistical methodology. Both the needs and limitations have changed in the twenty-first, and so has the methodology. Large-scale prediction algorithms—neural nets, deep learning, boosting, support vector machines, random forests—have achieved star status in the popular press. They are recognizable

(2020). Discussion of the Paper “Prediction, Estimation, and Attribution” by B. Efron. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 656-658.

(2020). Discussion of Paper by Brad Efron. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 659-659.

(2020). Comment: When Is It Data Science and When Is It Data Engineering? Journal of the American Statistical Association: Vol. 115, No. 530, pp. 660-662.

(2020). Discussion. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 663-664.

Abstract Professor Efron has presented us with a thought-provoking paper on the relationship between prediction, estimation, and attribution in the modern era of data science. While we appreciate many of his arguments, we see more of a continuum between the old and new methodology, and the opportunity for both to improve through their synergy.

(2020). Discussion of Professor Bradley Efron’s Article on “Prediction, Estimation, and Attribution”. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 667-671.

(2020). The Data Science Process: One Culture. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 672-674.

(2020). Rejoinder. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 675-677.

(2020). Innovative Strategies, Statistical Solutions and Simulations for Modern Clinical Trials. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1029-1030.

(2020). Model-Based Geostatistics for Global Public Health: Methods and Applications. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1030-1032.

(2020). Theory of Spatial Statistics: A Concise Introduction. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1033-1034.

Clustered data is ubiquitous in a variety of scientific fields. In this paper, we propose a flexible and interpretable modeling approach, called grouped heterogenous mixture modeling, for clustered data, which models cluster-wise conditional distributions by mixtures of latent conditional distributions common to all the clusters. In the model, we assume that clusters are divided into a finite number

(2020). Sufficient Dimension Reduction: Methods and Applications With R. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1032-1033.

Abstract What does statistics have to offer science and society, in this age of massive data, machine learning algorithms, and multiple online sources of tools for data analysis? I recall a few situations where statistics made a real difference and reinforced the impact of our discipline on society. Sometimes the difference lay in the insightful analysis and inference enabled by ground-breaking methods

(2020). Statistical Modelling by Exponential Families. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1032-1032.

(2020). Empirical Likelihood Methods in Biomedicine and Health. Journal of the American Statistical Association: Vol. 115, No. 530, pp. 1028-1029.

Comparing the spatial characteristics of spatiotemporal random fields is often at demand. However, the comparison can be challenging due to the high-dimensional feature and dependency in the data. We develop a new multiple testing approach to detect local differences in the spatial characteristics of two spatiotemporal random fields by taking the spatial information into account. Our method adopts

We consider estimation of mean and covariance functions of functional snippets, which are short segments of functions possibly observed irregularly on an individual specific subinterval that is much shorter than the entire study interval. Estimation of the covariance function for functional snippets is challenging since information for the far off-diagonal regions of the covariance structure is completely

In a thought-provoking paper, Efron (2011) investigated the merit and limitation of an empirical Bayes method to correct selection bias based on Tweedie’s formula first reported in Robbins (1956). The exceptional virtue of Tweedie’s formula for the normal distribution lies in its representation of selection bias as a simple function of the derivative of log marginal likelihood. Since the marginal likelihood

Integrative network modeling of data arising from multiple genomic platforms provides insight into the holistic picture of the interactive system, as well as the flow of information across many disease domains including cancer. The basic data structure consists of a sequence of hierarchically ordered datasets for each individual subject, which facilitates integration of diverse inputs, such as genomic

Numerical embedding has become one standard technique for processing and analyzing unstructured data that cannot be expressed in a predefined fashion. It stores the main characteristics of data by mapping it onto a numerical vector. An embedding is often unsupervised and constructed by transfer learning from large-scale unannotated data. Given an embedding, a downstream learning method, referred to

When databases are constructed from heterogeneous sources, it is not unusual that different encodings are used for the same outcome. In such case, it is necessary to recode the outcome variable before merging two databases. The method proposed for the recoding is an application of optimal transportation where we search for a bijective mapping between the distributions of such variable in two databases

We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimation and variable selection in linear regression with grouped variables. We further extend the SSGL to sparse generalized additive models (GAMs), thereby introducing the first nonparametric variant of the spike-and-slab lasso methodology. Our model simultaneously performs group selection and estimation, while our fully Bayes treatment

A Mode-Jumping Algorithm for Bayesian Factor Analysis. Journal of the American Statistical Association. Accepted 20 May 2020.

Differential privacy (DP), provides a framework for provable privacy protection against arbitrary adversaries, while allowing the release of summary statistics and synthetic data. We address the problem of releasing a noisy real-valued statistic vector T, a function of sensitive data under DP, via the class of K-norm mechanisms with the goal of minimizing the noise added to achieve privacy. First,

While sample sizes in randomized clinical trials are large enough to estimate the average treatment effect well, they are often insufficient for estimation of treatment-covariate interactions critical to studying data-driven precision medicine. Observational data from real world practice may play an important role in alleviating this problem. One common approach in trials is to predict the outcome

Nonuniform subsampling methods are effective to reduce computational burden and maintain estimation efficiency for massive data. Existing methods mostly focus on subsampling with replacement due to its high computational efficiency. If the data volume is so large that nonuniform subsampling probabilities cannot be calculated all at once, then subsampling with replacement is infeasible to implement

Online decision-making problem requires us to make a sequence of decisions based on incremental information. Common solutions often need to learn a reward model of different actions given the contextual information and then maximize the long-term reward. It is meaningful to know if the posited model is reasonable and how the model performs in the asymptotic sense. We study this problem under the setup

This article focuses on the relationship between a measure of creativity and the human brain network for subjects in a brain connectome dataset obtained using a diffusion weighted magnetic resonance imaging (DWI) procedure. We identify brain regions and interconnections that have a significant effect on creativity. Brain networks are often expressed in terms of symmetric adjacency matrices, with row

Assuming stationarity is unrealistic in many time series applications. A more realistic alternative is to assume piecewise stationarity, where the model can change at potentially many change points. We propose a three-stage procedure for simultaneous estimation of change points and parameters of high-dimensional piecewise vector autoregressive (VAR) models. In the first step, we reformulate the change

Innovative Strategies, Statistical Solutions and Simulations for Modern Clinical Trials. Journal of the American Statistical Association. Accepted .

Theory of Spatial Statistics: A Concise Introduction. Journal of the American Statistical Association. Accepted .

Model-Based Geostatistics for Global Public Health: Methods and Applications. Journal of the American Statistical Association. Accepted .

Many high dimensional and high-throughput biological datasets have complex sample correlation structures, which include longitudinal and multiple tissue data, as well as data with multiple treatment conditions or related individuals. These data, as well as nearly all high-throughput ‘omic’ data, are influenced by technical and biological factors unknown to the researcher, which, if unaccounted for

Density estimation plays a fundamental role in many areas of statistics and machine learning. Parametric, nonparametric and semiparametric density estimation methods have been proposed in the literature. Semiparametric density models are flexible in incorporating domain knowledge and uncertainty regarding the shape of the density function. Existing literature on semiparametric density models is scattered

Seasonal weather forecasts are crucial for long-term planning in many practical situations and skillful forecasts may have substantial economic and humanitarian implications. Current seasonal forecasting models require statistical postprocessing of the output to correct systematic biases and unrealistic uncertainty assessments. We propose a multivariate postprocessing approach utilizing covariance

This note comments on the Generalized Measure of Correlation (GMC) that was suggested by Zheng et al. (2012). The GMC concept was partly anticipated in some publications over 100 years earlier by Yule (1897, 1900, 1903, 1907, 1909) in the proceedings of the Royal Society, and by Kendall (1946). Other antecedents discussed include work on dependency by Renyi (1959) and Doksum and Samarov (1995), together

Mediation analysis is of rising interest in epidemiology and clinical trials. Among existing methods, the joint significance (JS) test yields an overly conservative type I error rate and low power, particularly for high-dimensional mediation hypotheses. In this article we develop a multiple-testing procedure that accurately controls the family-wise error rate (FWER) and the false discovery rate (FDR)

Brain-imaging data have been increasingly utilized to understand intellectual disabilities. Despite significant progress in biomedical research, the mechanisms for most of the intellectual disabilities remain unknown. Finding the underlying neurological mechanisms has proved difficult, especially in children due to the rapid development of their brains. We investigate verbal reasoning, which is a reliable

In time-to-event settings, the presence of competing events complicates the definition of causal effects. Here we propose the new separable effects to study the causal effect of a treatment on an event of interest. The separable direct effect is the treatment effect on the event of interest not mediated by its effect on the competing event. The separable indirect effect is the treatment effect on the

The Fisher randomization test (FRT) is appropriate for any test statistic, under a sharp null hypothesis that can recover all missing potential outcomes. However, it is often sought after to test a weak null hypothesis that the treatment does not affect the units on average. To use the FRT for a weak null hypothesis, we must address two issues. First, we need to impute the missing potential outcomes

Understanding the spatial extent of extreme precipitation is necessary for determining flood risk and adequately designing infrastructure (e.g., stormwater pipes) to withstand such hazards. While environmental phenomena typically exhibit weakening spatial dependence at increasingly extreme levels, limiting max-stable process models for block maxima have a rigid dependence structure that does not capture

We develop a unified theory and methodology for the inference of evolutionary Fourier power spectra for a general class of locally stationary and possibly nonlinear processes. In particular, simultaneous confidence regions (SCR) with asymptotically correct coverage rates are constructed for the evolutionary spectral densities on a nearly optimally dense grid of the joint time-frequency domain. A simulation

Friendship and antipathy exist in concert with one another in real social networks. Despite the role they play in social interactions, antagonistic ties are poorly understood and infrequently measured. One important theory of negative ties that has received relatively little empirical evaluation is balance theory, the codification of the adage “the enemy of my enemy is my friend” and similar sayings

This article introduces a new class of models for multiple networks. The core idea is to parametrize a distribution on labelled graphs in terms of a Fréchet mean graph (which depends on a user-specified choice of metric or graph distance) and a parameter that controls the concentration of this distribution about its mean. Entropy is the natural parameter for such control, varying from a point mass

Severe air pollution affects billions of people around the world, particularly in developing countries such as China. Effective emission control policies rely primarily on a proper assessment of air pollutants and accurate spatial clustering outcomes. Unfortunately, emission patterns are difficult to observe as they are highly confounded by many meteorological and geographical factors. In this study

Large-scale genome-wide association (GWAS) studies provide opportunities for developing genetic risk prediction models that have the potential to improve disease prevention, intervention or treatment. The key step is to develop polygenic risk score (PRS) models with high predictive performance for a given disease, which typically requires a large training data set for selecting truly associated single

In the insurance broker market, commissions received by brokers are closely related to so-called \customer value”: the longer a policyholder keeps their contract, the more profit there is for the company and therefore the broker. Hence, predicting the time at which a potential policyholder will surrender their contract is essential in order to optimize a commercial process and define a prospect scoring