Abstract

For a linear model on massive data, we propose an aggregated estimator depending on adaptive LASSO estimators. The proposed method allows the reduction of the data storage volume and the introduction of an aggregates estimator which automatically selects, with a probability converging to one, the significant explanatory variables. Moreover, the aggregated estimator, corresponding to the non null true parameters has the same asymptotic Normal law as the adaptive LASSO estimator on the all data. But, the estimator calculated on all data is practically impossible to calculate, for lack of calculation memory or storage, when the model is on massive data. Then, another interest of our method is that it can work around the data processing problem of insufficient memory allocated by statistical software when the observation number is very large. The empirical performance is investigated by a comparative simulation study. A real data example is used to illustrate the usefulness of our method.

摘要

对于海量数据的线性模型，我们提出了一个基于自适应 LASSO 估计器的聚合估计器。所提出的方法允许减少数据存储量并引入聚合估计器，该估计器自动选择具有收敛到一个概率的重要解释变量。此外，对应于非空真参数的聚合估计量与所有数据上的自适应 LASSO 估计量具有相同的渐近正态律。但是，当模型在海量数据上时，由于缺乏计算内存或存储，对所有数据计算的估计量实际上是不可能计算的。然后，我们方法的另一个有趣之处在于，它可以解决当观测数非常大时统计软件分配的内存不足的数据处理问题。通过比较模拟研究来研究经验性能。一个真实的数据示例用于说明我们方法的有用性。