Abstract

Many massive data sets are assembled through collections of information of a large number of individuals in a population. The analysis of such data, especially in the aspect of individualized inferences and solutions, has the potential to create significant value for practical applications. Traditionally, inference for an individual in the dataset is either solely relying on the information of the individual or from summarizing the information about the whole population. However, with the availability of big data, we have the opportunity, as well as a unique challenge, to make a more effective individualized inference that takes into consideration of both the population information and the individual discrepancy. To deal with the possible heterogeneity within the population while providing effective and credible inferences for individuals in a dataset, this article develops a new approach called the individualized group learning (iGroup). The iGroup approach uses local nonparametric techniques to generate an individualized group by pooling other entities in the population which share similar characteristics with the target individual, even when individual estimates are biased due to limited number of observations. Three general cases of iGroup are discussed, and their asymptotic performances are investigated. Both theoretical results and empirical simulations reveal that, by applying iGroup, the performance of statistical inference on the individual level are ensured and can be substantially improved from inference based on either solely individual information or entire population information. The method has a broad range of applications. An example in financial statistics is presented.

摘要

许多海量数据集是通过收集人口中大量个体的信息而汇集起来的。对此类数据的分析，尤其是在个性化推理和解决方案方面，具有为实际应用创造重大价值的潜力。传统上，对数据集中个体的推断要么完全依赖于个体的信息，要么是对整个人群信息的总结。然而，随着大数据的可用性，我们有机会，也有独特的挑战，可以做出更有效的个性化推断，同时考虑人口信息和个体差异。为了处理人群中可能存在的异质性，同时为数据集中的个体提供有效且可信的推理，本文开发了一种称为个体化群体学习 (iGroup) 的新方法。iGroup 方法使用局部非参数技术通过汇集人口中与目标个体具有相似特征的其他实体来生成个性化组，即使个体估计由于有限的观察而有偏差也是如此。讨论了 iGroup 的三个一般情况，并研究了它们的渐近性能。理论结果和实证模拟都表明，通过应用 iGroup，个人层面的统计推断的性能得到保证，并且可以从基于单独的个人信息或整个人口信息的推断中得到显着改善。该方法具有广泛的应用范围。给出了金融统计中的一个例子。