Abstract

Data analysis techniques that rely on standard statistical tools and algorithms often encounter problems when dealing with data sets that have large sample sizes. In this study, two statistical tests done in conducting simple linear regression analysis were revisited. In particular, the study simulated the effects of large sample sizes and amount of contamination in the data due to non-sampling errors on the false positive rate of the Kolmogorov-Smirnov (K-S) test in testing for normality of error terms. The study also characterized the effects of varying sample size and amount of contamination in the data on the false negative rate of the t-test in testing the significance of a regression coefficient. Lastly, an optimality index was developed to determine the sample sizes and the values of the percent noise at which both the false positive rate of the K-S test and the false negative rate of the t-test are minimized.

摘要

依赖标准统计工具和算法的数据分析技术在处理具有大样本量的数据集时经常会遇到问题。在本研究中，重新审视了在进行简单线性回归分析时进行的两项统计检验。特别是，该研究模拟了大样本量和非抽样误差导致的数据污染量对误差项正态性检验中柯尔莫哥洛夫-斯米尔诺夫（KS）检验假阳性率的影响。该研究还描述了不同样本量和数据中污染量对假阴性率的影响。-测试回归系数的显着性。最后，开发了最优性指数来确定样本大小和噪声百分比值，在该值下 KS 检验的假阳性率和 t 检验的假阴性率都最小化。