Abstract

Fisher’s exact test and Pearson’s chi-squared test are frequently used for testing associations of two binary variables in 2-by-2 contingency tables. In the single test setting, many studies have shown that the asymptotic Pearson’s chi-squared test cannot preserve the test size for small samples and Fisher Exact test tends to be overly conservative. Multiple unconditional exact tests were proposed for small samples as they perform better than the commonly used chi-square and Fisher’s exact test. No comparison of these approaches have been done in the multiple testing setting.This study examines the performances of two unconditional tests (Boschloo and Z-pooled test statistics are used) with Fisher’s exact test as well as asymptotic Pearson’s chi-squared test in a small sample multiple testing scenario via a simulation study. When testing simultaneously many null hypotheses, Benjamini-Hochberg (BH) procedure is typically applied to control the false discovery rate (FDR). The results show that in terms of sensitivity rate, the performances of Z-pooled and Boschloo Statistic are close to each other; Asymptotic chi-squared test is slightly better than the unconditional exact tests; Fisher’s exact test is the least powerful in all different settings. Boschloo’s test is more computation intensive. Z-pooled test is preferred if running time is a concern.

摘要

Fisher 精确检验和 Pearson 卡方检验经常用于检验 2×2 列联表中两个二元变量的关联。在单一检验设置中，许多研究表明渐近皮尔逊卡方检验无法保留小样本的检验规模，而费舍尔精确检验往往过于保守。针对小样本提出了多个无条件精确检验，因为它们比常用的卡方检验和 Fisher 精确检验表现更好。尚未在多重测试环境中对这些方法进行比较。本研究检查了两种无条件检验（使用 Boschloo 和 Z 池检验统计量）与 Fisher 精确检验以及渐近 Pearson 卡方检验的小规模测试的性能。通过模拟研究对多个测试场景进行采样。当同时测试许多零假设时，通常应用 Benjamini-Hochberg (BH) 程序来控制错误发现率 (FDR)。结果表明，在敏感率方面，Z-pooled和Boschloo Statistic的表现接近；渐近卡方检验略优于无条件精确检验；Fisher 精确检验在所有不同设置中都是最不有效的。Boschloo 的测试计算量更大。如果考虑运行时间，Z 池测试是首选。Z-pooled 和 Boschloo Statistic 的性能接近；渐近卡方检验略优于无条件精确检验；Fisher 精确检验在所有不同设置中都是最不有效的。Boschloo 的测试计算量更大。如果考虑运行时间，Z 池测试是首选。Z-pooled 和 Boschloo Statistic 的性能接近；渐近卡方检验略优于无条件精确检验；Fisher 精确检验在所有不同设置中都是最不有效的。Boschloo 的测试计算量更大。如果考虑运行时间，Z 池测试是首选。