Abstract

Consider the online testing of a stream of hypotheses where a real-time decision must be made before the next data point arrives. The error rate is required to be controlled at all decision points. Conventional simultaneous testing rules are no longer applicable due to the more stringent error constraints and absence of future data. Moreover, the online decision-making process may come to a halt when the total error budget, or alpha-wealth, is exhausted. This work develops a new class of structure-adaptive sequential testing (SAST) rules for online false discovery rate (FDR) control. A key element in our proposal is a new alpha-investing algorithm that precisely characterizes the gains and losses in sequential decision making. SAST captures time varying structures of the data stream, learns the optimal threshold adaptively in an ongoing manner and optimizes the alpha-wealth allocation across different time periods. We present theory and numerical results to show that SAST is asymptotically valid for online FDR control and achieves substantial power gain over existing online testing rules.

摘要

考虑对一系列假设进行在线测试，其中必须在下一个数据点到达之前做出实时决策。要求在所有决策点都控制错误率。常规同步测试规则由于更严格的错误限制和未来数据的缺失，不再适用。此外，当总错误预算或阿尔法财富耗尽时，在线决策过程可能会停止。这项工作开发了一类新的结构自适应顺序测试 (SAST) 规则，用于在线错误发现率 (FDR) 控制。我们提案中的一个关键要素是一种新的 alpha 投资算法，它可以精确地描述顺序决策中的收益和损失。SAST 捕获数据流的时变结构，以持续的方式自适应地学习最佳阈值，并优化不同时间段的 alpha-wealth 分配。