Abstract

Change point detection has been widely used in quality control, earthquake disaster prediction and other fields. Existing change-point analysis methods rarely take into account the computational complexity, memory requirements and privacy issues under large data size. In this paper, we propose a distributed fast algorithm for change-point estimation when data are divided into many computers. Based on a subsequence data stored in one single machine, we get a change-point pre-estimator which is used to construct an interval covering the true change point with large probability, and then search the change-point more precisely on this interval among all machines. The final estimator by the above algorithm is proved to have consistency and limiting distribution with the same performance under the data-centralized case. The effectiveness of our algorithm is verified by sufficient numerical experiments which show that the asymptotic properties of our method are very close to that of traditional one, but with much less computation time.

摘要

变点检测已广泛应用于质量控制、地震灾害预测等领域。现有的变点分析方法很少考虑大数据量下的计算复杂度、内存需求和隐私问题。在本文中，我们提出了一种分布式快速算法，用于将数据划分到多台计算机时进行变化点估计。基于单机存储的一个子序列数据，我们得到一个变化点预估计器，用于构造一个覆盖真实变化点的大概率区间，然后在这个区间上更精确地搜索变化点。机器。证明了上述算法的最终估计量在数据集中的情况下具有一致性和极限分布，具有相同的性能。