Abstract
We congratulate the author for this interesting paper which introduces a novel method for the data segmentation problem that works well in a classical change point setting as well as in a frequent jump situation. Most notably, the paper introduces a new model selection step based on finding the ‘steepest drop to low levels’ (SDLL). Since the new model selection requires a complete (or at least relatively deep) solution path ordering the change point candidates according to some measure of importance, a new recursive variant of the Wild Binary Segmentation (Fryzlewicz in Ann Stat 42:2243–2281, 2014, WBS) named WBS2, has been proposed for candidate generation.
1 Theoretical properties
One of the main strengths of the proposed methodology, possibly due to the SDLL, is that it can work well both in a change point regime as well as in a frequent jump regime: In a change point regime the minimum distance to the next change, \(\delta _i:=\min (\eta _i-\eta _{i-1},\eta _{i+1}-\eta _i)\), is reasonably large while the magnitude of the change \(f^\prime _i\) is bounded from above and can be small (even tend to zero as \(T \rightarrow \infty\)). In a frequent jump regime \(\delta _i\) is small (related to outlier detection) and necessarily corresponding jumps \(f_i^{\prime }\) need to be large to be detectable. In both situations, an adaptation of Lemma 1 of Wang et al. (2018) shows that no consistent estimator of the locations of change point exists when \(\sigma ^{-2} \min _{1 \le i \le N} (\delta _i\,(f_i^{\prime })^2) < \log (T)\).
While WBS2.SDLL is shown to perform well in both regimes numerically, the paper does not provide a theoretical underpinning of this good behaviour, in the sense that only a linear-time change point setting with \(\delta _T:=\min _i\delta _i\) being of the same order as the sample size T is considered: Such an assumption is not necessary for consistent change point detection and, moreover, it excludes models such as extreme.teeth (ET) and extreme.extreme.teeth (EET), which are reasonably considered as belonging to the frequent jump regime with \(\delta _T \le 5\). In the future, it will be very exciting to see which theoretical framework will help us to better understand the performance of statistical procedures that aim at handling both regimes simultaneously.
In addition, the best currently available results for the localisation rate attained by WBS as well as the requirement on the magnitude of changes for their detection, are sub-optimal when \(\delta _T/T \rightarrow 0\) (see Appendix A of Cho and Kirch 2019). Baranowski et al. (2019) and Wang et al. (2018) suggest modifications of WBS that alleviate the sub-optimality at the cost of introducing additional tuning parameters such as a threshold or an upper bound on the length of random intervals. However, even in these papers, the assumptions are formulated in terms of \(\min _i\delta _i\,\min _i (f_i^{\prime })^2\), which does not reflect that the strength of multiscale procedures lies in their ability to handle data sets containing both small changes with long distances to neighbouring change points, as well as large changes with shorter distances (see e.g., the mix model). Cho and Kirch (2019) consider multiscale change point situations by working with \(\min _i (\delta _i\,(f_i^{\prime })^2)\) in the theoretical investigation of a more systematic moving sum (MOSUM)-type procedure for candidate generation.
2 SDLL with alternative candidate generation methods
As already pointed out by the author, both components of the proposed algorithm, i.e., candidate generation and model section, can be used in combination with other methods. For example, in Cho and Kirch (2019), a version of WBS2 has been adopted as a candidate generation method for the localised pruning method proposed for model selection. We will now show that deterministic candidate generation methods, such as the multiscale MOSUM procedure (Chan and Chen 2017; Cho and Kirch 2019), can be used with SDLL. Our first tentative attempt at generating a complete solution path of candidates with a reasonable measure of importance attached, is described in Sect. 3 below. Based on some initial simulation results reported in Table 1, we conclude that deterministic candidate generation methods can be a good alternative, and that this approach merits further research. Such a deterministic method will always yield the same result when applied to the same data set, whereas WBS-based methods can produce different outcomes in different runs (as observed in Cho and Kirch 2019 on array comparative genomic hybridization data sets). In particular, WBS-based results are reproducible only if the seed of the random number generation is also reported. In Section 4.1 of the present paper, the use of a ‘median’ of several runs is proposed to reduce this problem, which clearly comes at the cost of additional computation time.
3 MOSUM-candidate generation and some simulations
Many of the methods included in the comparative simulation studies of the present paper have been designed for the change point regime with their default parameters chosen accordingly, e.g., to save computation time. For example, the algorithm referred to as ‘MOSUM’ in the present paper, implemented in the R package mosum (Meier et al. 2019a), has a tuning parameter that relates to the smallest \(\delta _T\) permitted, and its default value is set at 10, which we consider as a reasonable lower bound for a change point problem. Also, the default choice of the parameter \(\alpha \in [0, 1]\), which stems from change point testing and sets a threshold for candidate generation in the algorithm, is somewhat conservative (\(\alpha = 0.1\)) and not very meaningful in the frequent jump regime. Moving away from the change point regime, we set the minimum bandwidth as small as possible in generating the bandwidth set \({\mathcal{G}}\),Footnote 1 and also set a more liberal threshold with \(\alpha = 0.9\). With these choices, MOSUM shows much better performance than that reported in the present paper, see Table 1 below.
Additionally, we explore the possibility of deterministic candidate generation based on moving sum statistics for a given set of bandwidth pairs \((G_l, G_r) \in{\mathcal{G}} \times{\mathcal{G}}\):
At each scale \((G_l, G_r)\), we identify all \({{\widetilde{k}}}\) which maximises \(\vert{{\tilde{M}}}_k(G_l, G_r; X) \vert\) locally within \(({{\widetilde{k}}}- G_l,{{\widetilde{k}}} + G_r)\), denote the collection of such \({{\widetilde{k}}}\) by \({\mathcal{K}}(G_l, G_r)\), and set \(M_k(G_l, G_r; X) ={{\tilde{M}}}_k(G_l, G_r; X) \cdot{\mathbb{I}}\{k \in ({{\widetilde{k}}} - G_l,{{\widetilde{k}}} + G_r), \,{{\widetilde{k}}} \in{\mathcal{K}}(G_l, G_r)\}\). We aggregate the MOSUM statistics generated at multiple scales as \(V(k) = \sum _{(G_l, G_r)} M_k(G_l, G_r; X)\), and then generate a solution path as in Algorithm 1, which is complete if the scale (1, 1) is included.
Referring to the methodology combining Algorithm 1 with SDLL as MOSUM.SDLL, Table 1 shows the results from applying WBS2.SDLL, MOSUM.SDLL (both with \(\lambda = 0.9\)) and MOSUM (with the aforementioned choice of parameters) to ET and EET summarised over 1000 realisations. All methods perform better for EET than for ET since the signal-to-noise ratio \(\sigma ^{-2} \min _i \delta _i\,(f_i^\prime )^2\) is greater for ET (see also Sect. 1 above).
As already mentioned, MOSUM adapted for the frequent jump regime works considerably better than the default version calibrated for the change point regime. While being more conservative than the SDLL-based methods for ET, MOSUM still outperforms the others in terms of the absolute and the squared error measures and overall, it returns reasonably good estimators at a fraction of the time. MOSUM.SDLL shows that the deterministic candidate generation provides a promising alternative to WBS2: It performs slightly worse than WBS2.SDLL in identifying the correct number of change points (\(N = 199\)) but the mean squared error of \({{\widehat{f}}}\) indicates that MOSUM.SDLL may return estimators of better localisation accuracy.
Notes
We generate \({\mathcal{G}}\) as detailed in Section 3.5 of Meier et al. (2019b) with \(G_1 = 1\) but only use bandwidths \(\ge 2\) due to the necessity of local variance estimation.
References
Baranowski, R., Chen, Y., & Fryzlewicz, P. (2019). Narrowest-over-threshold detection of multiple change-points and change-point-like features. Journal of the Royal Statistical Society: Series B, 81, 649–672.
Chan, H. P., & Chen, H. (2017), Multi-sequence segmentation via score and higher-criticism tests (arXiv preprint). arXiv:1706.07586.
Cho, H., & Kirch, C. (2020). Two-stage data segmentation permitting multiscale change points, heavy tails and dependence (arXiv preprint). arXiv:1910.12486.
Fryzlewicz, P. (2014). Wild Binary Segmentation for multiple change-point detection. The Annals of Statistics, 42, 2243–2281.
Fryzlewicz, P. (2020), Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection. Journal of the Korean Statistical Society.
Meier, A., Cho, H., & Kirch, C. (2019a). mosum: Moving sum based procedures for changes in the mean. R package version 1.2.3.
Meier, A., Kirch, C., & Cho, H. (2019b) mosum: A package for moving sums in change point analysis. Journal of Statistical Software(to appear).
Wang, D., Yu, Y., & Rinaldo, A. (2018), Univariate mean change point detection: penalization, CUSUM and optimality arXiv preprint. arXiv:1810.09498.
Acknowledgements
Open Access funding provided by University of Bristol.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cho, H., Kirch, C. Discussion of ‘Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection’. J. Korean Stat. Soc. 49, 1076–1080 (2020). https://doi.org/10.1007/s42952-020-00076-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42952-020-00076-3