Change point detection for nonparametric regression under strongly mixing process

Abstract

In this article, we consider the estimation of the structural change point in the nonparametric model with dependent observations. We introduce a maximum-CUSUM-estimation procedure, where the CUSUM statistic is constructed based on the sum-of-squares aggregation of the difference of the two Nadaraya-Watson estimates using the observations before and after a specific time point. Under some mild conditions, we prove that the statistic tends to zero almost surely if there is no change, and is larger than a threshold asymptotically almost surely otherwise, which helps us to obtain a threshold-detection strategy. Furthermore, we demonstrate the strong consistency of the change point estimator. In the simulation, we discuss the selection of the bandwidth and the threshold used in the estimation, and show the robustness of our method in the long-memory scenario. We implement our method to the data of Nasdaq 100 index and find that the relation between the realized volatility and the return exhibits several structural changes in 2007–2009.

Appendix: Proofs

In this section, we provide the detailed proofs of the theoretical results in Sect. 3. Before proving the main theorems, we state and prove some lemmas.

The following lemma plays a crucial role in deriving some uniform bounds of an $\alpha$-mixing process.

Lemma 1

(Theorem 1.3 of Bosq (1998)) Let $(X_t , t\in {\mathbb {Z}})$ be a zero-mean real-valued process such that $\sup \limits _{1\le t\le n}\Vert X_t\Vert _\infty \le b$. Let $S_n=\sum _{t=1}^nX_t$. Then

• (i) For each integer $q\in [1,\frac{n}{2}]$ and each $\varepsilon >0$,

  $$\begin{aligned} \mathsf P (|S_n|>n\varepsilon )\le 4\exp \left( -\frac{\varepsilon ^2}{8b^2}q\right) +22\left( 1+\frac{4b}{\varepsilon }\right) ^{1/2}q\alpha \left( \left\lfloor \frac{n}{2q}\right\rfloor \right) . \end{aligned}$$

• (ii) For each integer $q\in [1,\frac{n}{2}]$ and each $\varepsilon >0$,

  $$\begin{aligned} \mathsf P (|S_n|>n\varepsilon )\le 4\exp \left( -\frac{\varepsilon ^2}{8v^2(q)}q\right) +22\left( 1+\frac{4b}{\varepsilon }\right) ^{1/2}q\alpha \left( \left\lfloor \frac{n}{2q}\right\rfloor \right) \end{aligned}$$

  with $v^2(q)=\frac{2}{p^2}\sigma ^2(q)+\frac{b\varepsilon }{2}$, $p=\frac{n}{2q}$,

  $$\begin{aligned} \sigma ^2(q)= & {} \max _{0\le j\le 2q-1}\mathsf{E }\left\{ (\lfloor jp\rfloor +1-jp)X_{\lfloor jp\rfloor +1}+X_{\lfloor jp\rfloor +2}+\right. \\&\left. \cdots +X_{\lfloor (j+1)p\rfloor }+((j+1)p-\lfloor (j+1)p\rfloor ) X_{\lfloor (j+1)p+1\rfloor }\right\} ^{2}. \end{aligned}$$

The following lemma demonstrates the bounds of the auto-covariances of $K_{h_n}(x-X_i)$ and $Y_i I_{(|Y_i|\le T)} K_{h_n}(x-X_i)$, that is, a truncated version of $Y_iK_{h_n}(x-X_i)$, which facilitates the application of Lemma 1 to the N–W estimator.

Lemma 2

Suppose $\{X_t,Y_t\}_{t=1}^n$ is an $\alpha$-mixing process, and Assumptions 4–6 are satisfied. Then we have

• (i)
  $$\begin{aligned} \mathsf Var \left( K_{h_n}(x-X_i)\right) \le c_1 h_n^{-1}, \end{aligned}$$

  (6.1)

  and

  $$\begin{aligned}&\left| \mathsf Cov \left( K_{h_n}(x-X_i),K_{h_n}(x-X_j)\right) \right| \nonumber \\&\quad \le \left\{ \begin{aligned}&c_2\min \{h_n^{-1+q_F},h_n^{-2}|i-j|^{-\gamma }\}, \{X_t,Y_t\}_{t=1}^n \in PSM,\\&c_2\min \{h_n^{-1+q_F},h_n^{-2}\rho ^{|i-j|}\}\ \ \ \ , \{X_t,Y_t\}_{t=1}^n \in GSM\\ \end{aligned} \right. \end{aligned}$$

  (6.2)

  for any $1\le i\ne j\le n$ and $x \in {\mathbb {R}}$, where $c_1$ and $c_2$ are two positive constants, and $q_F$ is defined in Assumption 5.

• (ii)
  $$\begin{aligned} \mathsf Var (Y_i I_{(|Y_i|\le T)} K_{h_n}(x-X_i))\le c_3 h_n^{-1}, \end{aligned}$$

  (6.3)

  and

  $$\begin{aligned}&\left| \mathsf Cov \left( Y_i I_{(|Y_i|\le T)} K_{h_n}(x-X_i),Y_j I_{(|Y_j|\le T)} K_{h_n}(x-X_j)\right) \right| \nonumber \\&\quad \le \left\{ \begin{aligned}&c_4\min \{h_n^{-1+q_F},h_n^{-2}|i-j|^{-\gamma }\}, \ \{X_t,Y_t\}_{t=1}^n \in PSM,\\&c_4\min \{h_n^{-1+q_F},h_n^{-2}\rho ^{|i-j|}\}\ \ \ \ \ , \{X_t,Y_t\}_{t=1}^n \in GSM\\ \end{aligned} \right. \end{aligned}$$

  (6.4)

  for any $1\le i\ne j\le n$ and $x \in {\mathbb {R}}$, where $c_3$ and $c_4$ are two positive constants which do not dependent on T.

Proof of Lemma 2

(i) Noting that f, the density function of X, is bounded and $\Vert K\Vert _2<\infty$, we can prove that

$$\begin{aligned} \mathsf Var (K_{h_n}(x-X_i))\le & {} \mathsf{E }\left[ K_{h_n}^2(x-X_i)\right] =\int _{{\mathbb {R}}} K_{h_n}^2(u-x)f(u)du\\= & {} h_n^{-1}\int _{{\mathbb {R}}} K^2(z)f(x+h_nz)dz \le c_1 h_n^{-1} \end{aligned}$$

with variable substitution $z=(u-x)/h_n$ and selecting $c_1=\Vert f\Vert _\infty \Vert K\Vert _2^2$. In terms of the covariance, we have

$$\begin{aligned}&\mathsf Cov \left( K_{h_n}(x-X_i),K_{h_n}(x-X_j)\right) \\&\quad =\mathsf{E }\left[ K_{h_n}(x-X_i)K_{h_n}(x-X_j)\right] -\mathsf{E }\left[ K_{h_n}(x-X_i)\right] \mathsf{E }\left[ K_{h_n}(x-X_j)\right] \\&\quad =\int _{{{\mathbb {R}}}^2} K_{h_n}(x-u)K_{h_n}(x-v)F^{(|i-j|)}(u,v)dudv, \end{aligned}$$

where $F^{(|i-j|)}$ is defined in Assumption 5. Letting $\bar{p_F}$ satisfy $p_F^{-1}+\bar{p_F}^{-1}=1$ and using Hölder inequality, we can prove that

$$\begin{aligned} \left| \mathsf Cov \left( K_{h_n}(x-X_i),K_{h_n}(x-X_j)\right) \right|\le & {} h_n^{-2}\cdot h_n^{2/\bar{p_F}}\Vert K\Vert _{\bar{p_F}}^2\cdot \Vert F^{|i-j|}\Vert _{p_F}\nonumber \\\le & {} c_{2,1} h_n^{-1+q_F}, \end{aligned}$$

(6.5)

where $c_{2,1}$ is equal to $\Vert K\Vert _{\bar{p_F}}^2\cdot C_2$, noting that $\Vert K\Vert _{\bar{p_F}}<\infty$, which is implied by $\Vert K\Vert _1<\infty$ and $\Vert K\Vert _\infty <\infty$ in Assumption 4. Besides, by using the Billingsley’s inequality (c.f. Chapter 1 of Bosq (1998)), we have

$$\begin{aligned} \left| \mathsf Cov \left( K_{h_n}(x-X_i),K_{h_n}(x-X_j)\right) \right|\le & {} 4\Vert K_{h_n}(x-X_i)\Vert _\infty \Vert K_{h_n}(x-X_j)\Vert _\infty \cdot \alpha (|i-j|)\nonumber \\\le & {} c_{2,2}h_n^{-2}\alpha (|i-j|), \end{aligned}$$

(6.6)

where $c_{2,2}=4\Vert K\Vert _\infty ^2$. Note that we have $\alpha (|i-j|)\le C_1\rho ^{|i-j|}$ in Assumption 2 for GSM or $\alpha (|i-j|)\le C_1|i-j|^{-\gamma }$ in Assumption 1 for PSM. Combining (6.5) and (6.6), and taking $c_2=\max \{c_{2,1},c_{2,2}C_1\}$, we can prove (6.2).

(ii) We denote $I_{(|Y_i|\le T)}$ as $I_i$ for simplicity. Noting that $\Vert K\Vert _2<\infty$, $\Vert f\Vert _\infty <\infty$ and $\sup _{x\in \mathsf supp [X_1]}\mathsf{E }\left[ Y_i^2|X_i=x\right] <C_4$ by Assumptions 4, 5 and 6 , respectively, we have

$$\begin{aligned} \mathsf Var (Y_iI_iK_{h_n}(x-X_i))\le & {} \mathsf{E }[Y_i^2K_{h_n}^2(x-X_i)]\\= & {} \int _{{\mathbb {R}}} \mathsf{E }[Y_i^2|X_i=u]K_{h_n}^2(x-u)f(u)du\le c_3 h_n^{-1} \end{aligned}$$

by selecting $c_3=C_4\cdot \Vert f\Vert _\infty \cdot \Vert K\Vert _2^2$.

For the covariance, set $A=\{(y_i,y_j):|y_i|\le T,|y_j|\le T\}$, we have

$$\begin{aligned}&\left| \mathsf Cov \left( Y_i I_iK_{h_n}(x-X_i),Y_jI_j K_{h_n}(x-X_j)\right) \right| \nonumber \\&\quad =\left| \mathsf{E }\left[ Y_i I_iK_{h_n}(x-X_i)\cdot Y_j I_jK_{h_n}(x-X_j)\right] \right. \nonumber \\&\qquad \left. -\mathsf{E }\left[ Y_i I_iK_{h_n}(x-X_i)\right] \cdot \mathsf{E }[Y_j I_jK_{h_n}(x-X_j)]\right| \nonumber \\&\quad =\left| \int _{{\mathbb {R}}^2} \int _ {A} \left\{ y_iy_jp(y_i,y_j,u,v)-y_ip(y_i,u)\cdot y_jp(y_j,v)\right\} dy_idy_j\right. \cdot \nonumber \\&\qquad \left. K_{h_n}(x-u)K_{h_n}(x-v)dudv\right| \nonumber \\&\quad \le \int _{{\mathbb {R}}^2} \int _ {{\mathbb {R}}^2} \left| y_iy_jp(y_i,y_j,u,v)-y_ip(y_i,u)y_jp(y_j,v)\right| dy_idy_j\cdot \nonumber \\&\qquad |K_{h_n}(x-u)K_{h_n}(x-v)|dudv\nonumber \\&\quad =\int _{{\mathbb {R}}^2} G^{|i-j|}(u,v)\cdot \left| K_{h_n}(x-u)K_{h_n}(x-v)\right| dudv\nonumber \\&\quad \le h_n^{-2}\cdot h_n^{2/\bar{p_G}}\Vert K\Vert _{\bar{p_G}}^2\cdot \Vert G^{|i-j|}\Vert _{p_G}\le c_{4,1} h_n^{-1+q_G}, \end{aligned}$$

where $c_{4,1}=C_3\Vert K\Vert _{\bar{p_G}}^2$ and $C_3$ are defined in Assumption 5. Note that the last step follows from the Hölder inequality similar to (6.5) with $\bar{p_G}$ satisfying $p_G^{-1}+\bar{p_G}^{-1}=1$.

Next we prove the second part in the minimization function in (6.4). Note that, for any $m\ge 1$ and $i=1,\cdots ,n$,

$$\begin{aligned} \mathsf{E }\left[ |Y_iI_iK_{h_n}(X_i-x)|^m\right]\le & {} \mathsf{E }\left[ |Y_iK_{h_n}(X_i-x)|^m\right] \nonumber \\\le & {} h_n^{-m}\cdot \mathsf{E }[|Y_i|^m]\cdot \Vert K\Vert _\infty ^m=O(h_n^{-m}), \end{aligned}$$

(6.7)

and $e^{C_5(\mathsf{E }\left[ |Y_i|^m\right] )^{1/m}}\le \mathsf{E }\left[ e^{C_5|Y_i|}\right] \le C_6$ by Assumption 6 and Jensen inequality. By Corollary 1.1 in Bosq (1998) together with (6.7), we have, for any $m>2$,

$$\begin{aligned}&\left| \mathsf Cov \left( Y_iI_iK_{h_n}(X_i-x),Y_jI_jK_{h_n}(X_j-x)\right) \right| \\&\quad \le 2m/(m-2) \cdot (h_n^{-m}\cdot \mathsf{E }[|Y_i|^m]\cdot \Vert K\Vert _\infty ^m)^{2/m}\cdot [2\alpha ([i-j])]^{1-2/m}\\&\quad \le c_{4,2}(m) \cdot h_n^{-2}\cdot [\alpha ([i-j])]^{1-2/m} \end{aligned}$$

with $c_{4,2}(m)=2^{2-2/m}\cdot m/(m-2)(\mathsf{E }\left[ |Y_i|^m\right] \Vert K\Vert _\infty ^m)^{2/m}$. Let m tend to infinity, we have $c_{4,2}(m)\rightarrow 4\cdot (\log (C_6)/C_5)^{2}\Vert K\Vert _\infty ^2:=c_{4,2}$, thus

$$\begin{aligned} \left| \mathsf Cov \left( Y_iI_iK_{h_n}(X_i-x),Y_jI_jK_{h_n}(X_j-x)\right) \right| \le c_{4,2} \cdot h_n^{-2}\cdot \alpha ([i-j]). \end{aligned}$$

Using $\alpha (|i-j|)\le C_1\rho ^{|i-j|}$ in Assumption 2 or $\alpha (|i-j|)\le C_1|i-j|^{-\gamma }$ in Assumption 1, and taking $c_4=\max \{c_{4,1},c_{4,2}C_1\}$, we can prove (6.4). $\square$

The following lemma shows the uniform bound of the N–W estimator of a PSM process. Note that (i) the bandwidth is selected based on the sample size n, (ii) the estimator ${\widehat{f}}_{s,u}(x;h_n)$ is constructed based on subsample set $\{X_t,Y_t\}_{t=s}^u$, for $1\le s\le u\le n$, and (iii) the uniform bound is considered with respect to time t.

Lemma 3

Suppose the process $\{X_t,Y_t\}_{t=1}^n$ is PSM and Assumptions 3–6 are satisfied. Let ${\widehat{f}}_{1,t}(x;h_n)$ and ${\widehat{f}}_{t+1,n}(x;h_n)$ be defined in (2.4). Then we have for $\forall x \in {\mathbb {R}}$, under the model (2.1)

$$\begin{aligned} \max _{ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right| =O_{a.s.}\left( \frac{\log n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.8)

and

$$\begin{aligned} \max _{ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{t+1,n}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{t+1,n}(x;h_n)\right] \right| =O_{a.s.}\left( \frac{\log n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.9)

Proof of Lemma 3

It is clear that if for some $\eta >0$,

$$\begin{aligned} \sum _{n=1}^\infty \mathsf P \left( \max _{ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right| >\eta \cdot \frac{\log n}{\sqrt{nh_n}}\right) <\infty , \end{aligned}$$

(6.10)

we can show (6.8) by using the Borel-Cantelli lemma. Next we prove (6.10). Actually,

$$\begin{aligned}&\mathsf P \left( \max _{ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right|>\eta \cdot \frac{\log n}{\sqrt{nh_n}}\right) \\&\quad \le \sum _{t=\varDelta _n}^{n-\varDelta _n}\mathsf P \left( \left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right|>\eta \cdot \frac{\log n}{\sqrt{nh_n}}\right) \\&\quad \le \sum _{t=\varDelta _n}^{n-\varDelta _n}\mathsf P \left( \left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right| >\eta \sqrt{\delta }\cdot \frac{\log t}{\sqrt{th_n}}\right) , \end{aligned}$$

noting that $\log n/\sqrt{n}>\sqrt{\delta } \log t/\sqrt{t}$ when $\lfloor n\delta \rfloor \le t<n$. A sufficient condition is that for some $\eta >0$ and $\delta _1>0$

$$\begin{aligned} \mathsf P \left( \left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right| >\eta \cdot \frac{\log t}{\sqrt{th_n}}\right) \le c_5\cdot n^{-(2+\delta _1)} \end{aligned}$$

(6.11)

when n is large enough, where $c_5$ is a positive constant (we still use the notation $\eta$ for $\eta \sqrt{\delta }$).

Next we prove (6.11) using Lemma 1(ii). Let $Z_{1,s,n}=K_{h_n}(x-X_s)-\mathsf{E }\left[ K_{h_n}(x-X_s)\right]$ for $s=1,\cdots ,n$, and denote the partial sum of $Z_{1,s,n}$ as $S_t=\sum _{s=1}^{t}Z_{1,s,n}$.

Firstly, we want to derive the order of $\sigma ^2(q)$ and $v^2(q)$ defined in Lemma 1 with the sequence $\{X_t\}_{t=1}^n$ replaced by the sequence $\{Z_{1,s,n}\}_{s=1}^t$. Taking $\varepsilon =\varepsilon _t=(th_n)^{-1/2}\log t$, $q=q_t=\lfloor t^{1/2}h_n^{-1/2}\rfloor$, and $p=t/(2q)$, we have $|Z_{1,s,n}|\le \Vert K \Vert _{\infty }\cdot h_n^{-1}$ and $q_t\le t/2$ for large n. Using the partition method similar to the proof of Theorem 3.3 in Johannes and Rao (2011), we have (define $p'=\lfloor p\rfloor +2$)

$$\begin{aligned} \sigma ^2(q)\le & {} \sum _{i=1}^{p'}\mathsf Var (K_{h_n}(X_i-x))+2\sum _{i>j}^{p'}|\mathsf Cov (K_{h_n}(X_i-x),K_{h_n}(X_j-x))|\\= & {} \sum _{i=1}^{p'}\mathsf Var (K_{h_n}(X_i-x))+2\sum _{i=2}^{p'}(p'-i+1)|\mathsf Cov (K_{h_n}(X_i-x),K_{h_n}(X_1-x))|\\\le & {} c_1p'h_n^{-1} +2p'\sum _{i=2}^B|\mathsf Cov (K_{h_n}(X_i-x),K_{h_n}(X_1-x))|\\&+2p'\sum _{i=B+1}^{p'}|\mathsf Cov (K_{h_n}(X_i-x),K_{h_n}(X_1-x))| \end{aligned}$$

with the partition point $B=\lfloor h_n^{-q_F}\rfloor$. Then using Lemma 2, we can obtain for large n

$$\begin{aligned} \sigma ^2(q)\le & {} c_1p'h_n^{-1}+2p'\sum _{i=2}^B c_2h_n^{-1+q_F}+2p'\sum _{i=B+1}^{p'}c_2h_n^{-2}(i-1)^{-\gamma }\nonumber \\\le & {} c_1p'h_n^{-1}+2p'B\cdot c_2 h_n^{-1+q_F}+2p'h_n^{-2}\cdot 2c_2 B^{1-\gamma }\nonumber \\\le & {} c_1p'h_n^{-1}+2p'c_2h_n^{-q_F}\cdot h_n^{-1+q_F}+4c_2p'h_n^{-2+q_F(\gamma -1)}\nonumber \\\le & {} (c_1+6c_2)\cdot p'h_n^{-1}, \end{aligned}$$

(6.12)

where the term $B^{1-\gamma }$ is induced by substituting the sum with an integral and the last row follows from $q_F(\gamma -1)>1$ in Assumption 5. So for $v^2(q)$, we have

$$\begin{aligned} v^2(q)= & {} 2\sigma ^2(q)/p^2+\Vert K \Vert _{\infty }\cdot h_n^{-1}\varepsilon _t/2\nonumber \\\le & {} 2(c_1+6c_2)h_n^{-1}p'p^{-2}+\Vert K \Vert _{\infty }\cdot h_n^{-1}\varepsilon _t/2\nonumber \\\le & {} \Vert K \Vert _{\infty }\cdot h_n^{-1}\varepsilon _t \end{aligned}$$

(6.13)

for n large enough, noting that $p'/(p^2)\simeq 2\varepsilon _t/\log t=o(\varepsilon _t)$ when $\lfloor \delta n\rfloor \le t\le n-\lfloor \delta n\rfloor$. Then using Lemma 1(ii) and (6.13),

we have for $\eta >0$

$$\begin{aligned} (6.13)= & {} \mathsf P \left( \left| S_t\right| >t\cdot \eta \varepsilon _t\right) \nonumber \\\le & {} 4\exp \left( -\frac{\eta ^2\varepsilon _t}{8 \Vert K\Vert _{\infty }}q_th_n\right) +22\left( 1+\frac{4\Vert K \Vert _{\infty }\cdot h_n^{-1}}{\eta \varepsilon _t}\right) ^{1/2}q\alpha \left( \left\lfloor \frac{t}{2q_t} \right\rfloor \right) \nonumber \\:= & {} A_{1,t}+A_{2,t}. \end{aligned}$$

(6.14)

Because $q_t\simeq t^{1/2}h_n^{-1/2}$ and thereby $\varepsilon _tq_th_n\simeq (\log t)$, by selecting $\eta >\sqrt{8(2+\delta _1)\Vert K\Vert _\infty }$, we have $A_{1,t}\le 4t^{-{\eta ^2}/(8\Vert K\Vert _\infty )}=O\left( n^{-(2+\delta _1)}\right)$. In terms of $A_{2,t}$, noting that $(h_n^{-1}/\varepsilon _t)^{1/2}q_t\le t^{3/4}h_n^{-3/4}\le n^{3/4}h_n^{-3/4}\rightarrow \infty$ and $\alpha \left( \left\lfloor \frac{t}{2q_t}\right\rfloor \right) \le C_1 {\left\lfloor \frac{t}{2q_t}\right\rfloor }^{-\gamma }\le C_1 {\left\lfloor \frac{\sqrt{th_n}}{2}\right\rfloor }^{-\gamma }=O(n^{-\gamma /2}h_n^{-\gamma /2})$, we can obtain

$$\begin{aligned} A_{2,t}=O\left( n^{-\gamma /2+3/4}h_n^{-3/4-\gamma /2}\right) =O\left( n^{-(2+\delta _1)}\right) \end{aligned}$$

when $h_n\simeq n^{-\omega }$ for some $0<\omega \le \frac{\gamma /2-11/4-\delta _1}{\gamma /2+3/4}<1-\frac{14}{2\gamma +3}$.

The proof of (6.9) is similar to the proof of (6.8) by considering the sequence $\{Z_{1,s,n}\}_{s=t+1}^n$. Thus we omit the proof. Then we complete the proof of this lemma. $\square$

Lemma 4

Suppose the process $\{X_t,Y_t\}_{t=1}^n$ is GSM and Assumptions 3–6 are satisfied. Let ${\widehat{f}}_{1,t}(x;h_n)$ and ${\widehat{f}}_{t+1,n}(x;h_n)$ be defined in (2.4), then we have for $\forall x \in {\mathbb {R}}$, under the model (2.1)

$$\begin{aligned} \max _{\ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{1,t}(x;h_n)\right] \right| =O_{a.s.}\left( \frac{\log n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.15)

and

$$\begin{aligned} \max _{\ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{f}}_{t+1,n}(x;h_n)-\mathsf{E }\left[ {\widehat{f}}_{t+1,n}(x;h_n)\right] \right| =O_{a.s.}\left( \frac{\log n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.16)

Proof of Lemma 4

The proof of this lemma is similar to the proof of Lemma 3. Because of similarity, we only prove (6.15).

We need to prove (6.11) by Lemma 1(ii). Using the same notation with Lemma 3, we have $\sigma ^2(q)=O(p'h_n^{-1})$, hence $v^2(q)\le \Vert K \Vert _{\infty }\cdot h_n^{-1}\varepsilon _t$ for n large enough (see Lemma 2.1 of Bosq (1998)). Then we still have (6.14). By selecting $\eta >\sqrt{8(2+\delta _1)\Vert K\Vert _\infty }$, we have

$$\begin{aligned} A_{1,t}\le 4t^{-{\eta ^2}/(8\Vert K\Vert _\infty )}=O\left( n^{-(2+\delta _1)}\right) . \end{aligned}$$

(6.17)

In terms of $A_{2,t}$, note that $\log t/\sqrt{th_n}\rightarrow 0$, which implies that $th_n$ and thus $\log t-\log h_n^{-1}\rightarrow \infty$, and therefore $\log t$ and $\log h_n^{-1}$ can be bounded by $\sqrt{th_n}$. We have

$$\begin{aligned} A_{2,t}= & {} 22\left( 1+\frac{4\Vert K \Vert _{\infty }\cdot h_n^{-1}}{\eta \varepsilon _t}\right) ^{1/2}q\alpha \left( \left\lfloor \frac{t}{2q_t}\right\rfloor \right) \nonumber \\\le & {} c_7\cdot t^{3/4}h_n^{-3/4}\cdot \rho ^{\sqrt{th_n}/2}\nonumber \\= & {} c_7\cdot \exp \left\{ \frac{3}{4}\left( \log t+\log h_n^{-1}\right) \right\} \cdot \exp \left\{ -\frac{1}{2}\log \left( \frac{1}{\rho }\right) \cdot \sqrt{th_n}\right\} \nonumber \\\le & {} c_7\cdot \exp \left\{ -c_8\sqrt{th_n}\right\} \nonumber \\= & {} c_7\cdot t^{ -c_8/\varepsilon _{t}} =o\left( n^{-(2+\delta _1)}\right) , \end{aligned}$$

(6.18)

where $c_7$ and $c_8$ are two positive constants, noting that $\varepsilon _t\rightarrow 0$. Combining (6.14), (6.17) and (6.18), we can prove (6.11). Thus we complete the proof of this lemma. $\square$

Lemma 5

Suppose the process $\{X_t,Y_t\}_{t=1}^n$ is PSM and Assumptions 3–6 are satisfied. Let ${\widehat{g}}_{1,t}(x;h_n)$ and ${\widehat{g}}_{t+1,n}(x;h_n)$ be defined in (2.3). Then we have for $\forall x \in {\mathbb {R}}$, under the model (2.1)

$$\begin{aligned} \ \max _{\varDelta _n\le t\le n- \varDelta _n}\left| {\widehat{g}}_{1,t}(x;h_n)-\mathsf{E }[\widehat{g}_{1,t}(x;h_n)] \right| =O_{a.s.}\left( \frac{\log ^2n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.19)

and

$$\begin{aligned} \ \max _{\varDelta _n\le t\le n- \varDelta _n}\left| {\widehat{g}}_{t+1,n}(x;h_n)-\mathsf{E }[\widehat{g}_{t+1,n}(x;h_n)] \right| =O_{a.s.}\left( \frac{\log ^2n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.20)

Proof of Lemma 5

The proof of this lemma is similar to that of Lemma 3. The only difference is that $Y_i$ may not be bounded, and we need to adopt the idea of truncation before using Lemma 1(ii). Analogously, our goal is to prove for some $\eta >0$

$$\begin{aligned}&\sum _{n=1}^\infty \mathsf P \left( \max _{ \varDelta _n\le t\le n-\varDelta _n}\left| {\widehat{g}}_{1,t}(x;h_n)-\mathsf{E }\left[ {\widehat{g}}_{1,t}(x;h_n)\right] \right| >\eta \frac{\log ^2 n}{\sqrt{nh_n}}\right) <\infty , \end{aligned}$$

which can be proved by

$$\begin{aligned} \mathsf P \left( \left| {\widehat{g}}_{1,t}(x;h_n)-\mathsf{E }[{\widehat{g}}_{1,t}(x;h_n)]\right| >\eta \varepsilon _t\right) =O\left( n^{-(2+\delta _1)}\right) , \end{aligned}$$

(6.21)

where $\varepsilon _t=\log ^2t/\sqrt{th_n}$ and $\delta _1$ is a tiny positive constant.

Next, we prove (6.21). For $s=1,\cdots ,n$ , define $\bar{Y}_s=Y_sI_{(|Y_s|\le T_{t})}$ and $\widetilde{Y}_s=Y_sI_{(|Y_s|>T_{t})}$ with $T_{t}=c_9\log t$, where $c_9$ is a positive constant which will be determined later. Then

$$\begin{aligned} Z_{2,s,n}=: & {} Y_sK_{h_n}(X_s-x)-\mathsf{E }[Y_sK_{h_n}(X_s-x)]\nonumber \\= & {} (\bar{Y}_sK_{h_n}(X_s-x)-\mathsf{E }[\bar{Y}_sK_{h_n}(X_s-x)])\nonumber \\&+(\widetilde{Y}_sK_{h_n}(X_s-x)-\mathsf{E }[\widetilde{Y}_sK_{h_n}(X_s-x)])\nonumber \\:= & {} \bar{Z}_{2,s,n}+\widetilde{Z}_{2,s,n}. \end{aligned}$$

(6.22)

Denote the partial sums in (6.22) as ${\bar{S}}_{t,n}=\sum _{s=1}^t\bar{Z}_{2,s,n}$ and ${\widetilde{S}}_{t,n}=\sum _{s=1}^t\widetilde{Z}_{2,s,n}$. To use Lemma 1(ii), set $q=q_t=\lfloor t^{1/2}h_{n}^{-1/2}\rfloor$. Then (6.21) can be written as follows,

$$\begin{aligned} \mathsf P \left( \left| {\widehat{g}}_{1,t}(x;h_n)-\mathsf{E }[{\widehat{g}}_{1,t}(x;h_n)]\right|>\eta \varepsilon _t\right)= & {} \mathsf P \left( |{\bar{S}}_{t,n}+{\widetilde{S}}_{t,n}|>t\cdot \eta \varepsilon _t\right) \nonumber \\\le & {} \mathsf P \left( |{\bar{S}}_{t,n}|>t\cdot \frac{\eta }{2} \varepsilon _t\right) +\mathsf P \left( |{\widetilde{S}}_{t,n}|>t \cdot \frac{\eta }{2} \varepsilon _t\right) ,\nonumber \end{aligned}$$

so we can prove this lemma by showing that

$$\begin{aligned} \mathsf P \left( |{\bar{S}}_{t,n}|>t\cdot \frac{\eta }{2} \varepsilon _t\right) =O\left( n^{-(2+\delta _1)}\right) \end{aligned}$$

(6.23)

and

$$\begin{aligned} \mathsf P \left( |{\widetilde{S}}_{t,n}|>t \cdot \frac{\eta }{2} \varepsilon _t\right) =O\left( n^{-(2+\delta _1)}\right) . \end{aligned}$$

(6.24)

For (6.23), before using the similar method by the inequality in Lemma 1(ii) like before, we still need to show the bound of $\sigma ^2(q)$. Together with Lemma 2(ii), it immediately follows that, like (6.12) by using $B = \lfloor h_n^{-q_G}\rfloor$, for large n

$$\begin{aligned} \sigma ^2(q)\le & {} c_3p'h_n^{-1}+2p'\sum _{i=2}^B c_4h_n^{-1+q_G}+2p'\sum _{i=B+1}^{p'}c_4h_n^{-2}(i-1)^{- \gamma }\\\le & {} c_3p'h_n^{-1}+2p'B\cdot c_4h_n^{-1+q_G}+2p'h_n^{-2}c_4\cdot 2B^{1- \gamma }\\\le & {} c_3p'h_n^{-1}+2p'c_4h_n^{-q_G}\cdot h_n^{-1+q_G}+4p'c_4h_n^{-2+q_G( \gamma -1)}\\\le & {} (c_3+6c_4)\cdot p'h_n^{-1} \end{aligned}$$

when $q_G( \gamma -1)>1$. Hence

$$\begin{aligned} v^2(q)\le \left( \frac{4(c_3+6c_4) }{c_9\Vert K\Vert _\infty \log ^3 t}+1\right) \Vert K \Vert _{\infty } h_{n}^{-1}\varepsilon _t T_{t} \le 2\Vert K \Vert _{\infty } h_{n}^{-1}\varepsilon _t T_{t} \end{aligned}$$

when n is sufficiently large. Then we can use Lemma 1(ii) like the proof before and derive that for $\eta >0$

$$\begin{aligned} \mathsf P \left( |{\bar{S}}_{t,n}|>t\cdot \frac{\eta }{2}\varepsilon _t\right)\le & {} 4\exp \left( -\frac{\eta ^2\varepsilon _t}{64 \Vert K \Vert _{\infty }T_{t} }qh_n\right) \\&+22\left( 1+\frac{16\Vert K\Vert _\infty h_n^{-1}T_{t}}{\eta \varepsilon _t}\right) ^{1/2}q\alpha \left( \left\lfloor \frac{t}{2q}\right\rfloor \right) \nonumber \\:= & {} A_{3,t}+A_{4,t}. \end{aligned}$$

For $A_{3,t}$, we have

$$\begin{aligned} A_{3,t}\simeq & {} 4\exp \left( -\frac{\eta ^2}{64c_9\Vert K \Vert _{\infty }}\cdot \log t \right) =4 t^{-\eta ^2/(64c_9\Vert K \Vert _{\infty })} =O\left( n^{-(2+\delta _1)}\right) \end{aligned}$$

by selecting $\eta >8\sqrt{c_9(2+\delta _1)\Vert K \Vert _{\infty }}$. For $A_{4,t}$, we have

$$\begin{aligned} A_{4,t}= & {} 22\left( 1+\frac{16c_2\Vert K\Vert _\infty \sqrt{t}}{\eta \sqrt{h_n}\log t}\right) ^{1/2}\frac{\sqrt{t}}{\sqrt{h_n}}\left( \frac{\sqrt{th_n}}{4}\right) ^{-\gamma }\nonumber \\\le & {} c_{10}\frac{t^{3/4-\gamma /2}}{h_n^{3/4+\gamma /2}}\frac{1}{\log ^{1/2} t}\nonumber \\\le & {} c_{11}\frac{1}{n^{\gamma /2-3/4}h_n^{3/4+\gamma /2}}\nonumber \\= & {} O\left( n^{-(2+\delta _1)}\right) , \end{aligned}$$

where the existence of $\delta _1$ in the last equality follows from Assumption 3. Then we have proved (6.23).

In terms of (6.24), using Cauchy-Schwarz and Markov inequality, we have

$$\begin{aligned} \mathsf P \left( |{\widetilde{S}}_{t,n}|>t\cdot \frac{\eta }{2}\varepsilon _t\right)\le & {} \frac{2\mathsf{E }\left[ |{\widetilde{S}}_{t,n}|\right] }{t \eta \varepsilon _t}\nonumber \\\le & {} \frac{2\mathsf{E }\left[ |Y_1K_h(X_1-x)I_{(|Y_1|> T_{t})}|\right] }{ \eta \varepsilon _t}\nonumber \\\le & {} \frac{2\left\{ {\mathsf {E}}\left[ \left| Y_1K_h(X_1-x)\right| ^2\right] \right\} ^{1/2}\left\{ {\mathsf {P}}\left( C_5\left| Y_1\right|>C_5T_{t}\right) \right\} ^{1/2}}{\eta \varepsilon _t}\nonumber \\= & {} \frac{2\left\{ \int \mathsf{E }[Y_1^2|X_1=u]K_{h_n}^2(u-x)f(u)du\right\} ^{1/2} \left\{ {\mathsf {P}}(e^{C_5|Y_1|}>e^{C_5T_{t}})\right\} ^{1/2}}{\eta \varepsilon _t}\nonumber \\= & {} O\left( \frac{\sqrt{th_n}}{\log ^2 t}\cdot h_n^{-1/2}\cdot t^{-\frac{c_9C_5}{2}}\right) \nonumber \\= & {} O\left( n^{-(2+\delta _1)}\right) \end{aligned}$$

by letting $c_9>(5+2\delta _1)/C_5$, noting that $\Vert K\Vert _2<\infty$, f is bounded, $\sup \limits _{x\in \mathsf supp [{ X_1}]}$$\mathsf{E }\left[ Y_1^2|X_1=x\right] <\infty$, and $\mathsf{E }\left[ e^{C_5|Y_1|}\right] <\infty$.

Combing (6.23) and (6.24), we obtain (6.21). Hence the proof is completed. $\square$

Lemma 6

Suppose the process $(X_t,Y_t)$ is GSM and Assumptions 3–6 are satisfied. Let ${\widehat{g}}_{1,t}(x;h_n)$ and ${\widehat{g}}_{t+1,n}(x;h_n)$ be defined in (2.3). Then we have for $\forall x \in {\mathbb {R}}$, under the model (2.1)

$$\begin{aligned} \max _{\varDelta _n\le t\le n- \varDelta _n}|{\widehat{g}}_{1,t}(x;h_n)-\mathsf{E }[\widehat{g}_{1,t}(x;h_n)] |=O_{a.s.}\left( \frac{\log ^2n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.25)

and

$$\begin{aligned} \max _{\varDelta _n\le t\le n- \varDelta _n}|{\widehat{g}}_{t+1,n}(x;h_n)-\mathsf{E }[\widehat{g}_{t+1,n}(x;h_n)] |=O_{a.s.}\left( \frac{\log ^2n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.26)

Proof of Lemma 6

We still use the notation in Lemma 5, and want to show (6.23) and (6.24). Together with Lemma 2(ii), it still holds that

$$\begin{aligned} \sigma ^2(q)\le & {} c_3p'h_n^{-1}+2p'\left( \sum _{i=2}^B c_4h_n^{-1+q_G}+\sum _{i=B+1}^{p'}c_4h_n^{-2}\rho ^{(i-1) }\right) \\\le & {} (c_3+2c_4)p'h_n^{-1}+2c_4p' h_n^{-2}\cdot (1-\rho )^{-1}\rho ^{h_n^{-q_G} }\\= & {} O(p'h_n^{-1}), \end{aligned}$$

noting that $\frac{p'h_n^{-2}}{p'h_n^{-1}}\cdot \rho ^{h_n^{-q_G} }=h_n^{-1}\rho ^{h_n^{-q_G} }\simeq (h_n^{-q_G} )^{\frac{1}{q_G}}\cdot \rho ^{h_n^{-q_G} }\rightarrow 0$. So we only need to show the part $A_{4,t}$ containing mixing-coefficient as follows,

$$\begin{aligned} A_{4,t}= & {} 22\left( 1+\frac{16\Vert K\Vert _\infty h_n^{-1}T_{t}}{\eta \varepsilon _t}\right) ^{1/2}q\alpha \left( \left\lfloor \frac{t}{2q}\right\rfloor \right) \nonumber \\\simeq & {} 22\left( 1+\frac{16c_9\Vert K\Vert _\infty \sqrt{t}}{\eta \sqrt{h_n}\log t}\right) ^{1/2}\frac{\sqrt{t}}{\sqrt{h_n}}\cdot \rho ^{\sqrt{th_n}/2}\nonumber \\\le & {} c_{12}\frac{t^{3/4}}{h_n^{3/4}}\frac{1}{\log ^{1/2} t}\cdot \rho ^{\sqrt{th_n}/2}\nonumber \\\le & {} c_{13}\exp \{-c_{14}\log ^2 t(\varepsilon _t)^{-1}\}\nonumber \\= & {} O\left( n^{-(2+\delta _1)}\right) , \end{aligned}$$

(6.27)

where $c_{12}, c_{13}$ and $c_{14}$ are strictly positive constants, noting that the last second row is deduced like (6.18), and $\varepsilon _t\rightarrow 0$. $\square$

Lemma 7

Suppose that the assumptions in Lemmas 3 and 5 (or Lemmas 4 and 6 ) are satisfied. Let ${{\widehat{\varphi }}}_{1,t}(x;h_n)$ and ${{\widehat{\varphi }}}_{t+1,n}(x;h_n)$ be defined in (2.2). If the grid point $x_i\in {\mathcal {X}}$, then we have under the model (2.1)

$$\begin{aligned} \max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| {\widehat{\varphi }}_{1,t}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| =O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) , \end{aligned}$$

(6.28)

and

$$\begin{aligned} \max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| {\widehat{\varphi }}_{t+1,n}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{t+1,n}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{t+1,n}(x_i;h_n)]}\right| =O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.29)

Proof of Lemma 7

Because of similarity, we only show the first equation. Consider the decomposition

$$\begin{aligned}&{\widehat{\varphi }}_{1,t}(x_i)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\\&\quad = \frac{\widehat{g}_{1,t}(x_i;h_n)-\widehat{f}_{1,t}(x_i;h_n)\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\\&-\frac{\widehat{f}_{1,t}(x_i;h_n)-\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}{\widehat{f}_{1,t}(x_i;h_n)}\cdot \frac{\widehat{g}_{1,t}(x_i;h_n)-\widehat{f}_{1,t}(x_i;h_n)\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}. \end{aligned}$$

By Lemma 3, $\widehat{f}_{1,t}(x_i;h_n)-\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]=o_{a.s.}(1)$ uniformly over t. Since $x_i\in {{{\mathcal {X}}}}=(\mathsf supp [X_1])^\circ$, the density function f has a nonzero lower bound in a sufficient small neighbourhood of $x_i$. We have $\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]=\mathsf{E }\left[ K_{h_n}({X_1-x_i})\right] =h_n^{-1}\int _{{\mathbb {R}}} K((u-x_i)/h_n)f(u)du=\int _{{\mathbb {R}}} K(z)f(z h_n+x_i)dz\ge c_{15}$ for some positive constant $c_{15}$ if $h_n$ is small enough. Then, by Lemmas 3 and 5 , we have

$$\begin{aligned}&\max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| {\widehat{\varphi }}_{1,t}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| \\&\quad =\max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| \frac{\widehat{g}_{1,t}(x_i;h_n)-\widehat{f}_{1,t}(x_i;h_n)\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| \left( 1+o_{a.s.}(1)\right) \\&\quad \le \max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| \frac{\widehat{g}_{1,t}(x_i;h_n)-\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| \left( 1+o_{a.s.}(1)\right) \\&+\max \limits _{\varDelta _n\le t \le n-\varDelta _n}\left| \frac{\left( \widehat{f}_{1,t}(x_i;h_n)-\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]\right) \frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| \left( 1+o_{a.s.}(1)\right) \\= & {} O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) , \end{aligned}$$

noting that $\mathsf{E }\left[ |\widehat{g}_{1,t}(x_i;h_n)|\right] \le \mathsf{E }\left[ |Y_iK_{h_n}(X_1-x_i)|\right] =\int _{{\mathbb {R}}}\mathsf{E }\left[ |Y_1|\mid X_1=h_nz+x_i\right] \cdot K(z)f(h_nz+x_i)dz\le C_4||K||_1||f||_\infty$.

Hence we complete the proof of this lemma. $\square$

Proof of Theorem 1

Since there is no change point, we have $\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]=\mathsf{E }[\widehat{f}_{t+1,n}(x_i;h_n)]$ and $\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]=\mathsf{E }[\widehat{g}_{t+1,n}(x_i;h_n)]$, and therefore by Lemma 7

$$\begin{aligned} \max \limits _{\varDelta _n\le t \le n-\varDelta _n}W_{1,n}(t)\le & {} 2\max \limits _{\varDelta _n\le t \le n-\varDelta _n}\sum _{i=1}^{m}\left| {\widehat{\varphi }}_{1,t}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\right| ^2\\&+2\max \limits _{\varDelta _n\le t \le n-\varDelta _n}\sum _{i=1}^{m}\left| {\widehat{\varphi }}_{t+1,n}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{t+1,n}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{t+1,n}(x_i;h_n)]}\right| ^2\\= & {} O_{a.s.}\left( \frac{\log ^4 n}{nh_n}\right) . \end{aligned}$$

We complete the proof of the theorem. $\square$

Proof of Theorem 2

By definition,

$$\begin{aligned} \varLambda _{h_n}^2(x_i)= & {} \left( \mathsf{E }\left[ \left( \varphi _1(X_1)-\varphi _2(X_1)\right) K_{h_n}(x_i-X_1)\right] \right) ^2/(\mathsf{E }[K_{h_n}(X_1-x_i)])^2\\= & {} \left( \int _{{\mathbb {R}}}(\varphi _1(u)-\varphi _2(u))K_{h_n}(x_i-u)f(u)du\right) ^2/(\mathsf{E }[K_{h_n}(X_1-x_i)])^2\\= & {} \left( \int _{u\in {\mathcal {X}}\cap {\mathcal {Y}}}\left( \varphi _1(u)-\varphi _2(u)\right) K_{h_n}(x_i-u)f(u)du\right) ^2/(\mathsf{E }[K_{h_n}(X_1-x_i)])^2\\= & {} \left( \int _{(h_nz+x_i)\in {\mathcal {X}}\cap {\mathcal {Y}}}\left( \varphi _1(h_nz+x_i)-\varphi _2(h_nz+x_i)\right) K(z)f(h_nz+x_i)dz\right) ^2/\\&(\mathsf{E }[K_{h_n}(X_1-x_i)])^2. \end{aligned}$$

Note that $(\mathsf{E }[K_{h_n}(X_1-x_i)])$ is bounded. When $x_i\in {\mathcal {X}}\cap {\mathcal {Y}}$, both $\varphi _1(x)-\varphi _2(x)$ and f(x) are bounded away from zero in a small neighbour of $x_i$. Therefore $\varLambda _{h_n}^2(x_i)$ is also bounded away from zero when n is large enough.

Next we prove (3.2). Considering the special point $t=k$, it is obvious that

$$\begin{aligned} \max \limits _{\varDelta _n\le t< n-\varDelta _n}W_{1,n}(t)\ge W_{1,n}(k). \end{aligned}$$

Note that the sequences $\{ X_s,Y_s \}_{s=1}^k$ and $\{ X_s,Y_s \}_{s=k+1}^n$ are strictly stationary. By definition and Lemma 7, we have

$$\begin{aligned} W_{1,n}(k)= & {} \frac{k(n-k)}{n^2}\sum _{i=1}^m\left| {{\widehat{\varphi }}}_{1,k}(x_i;h_n)-{{\widehat{\varphi }}}_{k+1,n}(x_i;h_n)\right| ^2\nonumber \\= & {} \theta (1-\theta )\sum _{i=1}^m\left( \frac{\mathsf{E }[\widehat{g}_{1,k}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]}-\frac{\mathsf{E }[\widehat{g}_{k+1,n}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]}\right) ^2+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \nonumber \\= & {} \theta (1-\theta )\sum _{i=1}^m\varLambda ^2_{h_n}(x_i)+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.30)

Thus, we complete the proof of this theorem. $\square$

Proof of Theorem 3

(i) It is a direct corollary from Theorem 1 and 2 .

Now we prove (ii). From the statement in the proof of Theorem 2, we have

$$\begin{aligned} \max \limits _{\varDelta _n\le t\le k}W_{1,n}(t)\ge \theta (1-\theta )\sum _{i=1}^m\varLambda ^2_{h_n}(x_i) \end{aligned}$$

(6.31)

almost surely when $n\rightarrow \infty$. If we can show that

$$\begin{aligned} \max \limits _{\varDelta _n\le t\le k-n\varepsilon }W_{1,n}(t)<\theta (1-\theta )\sum _{i=1}^m\varLambda ^2_{h_n}(x_i) \end{aligned}$$

(6.32)

almost surely, for any small $\epsilon >0$ when $n\rightarrow \infty$, then (6.31) and (6.32) imply that ${\widehat{k}}\ge k-n\varepsilon$ almost surely when $n\rightarrow \infty$. Using the same method for the case when $k+n\varepsilon \le t\le n-\varDelta _n$, we can obtain ${\widehat{k}}\le k+n\varepsilon$ almost surely when $n\rightarrow \infty$. Combining these two inequalities we can show that $({\widehat{k}}-k)/n=o_{a.s.}(1)$ by letting $\varepsilon \rightarrow 0$.

Next we prove (6.32). When $t\le k-n\varepsilon$, on the one hand, we have

$$\begin{aligned} {\widehat{\varphi }}_{1,t}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}=O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \end{aligned}$$

(6.33)

uniformly in t by Lemma 7. On the other hand,

$$\begin{aligned}&{{\widehat{\varphi }}}_{t+1,n}(x_i;h_n)-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]} =\frac{{\widehat{g}}_{t+1,n}(x_i;h_n)}{{\widehat{f}}_{t+1,n}(x_i;h_n)}-\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\\&\quad =\frac{(k-t)\left( {\widehat{g}}_{t+1,k}(x_i;h_n)-\mathsf{E }[{\widehat{g}}_{t+1,k}(x_i;h_n)]\right) +(n-k)\left( {\widehat{g}}_{k+1,n}(x_i;h_n)-\mathsf{E }[{\widehat{g}}_{k+1,n}(x_i;h_n)]\right) }{(n-t){\widehat{f}}_{t+1,n}(x_i;h_n)}\\&\qquad +\frac{(k-t)\mathsf{E }[g_{t+1,k}(x_i;h_n)]+(n-k)\mathsf{E }[g_{k+1,n}(x_i;h_n)]-(n-t){\widehat{f}}_{t+1,n}(x_i;h_n)\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{(n-t){\widehat{f}}_{t+1,n}(x_i;h_n)}\\&\quad :=B_3(x_i)+B_4(x_i). \end{aligned}$$

We show that $B_3(x_i)$ is negligible and $B_4(x_i)$ is the leading term. Similar to the argument of proof for Lemma 5, we can show that the asymptotic order of ${\widehat{g}}_{t+1,k}(x_i;h_n)-\mathsf{E }[{\widehat{g}}_{t+1,k}(x_i;h_n)]$ is also $\frac{\log ^2 n}{\sqrt{nh_n}}$, as the sample size $k-t$ is of order n. Note that ${\widehat{f}}_{t+1,n}$ is bounded away from zero almost surely when n is large enough. Since $(k-t)/(n-t)<\theta /(1-\theta )$ and $(n-k)/(n-t)<1$, we have by Lemma 5 (or Lemma 6)

$$\begin{aligned} B_3(x_i)=O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.34)

In terms of $B_4(x_i)$, we have by Lemma 3

$$\begin{aligned} (n-t){\widehat{f}}_{t+1,n}(x_i;h_n)= & {} (k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]\\&+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]+O_{a.s.}\left( \frac{n\log n}{\sqrt{nh_n}}\right) . \end{aligned}$$

Therefore,

$$\begin{aligned} B_4(x_i)= & {} \frac{(k-t)\mathsf{E }[\widehat{g}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{g}_{k+1,n}(x_i;h_n)]}{(k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]} -\frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}\nonumber \\&+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \nonumber \\= & {} \frac{(k-t)\mathsf{E }[\widehat{g}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{g}_{k+1,n}(x_i;h_n)]}{(k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]}\nonumber \\&-\frac{\left( (k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]\right) \frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]}}{(k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]}\nonumber \\&+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \nonumber \\= & {} \frac{(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]}{(k-t)\mathsf{E }[\widehat{f}_{1,k}(x_i;h_n)]+(n-k)\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]} \cdot \nonumber \\&\left( \frac{\mathsf{E }[\widehat{g}_{k+1,n}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{k+1,n}(x_i;h_n)]}- \frac{\mathsf{E }[\widehat{g}_{1,t}(x_i;h_n)]}{\mathsf{E }[\widehat{f}_{1,t}(x_i;h_n)]} \right) \nonumber \\&+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) . \end{aligned}$$

(6.35)

Combining (6.33)–(6.35), we can obtain uniformly in t

$$\begin{aligned} W_{1,n}(t)= & {} \frac{t(n-t)}{n^2}\sum \limits _{i=1}^{m}|{{\widehat{\varphi }}}_{1,t}(x_i;h_n)- {{\widehat{\varphi }}}_{t+1,n}(x_i;h_n)|^2\nonumber \\= & {} \frac{t(n-t)}{n^2}\sum \limits _{i=1}^{m}B_4^2(x_i)+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \nonumber \\= & {} \frac{t(n-t)}{n^2}\cdot \frac{(n-k)^2}{(n-t)^2}\sum \limits _{i=1}^{m}\varLambda ^2_{h_n}(x_i)+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) \nonumber \\= & {} \frac{t(n-k)^2}{n^2(n-t)}\sum \limits _{i=1}^{m}\varLambda ^2_{h_n}(x_i)+O_{a.s.}\left( \frac{\log ^2 n}{\sqrt{nh_n}}\right) . \end{aligned}$$

Note that

$$\begin{aligned} \frac{t(n-k)^2}{n^2(n-t)}\le (1-\theta )^2\frac{\theta -\varepsilon }{1-\theta +\varepsilon } =(1-\theta )\cdot \frac{1-\theta }{1-\theta +\varepsilon }\cdot (\theta -\varepsilon )<\theta (1-\theta ). \end{aligned}$$

Thus, we have (6.32), and we complete the proof. $\square$