Kolmogorov–Smirnov simultaneous confidence bands for time series distribution function

Abstract

Claims about distributions of time series are often unproven assertions instead of substantiated conclusions for lack of hypotheses testing tools. In this work, Kolmogorov–Smirnov type simultaneous confidence bands (SCBs) are constructed based on simple random samples (SRSs) drawn from realizations of time series, together with smooth SCBs using kernel distribution estimator (KDE) instead of empirical cumulative distribution function of the SRS. All SCBs are shown to enjoy the same limiting distribution as the standard Kolmogorov–Smirnov for i.i.d. sample, which is validated in simulation experiments on various time series. Computing these SCBs for the standardized S&P 500 daily returns data leads to some rather unexpected findings, i.e., student’s t-distributions with degrees of freedom no less than 3 and the normal distribution are all acceptable versions of the standardized daily returns series’ distribution, with proper rescaling. These findings present challenges to the long held belief that daily financial returns distribution is fat-tailed and leptokurtic.

Appendix

Throughout this section, c denotes any positive constant and $\mathcal {O}_{p}$ (or $o_{p}$) a sequence of random variables of certain order in probability. In addition, $u_{p}$ denotes a sequence of random functions which are $o_{p}$ uniformly defined in the domain. For any continuous function $\phi$ defined on an interval $\mathcal {I}$, the modulus of continuity is defined as $\omega \left( \phi ,\varDelta \right) =\sup _{x,x^{\prime }\in \mathcal {I} ,\left| x-x^{\prime }\right| \le \varDelta }\left| \phi \left( x^{\prime }\right) -\phi \left( x\right) \right|$

Lemma 1

(Theorem 7.1,2, Brockwell and Davis (1991)) If $\left\{ X_t\right\} _{t=1}^n$ is the stationary process,

$$\begin{aligned} X_t = \mu +\sum _{j=-\infty }^{\infty }\psi _j \varepsilon _{t-j}, \quad \varepsilon _{t}\sim \text {IID}(0,\sigma ^2) \end{aligned}$$

with $\sum _{j=-\infty }^{\infty }\left| \psi _j\right| <\infty$ and $\sum _{j=-\infty }^{\infty }\left| \psi _j\right| \ne 0$, then $\overline{X}_n$ is AN$\left( \mu ,n^{-1}v\right)$, where $\overline{X}_n=n^{-1}\sum _{t=1}^n X_t$, $v=\sum _{h=-\infty }^{\infty }\gamma (h)=\sigma ^2\left( \sum _{j=-\infty }^{\infty }\psi _j \right) ^2$, and $\gamma (\cdot )$ is the autocovariance function of $\left\{ X_t\right\} _{t=1}^n$.

Lemma 2

(Theorem 8.1, Brockwell and Davis (1991)) If $\left\{ X_t\right\} _{t=1}^n$ is the zero-mean causal AR(p) process,

$$\begin{aligned} X_t -\phi _1X_{t-1}-\cdots -\phi _pX_{t-p}=Z_t, \quad Z_{t}\sim \text {IID}(0,\sigma ^2) \end{aligned}$$

and $\hat{\varvec{\phi }}$ is the Yule-Walker estimator of ${\varvec{\phi }}$, that is $\hat{\varvec{\phi }}={\varvec{\varGamma }}_p^{-1}\hat{\varvec{\gamma }}_p$ with $\hat{\varvec{\varGamma }}_p=\left\{ {\hat{\gamma }}(i-j)\right\} _{i,j=1}^p$ and $\hat{\varvec{\gamma }}_p=\left( {\hat{\gamma }}(1),\ldots ,{\hat{\gamma }}(p)\right) ^{\top }$, then

$$\begin{aligned} \sqrt{n}\left( \hat{\varvec{\phi }}-{\varvec{\phi }}\right) \overset{d}{ \rightarrow } N\left( 0, \sigma ^2 {\varvec{\varGamma }}_p^{-1}\right) \end{aligned}$$

where ${\varvec{\varGamma }}_p$ is the covariance matrix with ${\varvec{\varGamma }}_p=\left\{ \gamma (i-j)\right\} _{i,j=1}^p$. Moreover,

$$\begin{aligned} {\hat{\sigma }} ^2 \overset{p}{ \rightarrow } \sigma ^2, \end{aligned}$$

where ${\hat{\sigma }}^2={\hat{\gamma }}_0-\hat{\varvec{\phi }}^{\top }\hat{{\varvec{\gamma }}}_p$.

1.1 A.1 Preliminary results on weak convergence

The next weak convergence result extends (1) of the Donsker’s Theorem to strongly mixing time series.

Lemma 3

(Deo 1973) Let $\{\xi _n:-\infty<n <\infty \}$ be a strictly stationary sequence of random variables, $\{F_n\left( t\right) : 0\le t\le 1\}$ be the empirical process for $\xi _1, \xi _2, \ldots , \xi _n$, i.e., $F_n(t) =n^{-1}\sum _{i=1}^nI_{\left[ 0,t\right] }\left( \xi _i\right)$ where $I_{\left[ 0,t\right] }\left( \cdot \right)$ is the indicator function of the interval [0, t]. Suppose that $0\le \xi _0 \le 1$ and $\xi _0$ have continuous distribution function F with $F(0)=0$ and $F(1)=1$. Normalize $F_n(t)$ as

$$\begin{aligned} Y_n(t)=n^{1/2}\left( F_n\left( t\right) -F\left( t\right) \right) , \quad \quad 0\le t\le 1. \end{aligned}$$

For $0\le t\le 1$, define the function $g_t$ by

$$\begin{aligned} g_t(x)=I_{\left[ 0,t\right] }\left( x\right) -F(t), \end{aligned}$$

and suppose further that $\{\xi _n\}$ satisfies the mixing condition

$$\begin{aligned} \sum _{n=1}^{\infty } n^2\alpha (n)^{1/2-\tau }<\infty \quad \quad for ~ some\quad \tau \in \left( 0,1/2\right) . \end{aligned}$$

Then the sequence $\{Y_n(t): 0\le t\le 1\}$ of normalized empirical processes converges weakly in $\mathcal {D}[0,1]$ to a Gaussian random function $\{Y(t): 0\le t\le 1\}$ specified by $\mathbb {E}\left( Y\left( t\right) \right) =0$ and

$$\begin{aligned} \begin{aligned} \mathbb {E}\{Y(s)Y(t)\}&=\mathbb {E}\{g_s\left( \xi _0\right) g_t\left( \xi _0\right) \}+\sum _{k=1}^{\infty } \mathbb {E}\{g_s\left( \xi _0\right) g_t\left( \xi _k\right) \}\\&\quad +\sum _{k=1}^{\infty } \mathbb {E}\{g_s\left( \xi _k\right) g_t\left( \xi _0\right) \} \end{aligned} \end{aligned}$$

(14)

Furthermore, the series in (14) converges absolutely and the sample paths of Y are continuous with probability one.

The following lemma yields a uniformly continuous Gaussian limiting process $\zeta \left( \cdot \right)$ on $\mathbb {R}$ for the empirical process $N_{k}^{1/2}\left\{ F_{N_{k}}\left( \cdot \right) -F\left( \cdot \right) \right\}$, which is used in the proof of Theorem 2.

Lemma 4

Under Assumptions (A1) and (A2), there exists a mean-zero Gaussian process $Y\left( \cdot \right)$ whose sample path is continuous on $\left[ 0,1\right]$ with probability one such that as $k\rightarrow \infty$, $N_{k}^{1/2}\left\{ F_{N_{k}}\left( \cdot \right) -F\left( \cdot \right) \right\} \overset{d}{ \rightarrow }\zeta \left( \cdot \right) =Y\left( F\left( \cdot \right) \right)$. Furthermore, the process $\zeta \left( \cdot \right)$ is uniformly continuous on $\mathbb {R}$ with modulus of continuity $\omega \left( \zeta , \varDelta \right) \le \omega \left( Y, \omega \left( F,\varDelta \right) \right) \rightarrow 0 \ a.s.\ \text { as } \varDelta \rightarrow 0$.

Proof. Define a transformed time series $u_{i}=F\left( x_{t}\right)$, $i=0,\pm 1,\pm 2,\ldots$. For any $x\in \mathbb {R}$, let $t=F(x)\in [0,1]$, then $F_{N_{k}}(x)=F_{U,N_{k}}(t)$, in which

$$\begin{aligned} F_{U,N_{k}}(t)=N_{k}^{-1}\sum _{i=1}^{N_{k}}I\left\{ u_{i}\le t\right\} . \end{aligned}$$

The $\alpha$-mixing coefficients for $\left\{ u_{i}\right\} _{i=-\infty }^{\infty }$ is the same as those for $\left\{ x_{i}\right\} _{i=-\infty }^{\infty }$, which satisfy Assumption (A1) that $\alpha (n) \ll n^{ -6 -\epsilon }$, hence there exists $\tau \in \left( 0,1/2\right)$ such that $\alpha (n)^{1/2-\tau } \ll n^{-3}$, and thus $\sum _{n=1}^{\infty } n^2\alpha (n)^{1/2-\tau }<\infty$. Then applying Lemma 3 with $\xi _{i}$ replaced by $u_{i}$, one has $N_k^{1/2}\{F_{U,N_{k}}(t)-t\}\rightarrow Y(t)$.

Define $\zeta (x)=Y\left( F\left( x\right) \right)$, then $N_{k}^{1/2}\left\{ F_{N_{k}}\left( \cdot \right) -F\left( \cdot \right) \right\} \overset{d}{ \rightarrow }\zeta \left( \cdot \right)$ as $k\rightarrow \infty$ and

$$\begin{aligned}&\sup _{x, x'\in \mathbb {R}, \left| x-x'\right| \le \varDelta } \Big \vert \zeta \left( x\right) -\zeta \left( x'\right) \Big \vert =\sup _{x, x'\in \mathbb {R}, \left| x-x'\right| \le \varDelta } \Big \vert Y\left( F\left( x\right) \right) -Y\left( F\left( x'\right) \right) \Big \vert \\&\quad \le \sup _{t, t'\in \left[ 0,1 \right] , \left| t-t'\right| \le \omega \left( F,\varDelta \right) } \Big \vert Y\left( t\right) -Y\left( t'\right) \Big \vert \le \omega \left( Y, \omega \left( F,\varDelta \right) \right) \end{aligned}$$

The uniform continuity of $F(\cdot )$ is guaranteed by Assumption (A2), and almost sure uniform continuity of $Y(\cdot )$ by the fact that sample paths of $Y(\cdot )$ are almost surely continuous over the compact interval $\left[ 0,1\right]$. These facts imply that $\omega \left( Y, \omega \left( F,\varDelta \right) \right) \rightarrow 0 \ a.s.\ \text { as } \varDelta \rightarrow 0$, thus $\zeta$ is continuous with probability one and $\omega \left( \zeta , \varDelta \right) \le \omega \left( Y, \omega \left( F,\varDelta \right) \right)$.

1.2 A.2 Proof of Theorem 1

Define a transformed time series $u_{i}=F\left( x_{i}\right)$, $i=0,\pm 1,\pm 2,\ldots$ and for any $k=1,2,\ldots$, a finite population $\pi _{k,U}=\{u_{1},u_{2},\ldots \text {,}$ $u_{N_{k}}\}$ together with a simple random sample $U_{i}=F\left( X_{i}\right)$, $1\le i\le n_{k}$ from population $\pi _{k,U}$. For any $x\in \mathbb {R}$, let $t=F(x)\in \left[ 0,1 \right]$, then

$$\begin{aligned} F_{N_{k}}(x)=F_{U,N_{k}}(t),F_{n_{k}}(x)=F_{U,n_{k}}(t), \end{aligned}$$

(15)

in which

$$\begin{aligned} F_{U,N_{k}}(t)= & {} N_{k}^{-1}\sum _{i=1}^{N_{k}}I\left\{ u_{i}\le t\right\} , \end{aligned}$$

(16)

$$\begin{aligned} F_{U,n_{k}}(t)= & {} n_{k}^{-1}\sum _{i=1}^{n_{k}}I\left\{ U_{i}\le t\right\} . \end{aligned}$$

(17)

By Assumption (A1), the time series $\left\{ u_{t},t=0,\pm 1,\pm 2,\ldots \right\}$ is ergodic and has stationary distribution $\mathcal {U}\left( 0,1\right)$, hence almost surely $\lim _{k\rightarrow \infty }F_{U,N_{k}}(t)=t$ for $0\le t\le 1$. As $\lim _{k\rightarrow \infty }\min \left( n_{k},N_{k}-n_{k}\right) =\infty$ is contained in Assumption (A3), applying Theorem 14.1 of Rosén (1964), one obtains that as random elements taking values in the space $\mathcal {D}\left[ 0,1\right]$ of cadlag functions:

$$\begin{aligned} \lambda _{k}\left\{ F_{U,n_{k}}(t)-F_{U,N_{k}}(t)\right\} \overset{d}{ \rightarrow }B(t) \end{aligned}$$

almost surely. Lastly, Skorohod’s Representation Theorem (Theorem 6.7, Billingsley 1999) provides versions $B_{k}^{*}$ of Brownian bridge such that

$$\begin{aligned} \sup _{t\in \left[ 0,1\right] }\left| \lambda _{k}\left\{ F_{U,n_{k}}(t)-F_{U,N_{k}}(t)\right\} -B_{k}^{*}(t)\right| \rightarrow 0,\,a.s. \end{aligned}$$

which implies that

$$\begin{aligned} \sup _{x\in \mathbb {R}}\left| l_{k}\left\{ F_{n_{k}}(x)-F_{N_{k}}(x)\right\} -B_{k}^{*}(F\left( x\right) )\right| \rightarrow 0,\,a.s. \end{aligned}$$

The Theorem 1 is proved.

1.3 A.3 Proof of Theorem 2

Lemma 5

Under Assumptions (A1) to (A3), (A5), as $k \rightarrow \infty$,

$$\begin{aligned} \sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\Big \vert \left\{ F_{n_{k}}\left( x-hw\right) -F_{n_{k}}\left( x\right) \right\} -\left\{ F_{N_{k}}\left( x-hw\right) -F_{N_{k}}(x)\right\} \Big \vert =o_{p}\left( l_{k}^{-1}\right) . \end{aligned}$$

Proof

For the Brownian bridges $B_{k}^{*}\left\{ \cdot \right\}$ in Theorem 1,

$$\begin{aligned}&\sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\left| l_{k}\left\{ F_{n_{k}}\left( x-hw\right) -F_{N_{k}}\left( x-hw\right) \right\} -l_{k}\left\{ F_{n_{k}}(x)-F_{N_{k}}(x)\right\} \right| \nonumber \\&\quad \le \sup _{x,x^{\prime }\in \mathbb {R},\left| x-x^{\prime }\right| \le h}\left| l_{k}\left\{ F_{n_{k}}\left( x^{\prime }\right) -F_{N_{k}}\left( x^{\prime }\right) \right\} -l_{k}\left\{ F_{n_{k}}(x)-F_{N_{k}}\left( x\right) \right\} \right| \nonumber \\&\quad \le 2\sup _{x}\left| l_{k}\left\{ F_{n_{k}}\left( x\right) -F_{N_{k}}(x)\right\} -B_{k}^{*}\left\{ F(x)\right\} \right| \nonumber \\&\qquad \text { }+\sup _{x,x^{\prime }\in \mathbb {R},\left| x-x^{\prime }\right| \le h}\left| B_{k}^{*}\left\{ F\left( x^{\prime }\right) \right\} -B_{k}^{*}\left\{ F(x)\right\} \right| . \end{aligned}$$

(18)

Since $F\left( \cdot \right)$ is uniformly continuous by Assumption (A2), and Assumption (A5) implies that $h \rightarrow 0$ as $k\rightarrow \infty$, so $\omega \left( F,h\right) \rightarrow 0$ as $k\rightarrow \infty$. Assumptions (A1), (A3) ensure Theorem 1, so $l_{k}\left\{ F_{n_{k}}\left( \cdot \right) -F_{N_{k}}\left( \cdot \right) \right\} -B_{k}^{*}\left\{ F\left( \cdot \right) \right\} \rightarrow 0\ a.s.$ as $k\rightarrow \infty$. Thus, the expression in (6) is bounded by

$$\begin{aligned}&2\sup _{x\in \mathbb {R}}\left| l_{k}\left\{ F_{n_{k}}(x)-F_{N_{k}}(x)\right\} -B_{k}^{*}\left\{ F(x)\right\} \right| +\sup _{t,t^{\prime }\in \left[ 0,1\right] ,\left| t-t^{\prime }\right| \le \omega \left( F,h\right) }\left| B_{k}^{*}\left( t^{\prime }\right) -B_{k}^{*}\left( t\right) \right| \\&\quad =o_{a.s.}\left( 1\right) +o_{p}\left( 1\right) =o_{p}\left( 1\right) \text {.} \end{aligned}$$

In other words,

$$\begin{aligned} \sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\Big \vert \left\{ F_{n_{k}}\left( x-hw\right) -F_{n_{k}}\left( x\right) \right\} -\left\{ F_{N_{k}}\left( x-hw\right) -F_{N_{k}}(x)\right\} \Big \vert =o_{p}\left( l_{k}^{-1}\right) . \end{aligned}$$

(19)

$\square$

Lemma 6

Under Assumptions (A1), (A2), (A5), as $k \rightarrow \infty$,

$$\begin{aligned} \sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\Big \vert \left\{ F_{N_{k}}\left( x-hw\right) -F_{N_{k}}\left( x\right) \right\} -\left\{ F\left( x-hw\right) -F(x)\right\} \Big \vert =o_{p}\left( N_{k}^{-1/2}\right) . \end{aligned}$$

Proof

Next, since Lemma 4 implies that $N_{k}^{1/2}\left\{ F_{N_{k}}\left( \cdot \right) -F\left( \cdot \right) \right\} \overset{d}{ \rightarrow }\zeta \left( \cdot \right)$, Skorohod’s Representation Theorem (Theorem 6.7, Billingsley 1999) provides versions $\zeta _{k}\left( \cdot \right)$ of $\zeta \left( \cdot \right)$ such that

$$\begin{aligned} \sup _{x\in \mathbb {R} }\left| N_{k}^{1/2}\left\{ F_{N_{k}}\left( x \right) - F\left( x \right) \right\} -\zeta _{k}\left( x \right) \right| \rightarrow 0, \ a.s. \end{aligned}$$

Consequently

$$\begin{aligned}&\sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\left| N_{k}^{1/2}\left\{ F_{N_{k}}\left( x-hw\right) -F\left( x-hw\right) \right\} -N_{k}^{1/2}\left\{ F_{N_{k}}\left( x\right) -F\left( x\right) \right\} \right| \\&\quad \le \sup _{x,x^{\prime }\in \mathbb {R},\left| x-x^{\prime }\right| \le h}\left| N_{k}^{1/2}\left\{ F_{N_{k}}\left( x^{\prime }\right) -F\left( x^{\prime }\right) \right\} -N_{k}^{1/2}\left\{ F_{N_{k}}\left( x\right) -F\left( x\right) \right\} \right| \\&\quad \le 2\sup _{x\in \mathbb {R}}\left| N_{k}^{1/2}\left\{ F_{N_{k}}\left( x\right) -F\left( x\right) \right\} -\zeta _{k}\left( x\right) \right| +\omega \left( \zeta _{k},h\right) =o_{p}\left( 1\right) . \end{aligned}$$

Hence the following holds

$$\begin{aligned} \sup _{w\in \left[ -1,1\right] ,x\in \mathbb {R}}\Big \vert \left\{ F_{N_{k}}\left( x-hw\right) -F_{N_{k}}\left( x\right) \right\} -\left\{ F\left( x-hw\right) -F(x)\right\} \Big \vert =o_{p}\left( N_{k}^{-1/2}\right) . \end{aligned}$$

(20)

Lemma 7

Under the Assumptions (A2), (A5) and (A6), as $k\rightarrow \infty$,

$$\begin{aligned} \sup _{x\in \mathbb {R}}\left| \int _{-1}^{1}\{F\left( x-hw\right) -F\left( x\right) \} K\left( w\right) dw\right| =o\left( l_k^{-1}\right) . \end{aligned}$$

Proof

According to the assumptions of the cumulative distribution function, we discuss the problem in two cases.

Case 1: $\nu \ge 1$. Note that by Assumption (A6) $\int _{-1}^{1}K\left( w\right) w^{r}dw\equiv 0,r=1,\ldots ,l-1$, and by Assumption (A2) $F(\cdot )\in C^{\left( \nu ,\mu \right) }\left( \mathbb {R} \right)$. Hence

$$\begin{aligned}&\int _{-1}^{1}\left\{ F\left( x-hw\right) -F(x)\right\} K\left( w\right) dw\\&\quad =\int _{-1}^{1}\left\{ F\left( x-hw\right) -\sum _{r=0}^{\nu -1}\frac{ F^{\left( r\right) }(x)}{r!}\left( -hw\right) ^{r}\right\} K\left( w\right) dw\text {{}} \\&\quad =\int _{-1}^{1}\left\{ \int _{x}^{x-hw}\frac{F^{\left( \nu \right) }(t)}{ (\nu -1)!}\left( x-hw-t\right) ^{\nu -1}dt\right\} K\left( w\right) dw \\&\quad =\int _{-1}^{1}\left\{ \frac{F^{\left( \nu \right) }(x)}{(\nu -1)!}\left( -hw\right) ^{\nu }+\int _{x}^{x-hw}\frac{F^{\left( \nu \right) }(t)-F^{\left( \nu \right) }(x)}{(\nu -1)!}\left( x-hw-t\right) ^{\nu -1}dt\right\} K\left( w\right) dw \\&\quad =\int _{-1}^{1}\left\{ \int _{x}^{x-hw}\frac{F^{\left( \nu \right) }(t)-F^{\left( \nu \right) }(x)}{(\nu -1)!}\left( x-hw-t\right) ^{\nu -1}dt\right\} K\left( w\right) dw \end{aligned}$$

Furthermore, by Assumption (A2) $F^{\left( \nu \right) }(\cdot )\in C^{\left( 0,\mu \right) }\left( \mathbb {R}\right)$ and

$$\begin{aligned}&\sup _{x\in \mathbb {R}}\left| \int _{-1}^{1}\left\{ F\left( x-hw\right) -F\left( x\right) \right\} K\left( w\right) dw\right| \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}\left| \int _{x}^{x-hw}\frac{ F^{\left( \nu \right) }(t)-F^{\left( \nu \right) }(x)}{(\nu -1)!}\left( x-hw-t\right) ^{\nu -1}dt\right| K\left( w\right) dw \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}\left| (hw)^{\nu }\sup _{x\le t\le x-hw}\frac{\left| F^{\left( \nu \right) }(t)-F^{\left( \nu \right) }(x)\right| }{(\nu -1)!}\right| K\left( w\right) dw \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}\left| (hw)^{\nu }\sup _{x\le t\le x-hw}\frac{C\left| t-x\right| ^{\mu }}{(\nu -1)!}\right| K\left( w\right) dw \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}\left| (hw)^{\nu }\frac{ C(hw)^{\mu }}{(\nu -1)!}\right| K\left( w\right) dw \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}ch^{\nu +\mu }\left| w\right| ^{\nu +\mu }K\left( w\right) dw=\mathcal {O}\left( h^{\nu +\mu }\right) =o\left( l_{k}^{-1}\right) , \end{aligned}$$

(21)

which follows from Assumption (A5) that $\lim _{k\rightarrow \infty }l_{k}h_{n_{k}}^{\nu +\mu }=0$.

Case 2: $\nu =0$. By Assumption (A2) $F(x)\in C^{\left( 0,\mu \right) }\left( \mathbb {R}\right)$. Hence

$$\begin{aligned}&\sup _{x\in \mathbb {R}}\left| \int _{-1}^{1}\{F\left( x-hw\right) -F\left( x\right) \} K\left( w\right) dw\right| \nonumber \\&\quad \le \sup _{x\in \mathbb {R}}\int _{-1}^{1}C\left( hw\right) ^{\mu } K\left( w\right) dw \nonumber \\&\quad =\mathcal {O}\left( h^{\nu +\mu }\right) =o\left( l_{k}^{-1}\right) , \end{aligned}$$

(22)

which follows from Assumption (A5) that $\lim _{k\rightarrow \infty }l_{k}h_{n_{k}}^{\nu +\mu }=0$. $\square$

Proof of Theorem 2

Define $G(x)=\int _{-\infty }^{x}K\left( u\right) du$. By the definition of $\hat{F}_{k}(x)$, one obtains

$$\begin{aligned} \hat{F}_{k}(x)=n^{-1}\sum \limits _{i=1}^{n_{k}}\int _{-\infty }^{x}K_{h}\left( u-X_{i}\right) du=n_{k}^{-1}\sum \limits _{i=1}^{n_{k}}G\left( \frac{x-X_{i}}{h }\right) \text {.} \end{aligned}$$

Therefore, by the definition of $F_{n_{k}}(x)=n_{k}^{-1}\sum _{i=1}^{n_{k}}I \left( X_{i}\le x\right)$ in (4)

$$\begin{aligned} \hat{F}_{k}(x)&=\int _{-\infty }^{+\infty }G\left( \frac{x-u}{h}\right) dF_{n_{k}}\left( u\right) =\int _{-\infty }^{+\infty }h^{-1}K\left( \frac{x-u }{h}\right) F_{n_{k}}\left( u\right) du \\&=\int _{-1}^{1}K\left( w\right) F_{n_{k}}\left( x-hw\right) dw \end{aligned}$$

using integration by parts and a change of variable $w=\left( x-u\right) /h$ . The following decomposition plays an important role:

$$\begin{aligned} \hat{F}_{k}(x)-F_{n_{k}}(x)=\int _{-1}^{1}\left\{ F_{n_{k}}\left( x-hw\right) -F_{n_{k}}(x)\right\} K\left( w\right) dw. \end{aligned}$$

(23)

Since Assumption (A4) requires that $n_{k}/N_{k}=o\left( 1\right)$ and consequently $N_{k}^{-1/2}=o\left( l_{k}^{-1}\right)$, using Lemma 5 and Lemma 6 together with the triangle inequality imply that as $k\rightarrow \infty$

$$\begin{aligned} \left| \left\{ F_{n_{k}}\left( x-hw\right) -F_{n_{k}}(x)\right\} -\left\{ F\left( x-hw\right) -F(x)\right\} \right| =u_{p}\left( l_{k}^{-1}\right) . \end{aligned}$$

(24)

By Lemma 7 and applying (23), (24), (21) and (22), the following holds

$$\begin{aligned} \sup _{x\in \mathbb {R}}\left| \hat{F}_{k}(x)-F_{n_{k}}(x)\right| =\sup _{x\in \mathbb {R}}\left| \int _{-1}^{1}\left\{ F_{n_{k}}\left( x-hw\right) -F_{n_{k}}(x)\right\} K\left( w\right) dw\right| =o_{p}\left( l_{k}^{-1}\right) \text {.} \end{aligned}$$

Applying Theorem 1, one has $l_{k}\left\{ \hat{F} _{k}(x)-F_{N_{k}}(x)\right\} \overset{d}{\rightarrow }B\left\{ F(x)\right\}$ , proving (11).

Notice that under Assumption (A4), $n_{k}^{-1/2}/l_{k}^{-1}\rightarrow 1$, $N_{k}^{-1/2}=o\left( l_{k}^{-1}\right)$, $N_{k}^{-1/2}=o\left( n_{k}^{-1/2}\right)$ as $k\rightarrow \infty$, and that $N_{k}^{1/2}\left\{ F_{N_{k}}\left( \cdot \right) -F\left( \cdot \right) \right\} \overset{d}{\rightarrow }\zeta \left( \cdot \right)$ by Lemma . Hence, as $k\rightarrow \infty$

$$\begin{aligned} n_{k}^{1/2}D\left( F_{N_{k}},F\right) =n_{k}^{1/2}\mathcal {O}_{p}\left( N_{k}^{-1/2}\right) =\mathcal {O}_{p}\left( n_{k}^{1/2}N_{k}^{-1/2}\right) =o_{p}\left( 1\right) \text {.} \end{aligned}$$

Likewise, $l_{k}D\left( F_{N_{k}},F\right) =o_{p}\left( 1\right)$. These, together with (11) establish (12). The proof of Theorem 2 is complete by applying Slutsky’s Theorem.