Estimation and diagnostic for partially linear models with first-order autoregressive skew-normal errors

Abstract

Estimation and diagnostic procedures for partially linear models with first-order autoregressive [AR(1)] skew-normal errors are proposed in this paper. An EM iterative process with analytic expressions for the M and E-steps, which combines back-fitting and Newton–Raphson algorithms, is developed for the parameter estimation. A linear smoother for the estimation of the effective degrees of freedom concerning the nonparametric component is derived from the iterative process. Local influence analysis is developed based on the conditional expectation of the complete-data log-likelihood function, used in the EM algorithm. A simulation study is also conducted to evaluate the efficiency of the EM algorithm. Finally, the methodology developed through the paper is illustrated with a real data set on daily ozone concentration.

Appendices

Appendix A: Approximate standard errors

The variance–covariance matrix of the $\widehat{\varvec{\theta }}$, corresponding to the inverse of the observed information matrix, is obtained by treating the penalized likelihood as a usual likelihood Segal et al. (1994). Given the observed log-likelihood function $\ell (\varvec{\theta })$ in (5), the correspondence penalized log-likelihood function of $\varvec{\theta }=(\varvec{\beta }^\top ,\varvec{\gamma }^\top ,\sigma ^2,\delta ,\rho )^\top$ is of the form

$$\begin{aligned} \ell _p(\varvec{\theta })=\ell (\varvec{\theta })-\frac{\alpha }{2}\varvec{\gamma }^\top \mathbf {K}\varvec{\gamma }. \end{aligned}$$

(15)

The observed information matrix for $\varvec{\theta }$ may be written as

$$\begin{aligned} \mathbf {I}_{\varvec{\theta }\varvec{\theta }}=-\displaystyle \frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \varvec{\theta }\partial \varvec{\theta }^\top } \end{aligned}$$

with elements

$$\begin{aligned}&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \varvec{\beta }\partial \varvec{\beta }^\top }=\sum ^n_{i=1}\left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \frac{\partial B_i}{\partial \varvec{\beta }}\frac{\partial B_i}{\partial \varvec{\beta }^\top },\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \varvec{\gamma }\partial \varvec{\beta }^\top }=\sum ^n_{i=1}\left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \frac{\partial B_i}{\partial \varvec{\gamma }}\frac{\partial B_i}{\partial \varvec{\beta }^\top },\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \varvec{\gamma }\partial \varvec{\gamma }^\top }=\sum ^n_{i=1}\left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \frac{\partial B_i}{\partial \varvec{\gamma }}\frac{\partial B_i}{\partial \varvec{\gamma }^\top }-\alpha \mathbf {K},\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \rho ^2}=\sum ^n_{i=1}\left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \left( \frac{\partial B_i}{\partial \rho }\right) ^2,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \rho \partial \varvec{\beta }}=\sum ^n_{i=1}\left[ \left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \frac{\partial B_i}{\partial \rho }\frac{\partial B_i}{\partial \varvec{\beta }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \rho \partial \varvec{\beta }}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \rho \partial \varvec{\gamma }}=\sum ^n_{i=1}\left[ \left( -\frac{\sigma ^2}{\delta ^2}+W_\Phi '(B_i)\right) \frac{\partial B_i}{\partial \rho }\frac{\partial B_i}{\partial \varvec{\gamma }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \rho \partial \varvec{\gamma }}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \sigma ^2\partial \varvec{\beta }}=\sum ^n_{i=1}\left[ \left( -\frac{1}{\delta ^2}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \sigma ^2}+W_\Phi '(B_i)\frac{\partial B_i}{\partial \sigma ^2}\right) \frac{\partial B_i}{\partial \varvec{\beta }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \varvec{\beta }}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \sigma ^2\partial \varvec{\gamma }}=\sum ^n_{i=1}\left[ \left( -\frac{1}{\delta ^2}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \sigma ^2}+W_\Phi '(B_i)\frac{\partial B_i}{\partial \sigma ^2}\right) \frac{\partial B_i}{\partial \varvec{\gamma }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \varvec{\gamma }}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \sigma ^2\partial \rho }=\sum ^n_{i=1}\left[ \left( -\frac{1}{\delta ^2}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \sigma ^2}+W_\Phi '(B_i)\frac{\partial B_i}{\partial \sigma ^2}\right) \frac{\partial B_i}{\partial \rho } + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \rho }\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \delta \partial \varvec{\beta }}=\sum ^n_{i=1}\left[ \left( \frac{2\sigma ^2}{\delta ^3}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \delta }+W_\Phi '(B_i)\frac{\partial B_i}{\partial \delta }\right) \frac{\partial B_i}{\partial \varvec{\beta }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \delta \partial \varvec{\beta }}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \delta \partial \varvec{\gamma }}=\sum ^n_{i=1}\left[ \left( \frac{2\sigma ^2}{\delta ^3}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \delta }+W_\Phi '(B_i)\frac{\partial B_i}{\partial \delta }\right) \frac{\partial B_i}{\partial \varvec{\gamma }} + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \delta \partial \mathbf {f}}\right] ,\\&\frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \delta \partial \rho }=\sum ^n_{i=1}\left[ \left( \frac{2\sigma ^2}{\delta ^3}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \delta }+W_\Phi '(B_i)\frac{\partial B_i}{\partial \delta }\right) \frac{\partial B_i}{\partial \rho } + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \delta \partial \rho }\right] , \end{aligned}$$

$$\begin{aligned} \frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \sigma ^4}= & {} \frac{n}{2(\sigma ^2+\delta ^2)^2}-\frac{1}{\delta ^2}\sum ^n_{i=1}B_i\frac{\partial B_i}{\partial \sigma ^2}\\&+ \sum ^n_{i=1}\left[ \left( -\frac{1}{\delta ^2}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \sigma ^2}+W_\Phi '(B_i)\frac{\partial B_i}{\partial \sigma ^2}\right) \frac{\partial B_i}{\partial \sigma ^2} \right. \\&\left. + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \sigma ^4}\right] , \end{aligned}$$

$$\begin{aligned} \frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \delta \partial \sigma ^2}= & {} \frac{n\delta }{(\sigma ^2+\delta ^2)^2}+\sum ^n_{i=1}\left( \frac{1}{\delta ^3}B_i^2-\frac{1}{\delta ^2}B_i\frac{\partial B_i}{\partial \delta }\right) \\&+\sum ^n_{i=1}\left[ \left( \frac{2\sigma ^2}{\delta ^3}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \delta }+W_\Phi '(B_i)\frac{\partial B_i}{\partial \delta }\right) \frac{\partial B_i}{\partial \sigma ^2}\right. \\&\left. + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \delta \partial \sigma ^2}\right] , \end{aligned}$$

$$\begin{aligned} \frac{\partial ^2 \ell _{p}(\varvec{\theta })}{\partial \delta ^2}= & {} -\frac{n(\sigma ^2-\delta ^2)}{(\sigma ^2+\delta ^2)^2}+\frac{\sigma ^2}{\delta ^3}\sum ^n_{i=1}\left( -\frac{3}{\delta }B_i^2+2B_i\frac{\partial B_i}{\partial \delta }\right) \\&+\sum ^n_{i=1}\left[ \left( \frac{2\sigma ^2}{\delta ^3}B_i-\frac{\sigma ^2}{\delta ^2}\frac{\partial B_i}{\partial \delta }+W_\Phi '(B_i)\frac{\partial B_i}{\partial \delta }\right) \frac{\partial B_i}{\partial \delta } \right. \\&\left. + \left( -\frac{\sigma ^2}{\delta ^2}B_i+W_\Phi (B_i)\right) \frac{\partial ^2 B_i}{\partial \delta ^2}\right] , \end{aligned}$$

where $W_\Phi '(x)=-W_\Phi (x)(x+W_\Phi (x))$. The first and second derivatives of $B_i$, $i=1,\ldots ,n$, in relation to $\varvec{\theta }$ are given by

$$\begin{aligned} \frac{\partial B_i}{\partial \varvec{\beta }}= & {} -\frac{\delta }{\sigma (\sigma ^2+\delta ^2)^{1/2}}(\mathbf {x}_i-\rho \mathbf {x}_{i-1}),\\ \frac{\partial B_i}{\partial \varvec{\gamma }}= & {} -\frac{\delta }{\sigma (\sigma ^2+\delta ^2)^{1/2}}(\mathbf {n}_i-\rho \mathbf {n}_{i-1}),\\ \frac{\partial B_i}{\partial \rho }= & {} -\frac{\delta }{\sigma (\sigma ^2+\delta ^2)^{1/2}}(y_{i-1}-\mu _{i-1}),\\ \frac{\partial B_i}{\partial \sigma ^2}= & {} -\frac{\delta (2\sigma ^2+\delta ^2)}{2\sigma ^3(\sigma ^2+\delta ^2)^{3/2}}(y_i-\xi _i+b\delta ),\\ \frac{\partial B_i}{\partial \delta }= & {} \frac{\sigma ^2(y_i-\xi _i+b\delta )+ b\delta (\sigma ^2+\delta ^2)}{\sigma (\sigma ^2+\delta ^2)^{3/2}}\\ \frac{\partial ^2 B_i}{\partial \varvec{\beta }\partial \beta ^\top }= & {} \frac{\partial ^2 B_i}{\partial \varvec{\gamma }\partial \beta ^\top }=\frac{\partial ^2 B_i}{\partial \varvec{\gamma }\partial \varvec{\gamma }^\top }=\mathbf {0},\\ \frac{\partial ^2 B_i}{\partial \rho ^2}= & {} 0,\\ \frac{\partial ^2 B_i}{\partial \rho \partial \varvec{\beta }}= & {} \frac{\delta }{\sigma (\sigma ^2+\delta ^2)^{1/2}}\mathbf {x}_{i-1},\\ \frac{\partial ^2 B_i}{\partial \rho \partial \varvec{\gamma }}= & {} \frac{\delta }{\sigma (\sigma ^2+\delta ^2)^{1/2}}\mathbf {n}_{i-1}\\ \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \varvec{\beta }}= & {} \frac{\delta (2\sigma ^2+\delta ^2)}{2\sigma ^3(\sigma ^2+\delta ^2)^{3/2}}(\mathbf {x}_i-\rho \mathbf {x}_{i-1}),\\ \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \varvec{\gamma }}= & {} \frac{\delta (2\sigma ^2+\delta ^2)}{2\sigma ^3(\sigma ^2+\delta ^2)^{3/2}}(\mathbf {n}_i-\rho \mathbf {n}_{i-1}),\\ \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \rho }= & {} \frac{\delta (2\sigma ^2+\delta ^2)}{2\sigma ^3(\sigma ^2+\delta ^2)^{3/2}}(y_{i-1}-\mu _{i-1}),\\ \frac{\partial ^2 B_i}{\partial \delta \partial \varvec{\beta }}= & {} -\frac{\sigma }{(\sigma ^2+\delta ^2)^{3/2}}(\mathbf {x}_i-\rho \mathbf {x}_{i-1}),\\ \frac{\partial ^2 B_i}{\partial \delta \partial \varvec{\gamma }}= & {} -\frac{\sigma }{(\sigma ^2+\delta ^2)^{3/2}}(\mathbf {n}_i-\rho \mathbf {n}_{i-1}),\\ \frac{\partial ^2 B_i}{\partial \delta \partial \rho }= & {} -\frac{\sigma }{(\sigma ^2+\delta ^2)^{3/2}}(y_{i-1}-\mu _{i-1}),\\ \frac{\partial ^2 B_i}{\partial \sigma ^4}= & {} -\frac{\delta (y_i-\xi _i+b\delta )}{4\sigma ^5(\sigma ^2+\delta ^2)^{5/2}}[4\sigma ^2(\sigma ^2+\delta ^2)-3(2\sigma ^2+\delta ^2)^2],\\ \frac{\partial ^2 B_i}{\partial \sigma ^2\partial \delta }= & {} \frac{\sigma (\sigma ^2+\delta ^2)(y_i-\xi _i+2b\delta )-\frac{1}{2}\left[ \sigma ^2(y_i-\xi _i+b\delta )+b\delta (\sigma ^2+\delta ^2)\right] \left( \frac{\sigma ^2+\delta ^2}{\sigma }+3\sigma \right) }{\sigma ^2(\sigma ^2+\delta ^2)^{5/2}},\\ \frac{\partial ^2 B_i}{\partial \delta ^2}= & {} \frac{b(\sigma ^2+\delta ^2)(2\sigma ^2+3\delta ^2)-3\delta \left[ \sigma ^2(y_i-\xi _i+b\delta )+b\delta (\sigma ^2+\delta ^2)\right] }{\sigma (\sigma ^2+\delta ^2)^{5/2}}, \end{aligned}$$

where $\mathbf {x}_0=\mathbf {n}_0=\mathbf {0}$ and $y_0=\mu _0=0$.

Appendix B: Derivation of the matrices $\mathbf {N}$ and $\mathbf {K}$

Let $\mathbf {t}=(t_1,\ldots , t_m)$ and ndx the number of equidistant Knots desired by the user. It follows the commands in R (R Core Team 2019) for construction of the matrices $\mathbf {N}$ and $\mathbf {K}$: