PC priors for residual correlation parameters in one-factor mixed models

Abstract

Lack of independence in the residuals from linear regression motivates the use of random effect models in many applied fields. We start from the one-way anova model and extend it to a general class of one-factor Bayesian mixed models, discussing several correlation structures for the within group residuals. All the considered group models are parametrized in terms of a single correlation (hyper-)parameter, controlling the shrinkage towards the case of independent residuals (iid). We derive a penalized complexity (PC) prior for the correlation parameter of a generic group model. This prior has desirable properties from a practical point of view: (i) it ensures appropriate shrinkage to the iid case; (ii) it depends on a scaling parameter whose choice only requires a prior guess on the proportion of total variance explained by the grouping factor; (iii) it is defined on a distance scale common to all group models, thus the scaling parameter can be chosen in the same manner regardless the adopted group model. We show the benefit of using these PC priors in a case study in community ecology where different group models are compared.

Proofs of results in Sect. 4.2

Recall the definition of PC prior as an exponential distribution on the distance $d(\rho )$, with rate parameter $\lambda$,

$$\begin{aligned} \pi (d(\rho )) = \lambda \exp (-\,\lambda d(\rho )) \quad \lambda >0. \end{aligned}$$

If design is balanced then $m_j=m,\forall j=1,\ldots ,n$; recall that n is the number of groups while m is the number of within group observations. In this case, the distance function in Eq. (5) simplifies to

$$\begin{aligned} d(\rho ) = \sqrt{- n \log \left( |\varvec{R}(\rho )|\right) } \quad 0 \le \rho < 1. \end{aligned}$$

Fixing $\lambda =\lambda '/ \sqrt{n}$, the PC prior for $\rho$ results (by the change of variable rule)

$$\begin{aligned} \pi (\rho )= & {} \lambda \exp \left( -\,\lambda d(\rho )\right) \left| \frac{\partial d(\rho )}{\partial \rho }\right| \nonumber \\= & {} \lambda \exp \left( -\,\lambda d(\rho )\right) \left| -\frac{n}{2\sqrt{-n \log (|\varvec{R}(\rho )|)}} |\varvec{R}(\rho )|^{-1} \frac{\partial |\varvec{R}(\rho )|}{\partial \rho } \right| \nonumber \\= & {} \frac{1}{2}|\varvec{R}(\rho )|^{-1}\left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right| \frac{\lambda '}{ \sqrt{-\log \left( |\varvec{R}(\rho )|\right) }} \exp \left( -\,\lambda ' \sqrt{-\log \left( |\varvec{R}(\rho )|\right) }\right) . \nonumber \\ \end{aligned}$$

(14)

Below, the PC priors in Eqs. (8)–(10) are derived. In each case, the proof is completed by deriving the analytical expression for the term $\frac{1}{2}|\varvec{R}(\rho )|^{-1}\left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right|$ and plugging it in (14).

1.1 Exchangeable

Proof of Eq. (8)

Let us consider the compound symmetric matrix $\varvec{R}(\rho )$ as in (3), where subscript j is removed as we are working under a balanced design. Riebler et al. (2012) showed that

$$\begin{aligned} |\varvec{R}(\rho )| = (1+(m-1)\rho ) (1-\rho )^{m-1} \quad 0 \le \rho < 1, \end{aligned}$$

hence the distance function is equal to $d(\rho ) = \sqrt{-n \log \left\{ (1+(m-1)\rho ) (1-\rho )^{m-1}\right\} }$. The derivative term in (14) is

$$\begin{aligned} \left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right|= & {} \left| \underbrace{(m-1)(1-\rho )^{m-2}}_{>0} \left\{ \underbrace{(1-\rho ) - (1+(m-1)\rho )}_{<0} \right\} \right| \\= & {} (m-1)(1-\rho )^{m-2}\left\{ (1+(m-1)\rho ) - (1-\rho )\right\} . \end{aligned}$$

After some algebraic steps, we obtain

$$\begin{aligned} \frac{1}{2}|\varvec{R}(\rho )|^{-1}\left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right| = \frac{m-1}{2}\left( \frac{1}{1-\rho } - \frac{1}{1+(m-1)\rho }\right) , \end{aligned}$$

which completes the proof. $\square$

1.2 Autoregressive of order one

Proof of Eq. (9)

The PC prior for the lag-one correlation of an AR1 is derived by Sørbye and Rue (2017). Here we extend it to group models having within group correlation matrix $\varvec{R}(\rho )$ as in (4). It can be shown that

$$\begin{aligned} \varvec{R}(\rho )^{-1}&= \frac{1}{1-\rho ^2}\varvec{P} ; \quad \\ \varvec{P}&= \begin{bmatrix} 1&\quad -\,{\rho }&\quad 0&\quad \cdots&\quad \cdots&\quad \cdots&\quad 0 \\ -{\rho }&\quad 1+\rho ^2&\quad -\,{\rho }&\quad \ddots&\quad&\quad&\quad \vdots \\ 0&\quad -\, \rho&\quad 1+\rho ^2&\quad -\,\rho&\quad \ddots&\quad&\quad \vdots \\ \vdots&\quad \ddots&\quad \ddots&\quad \ddots&\quad \ddots&\quad \ddots&\quad \vdots \\ \vdots&\quad&\quad \ddots&\quad -\,\rho&\quad 1+\rho ^2&\quad -\,\rho&\quad 0\\ \vdots&\quad&\quad&\quad \ddots&\quad -\,\rho&\quad 1+\rho ^2&\quad -\,\rho \\ 0&\quad \cdots&\quad \cdots&\quad \cdots&\quad 0&\quad -\,{\rho }&\quad 1 \end{bmatrix}, \end{aligned}$$

where $|\varvec{P}| = 1-\rho ^2$. Thus the determinant of the AR1 correlation matrix is

$$\begin{aligned} |\varvec{R}(\rho )| = \frac{1}{|\varvec{R}(\rho )^{-1}|} = (1-\rho ^2)^{m-1} \quad 0 \le \rho < 1, \end{aligned}$$

hence the distance function is equal to $d(\rho ) = \sqrt{-n (m-1) \log (1-\rho ^2)}$. The derivative term in (14) is

$$\begin{aligned} \left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right|= & {} 2 \rho (m-1)(1-\rho ^2)^{m-2}. \end{aligned}$$

After some algebraic steps, we obtain

$$\begin{aligned} \frac{1}{2}|\varvec{R}(\rho )|^{-1}\left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right| =\frac{\rho (m-1)}{1-\rho ^2}, \end{aligned}$$

which completes the proof. $\square$

1.3 Ornstein Uhlenbeck

Proof of Eq. (10)

This proof follows straightforwardly from the AR1 case, by recognizing that $\phi = -\,\log (\rho )$, hence $\rho = \exp (-\,\phi )$. In this case, the determinant is

$$\begin{aligned} |\varvec{R}(\phi )| = (1-\exp (-\,2\phi ))^{m-1} \quad \phi >0, \end{aligned}$$

and the distance function is equal to $d(\phi ) = \sqrt{-n (m-1) \log (1-\exp (-\,2\phi ))}$. The derivative term in (14) is

$$\begin{aligned} \left| \frac{\partial |\varvec{R}(\phi )|}{\partial \phi }\right|= & {} 2(m-1)(1-\exp (-\,2\phi ))^{m-2} \exp (-\,2\phi ). \end{aligned}$$

After some algebraic steps, we obtain

$$\begin{aligned} \frac{1}{2}|\varvec{R}(\rho )|^{-1}\left| \frac{\partial |\varvec{R}(\rho )|}{\partial \rho }\right| = \frac{(m-1)\exp (-\,2\phi )}{1-\exp (-\,2\phi )}, \end{aligned}$$

which completes the proof. $\square$