Optimal designs for K-factor two-level models with first-order interactions on a symmetrically restricted design region

Abstract

We develop D-optimal designs for linear models with first-order interactions on a subset of the $2^K$ full factorial design region, when both the number of factors set to the higher level and the number of factors set to the lower level are simultaneously bounded by the same threshold. It turns out that in the case of narrow margins the optimal design is concentrated only on those design points, for which either the threshold is attained or the numbers of high and low levels are as equal as possible. In the case of wider margins the settings are more spread and the resulting optimal designs are as efficient as a full factorial design. These findings also apply to other optimality criteria.

Appendices

Appendix A: Proofs

For the proof of Lemma 1 we need some auxiliary results on the eigenvalues of ${\mathbf {M}}_{22}-m_2^2{\mathbf {J}}_{C(K,2)}$ for a symmetric invariant design when $K\ge 4$.

Lemma 3

Let

$$\begin{aligned} \lambda _{\varvec{1}}= 1+2(K-2)m_{2} +\textstyle {\frac{1}{2}}(K-2)(K-3)m_{4} - \textstyle {\frac{1}{2}}K(K-1)m_{2}^2 \end{aligned}$$

be the eigenvalue of ${\mathbf {M}}_{22}-m_2^2{\mathbf {J}}_{C(K,2)}$ associated with the eigenvector $\varvec{1}_{C(K,2)}$. Then $\lambda _{\varvec{1}}>0$ if and only if the support of ${\bar{\xi }}$ includes at least two distinct symmetric orbits.

Proof

First note that for each invariant design ${\bar{\xi }}_k$ on a single orbit ${\mathcal {O}}_k$ we get by inserting the moments that the corresponding eigenvalue $\lambda _{\varvec{1}}({\bar{\xi }}_k)$ is zero. Let ${\tilde{\xi }}_{k}$ be the corresponding symmetric invariant design on the symmetric orbit $\tilde{{\mathcal {O}}}_{k}$. Then $m_{2}({\tilde{\xi }}_k) = m_{2}({\bar{\xi }}_{k})$, $m_{4}({\tilde{\xi }}_k) = m_{4}({\bar{\xi }}_{k})$ and, hence, $\lambda _{\varvec{1}}({\tilde{\xi }}_k)=\lambda _{\varvec{1}}({\bar{\xi }}_k)=0$. Consequently at least two distinct symmetric orbits are needed for $\lambda _{\varvec{1}}>0$.

Now, let ${\bar{\xi }}={\bar{w}}_k{\tilde{\xi }}_k+{\bar{w}}_{\ell }{\tilde{\xi }}_{\ell }$ be a symmetric invariant design on the symmetric orbits $\tilde{{\mathcal {O}}}_{k}$ and $\tilde{{\mathcal {O}}}_{\ell }$, $k<\ell \le K/2$, ${\bar{w}}_k,{\bar{w}}_{\ell }>0$. As $m_2({\bar{\xi }}_k)$ is strictly decreasing in k we have $m_2({\bar{\xi }}_k)\ne m_2({\bar{\xi }}_{\ell })$. Thus $m_2({\bar{\xi }})^2<{\bar{w}}_k m_2({\bar{\xi }}_k)^2 + {\bar{w}}_{\ell } m_2({\bar{\xi }}_{\ell })^2$ by the strict concavity of the quadratic function, which implies $\lambda _{\varvec{1}}>0$. $\square$

Lemma 4

Let $\lambda _{{\mathbf {S}}}=1+(K-4)m_{2} - (K-3) m_{4}$ be the eigenvalue of ${\mathbf {M}}_{22}-m_2^2{\mathbf {J}}_{C(K,2)}$ associated with the remaining eigenvectors of ${\mathbf {S}}_{K}{\mathbf {S}}_{K}^{\top }$ orthogonal to $\varvec{1}_{C(K,2)}$. Then $\lambda _{{\mathbf {S}}}>0$ if and only if the support of ${\bar{\xi }}$ includes at least one symmetric orbit $\tilde{{\mathcal {O}}}_{k}$ for which $0<k<K/2$.

Proof

By inserting the moments we get for this eigenvalue

$$\begin{aligned} \lambda _{{\mathbf {S}}}({\bar{\xi }}_{k}) = \frac{(2k-K)^2(K^2 - (2k-K)^2)}{K(K-1)(K-2)}\,, \end{aligned}$$

which is equal to 0 for $k=0$ or $k=K/2$. Moreover, $\lambda _{{\mathbf {S}}}({\bar{\xi }}_k)$ is a polynomial in k of degree four, symmetric around K / 2, and with negative leading term. Hence, there cannot be any other root, and $\lambda _{{\mathbf {S}}}({\bar{\xi }}_k)>0$ for all $0<k<K/2$. $\square$

Lemma 5

Let $\lambda _{{\mathbf {I}}} = 1-2m_{2}+m_{4}$ be the eigenvalue of ${\mathbf {M}}_{22}-m_2^2{\mathbf {J}}_{C(K,2)}$ associated with the remaining eigenvectors orthogonal to those of ${\mathbf {S}}_{K}{\mathbf {S}}_{K}^{\top }$. Then $\lambda _{{\mathbf {I}}}>0$ if and only if the support of ${\bar{\xi }}$ includes at least one symmetric orbit $\tilde{{\mathcal {O}}}_{k}$ for which $k>1$.

Proof

By inserting the moments we see that $\lambda _{{\mathbf {I}}}({\bar{\xi }}_k)$ is a polynomial in k of degree four, symmetric around K / 2, and with positive leading term, which is equal to zero for $k=0$ and $k=1$. Hence, there cannot be any other root, and $\lambda _{{\mathbf {I}}}({\bar{\xi }}_k)>0$ for all $1<k\le K/2$. $\square$

Proof (Proof of Lemma 1)

We note that according to Freise et al. (2018) the matrix ${\mathbf {M}}_{11}$ associated with the main effects is nonsingular when at least two distinct symmetric orbits are involved in the design ${\bar{\xi }}$. Hence, the requirement of ${\mathbf {M}}_{11}$ to be nonsingular does not impose any additional condition besides those of Lemmas 3–5. Hence, the assertion follows from these lemmas. $\square$

Proof (Proof of Theorem 1)

For $K=2$ and $K=3$ we have $B_K<1$, such that the full factorial design can serve as the asserted symmetric invariant design.

Let $K\ge 4$. If $L=B_{K}$, then the design of Lemma 2 can be used. Let $L<B_{K}$ and let K be even. Then choose $\ell$ such that

$$\begin{aligned} B_{K} = \frac{K-\sqrt{3K-2}}{2} \le \ell \le \frac{K-\sqrt{K}}{2}\,. \end{aligned}$$

Such $\ell$ always exists (for $K\le 8$ see Table 1; note that $\sqrt{3K-2}-\sqrt{K}\ge 2$ for $K\ge 10$).

Let designs ${\bar{\xi }}_{(L)}$ and ${\bar{\xi }}_{(\ell )}$ be defined as

$$\begin{aligned} {\bar{\xi }}_{(L)}={\bar{w}}_L{\bar{\xi }}_L+(1-2{\bar{w}}_L){\bar{\xi }}_{K/2} + {\bar{w}}_L{\bar{\xi }}_{K-L} \quad \text {with}\quad {\bar{w}}_{L} = \frac{K}{2(2L-K)^2} \end{aligned}$$

(9)

and

$$\begin{aligned} {\bar{\xi }}_{(\ell )}={\bar{w}}_{\ell }{\bar{\xi }}_{\ell }+(1-2{\bar{w}}_{\ell }){\bar{\xi }}_{K/2} + {\bar{w}}_{\ell }{\bar{\xi }}_{K-\ell } \quad \text {with}\quad {\bar{w}}_{\ell } = \frac{K}{2(2\ell -K)^2}\,. \end{aligned}$$

(10)

According to Freise et al. (2018) the moments $m_{2}({\bar{\xi }}_{(L)})$ and $m_{2}({\bar{\xi }}_{(\ell )})$ are equal to 0, because $L,\ell <(K-\sqrt{K})/2$. Next we consider the convex combination

$$\begin{aligned} {\bar{\xi }}^{*} = \alpha {\bar{\xi }}_{(L)} + (1-\alpha ){\bar{\xi }}_{(\ell )} \quad \text {with}\quad \alpha = \frac{3K-2-(2\ell -K)^2}{4(\ell -L)(K - L - \ell )}\,. \end{aligned}$$

Also for the symmetric invariant design ${\bar{\xi }}^*$ we have $m_{2}({\bar{\xi }}^*) = 0$. Further we obtain

$$\begin{aligned} m_{4}({\bar{\xi }}^*) = \frac{4 \alpha (\ell -L)(K - L - \ell )+ (2\ell -K)^2 - (3K-2)}{(K-1)(K-2)(K-3)} = 0\,, \end{aligned}$$

which establishes the result for even K.

For K odd the proof is similar. For $L<B_{K}$ choose $\ell$ as above with the corresponding value for $B_{K}$ and the designs

$$\begin{aligned} {\bar{\xi }}_{(L)}={\bar{w}}_L{\bar{\xi }}_L+\big (\textstyle {\frac{1}{2}}-{\bar{w}}_L\big ){\bar{\xi }}_{(K-1)/2} +\big (\textstyle {\frac{1}{2}}-{\bar{w}}_L\big ){\bar{\xi }}_{(K+1)/2}+ {\bar{w}}_L{\bar{\xi }}_{K-L} \end{aligned}$$

and

$$\begin{aligned} {\bar{\xi }}_{(\ell )}={\bar{w}}_{\ell }{\bar{\xi }}_{\ell }+\big (\textstyle {\frac{1}{2}}-{\bar{w}}_{\ell }\big ){\bar{\xi }}_{(K-1)/2} +\big (\textstyle {\frac{1}{2}}-{\bar{w}}_{\ell }\big ){\bar{\xi }}_{(K+1)/2}+ {\bar{w}}_{\ell }{\bar{\xi }}_{K-\ell } \end{aligned}$$

with corresponding weights

$$\begin{aligned} {\bar{w}}_{L} =\frac{K-1}{2((2L-K)^2-1)} \quad \text {and}\quad {\bar{w}}_{\ell } =\frac{K-1}{2((2\ell -K)^2-1)}\,. \end{aligned}$$

Again such $\ell$ always exists (for $K\le 7$ see Table 1; for $K \ge 9$ note that $\sqrt{3K}-\sqrt{K}\ge 2$) and $m_{2}({\bar{\xi }}_{(L)})=m_{2}({\bar{\xi }}_{(\ell )})=0$.

Then the convex combination

$$\begin{aligned} {\bar{\xi }}^{*} = \alpha {\bar{\xi }}_{(L)} + (1-\alpha ){\bar{\xi }}_{(\ell )} \quad \text {with}\quad \alpha = \frac{3K-(2\ell -K)^2}{4(\ell -L)(K-L-\ell )} \end{aligned}$$

yields $m_{4}({\bar{\xi }}^*)=0$, and the result follows. $\square$

For the proof of Theorem 2 we will make use of the celebrated Kiefer-Wolfowitz equivalence theorem (Kiefer and Wolfowitz 1960). Therefore we first investigate the sensitivity function (functional derivative).

Lemma 6

Let ${\bar{\xi }}$ be a symmetric invariant design. Then the sensitivity function $\psi (\varvec{x})=\varvec{f}(\varvec{x})^{\top }{\mathbf {M}}({\bar{\xi }})^{-1}\varvec{f}(\varvec{x})$ is constant on the orbits, $\psi (\varvec{x})={\tilde{\psi }}(k)$ for $\varvec{x}\in {\mathcal {O}}_k$, say, and the function ${\tilde{\psi }}$ is a polynomial of degree at most 4, which is symmetric with respect to K / 2 $({\tilde{\psi }}(K-k)={\tilde{\psi }}(k))$.

Proof

Using the inverse of the information matrix in Eq. (8) we obtain for the sensitivity function

$$\begin{aligned} \psi (\varvec{x}) = c_{0} -2c_{2}\varvec{{\tilde{x}}}^\top \varvec{1}_{C(K,2)} +\varvec{x}^\top {\mathbf {M}}_{11}^{-1}\varvec{x} +\varvec{{\tilde{x}}}^\top {\mathbf {C}}_{22}\varvec{{\tilde{x}}}\,. \end{aligned}$$

(11)

Note that $\varvec{x}^\top \varvec{x} = K$ and $\varvec{{\tilde{x}}}^\top \varvec{{\tilde{x}}} = K(K-1)/2$. Further, for $\varvec{x} \in {\mathcal {O}}_{k}$, we get

$$\begin{aligned} \varvec{x}^\top \varvec{1}_{K} = 2 k - K\,, \quad \varvec{{\tilde{x}}}^\top \varvec{1}_{C(K,2)} = ((2k-K)^2 - K)/2 \end{aligned}$$

and

$$\begin{aligned} \varvec{{\tilde{x}}}^\top {\mathbf {S}}_{K}{\mathbf {S}}_{K}^\top \varvec{{\tilde{x}}} = (K - 2)(2 k-K)^{2} + K\,. \end{aligned}$$

This yields

$$\begin{aligned} \psi (\varvec{x}) = a_{4}(2k-K)^4+ a_{2}(2k-K)^{2}+a_{0} \end{aligned}$$

(12)

with coefficients

$$\begin{aligned} a_{0}&=c_{0} + \frac{K}{1-m_{2}} +\frac{K(K-1)}{2(1-2m_{2}+m_{4})} -\frac{\delta _{{\mathbf {S}}}K}{1-2m_{2}+m_{4}} -\frac{\delta _{{\mathbf {J}}}K^2}{4(1-2m_{2}+m_{4})}\\ a_{2}&=-\left( c_{2} + \frac{m_{2}}{(1-m_{2})(1+(K-1)m_{2})} +\frac{\delta _{{\mathbf {S}}}(K-2)}{1-2m_{2}+m_{4}} -\frac{\delta _{{\mathbf {J}}}K}{2(1-2m_{2}+m_{4})}\right) \\ a_{4}&=-\frac{\delta _{{\mathbf {J}}}}{4(1-2m_{2}+m_{4})}\,. \end{aligned}$$

Hence, ${{\tilde{\psi }}}$ is a polynomial of degree four in k. The symmetry around K / 2 follows, since only even powers of k occur. $\square$

Lemma 7

Let $B_{K}<L<K/2$ and ${\bar{\xi }}^*$ an optimal symmetric invariant design on ${\mathcal {X}}_{L,K-L}$. Then the orbitwise sensitivity function ${\tilde{\psi }}$ has a positive leading term for the fourth order monomial $k^4$.

Proof

We will use the same notation as in Lemma 6 and its proof.

Let the coefficient of the fourth order monomial in ${\tilde{\psi }}$ be nonpositive or equivalently let $a_{4}\le 0$. Then the function ${\tilde{\psi }}$ as a function in k has either a single maximum at $k=K/2$, two (symmetric) maxima outside $(L,K-L)$ (respectively at $k=L$ and $k=K-L$ for the admitted orbits), two (symmetric) maxima inside $(L,K-L)$, or is constant.

In the first two cases ${\bar{\xi }}^*$ is supported on one symmetric orbit only. It follows from Lemma 1, that the information matrix has to be singular. But in this case the function ${\tilde{\psi }}$ would not be defined, which is a contradiction.

In the last case, if the orbitwise sensitivity is constant, we have ${\tilde{\psi }}(k)\le p$ for all $k\in \{0,\ldots ,K\}$ and, consequently, ${\bar{\xi }}^*$ is optimal on the unrestricted design region ${\mathcal {X}}_{0,K}$.

In the third case, i.e. two maxima inside of $(L,K-L)$, the optimal invariant design ${\bar{\xi }}^*$ has all its weight on either one or two symmetric orbits. If these orbits do not satisfy the conditions in Lemma 1, the information matrix would be singular, which leads to a contradiction. Otherwise the design is optimal on ${\mathcal {X}}_{0,K}$, with the same argument as for the constant case.

Now let ${\bar{\xi }}^*$ be optimal on ${\mathcal {X}}_{0,K}$. Then its information matrix is ${\mathbf {I}}_{p}$. It follows that $m_{2}=m_{4} = 0$ and thus

$$\begin{aligned} \sum _{k=L}^{K-L}{\bar{w}}_{k}(2 k - K)^2 = K \quad \text {and}\quad \sum _{k=L}^{K-L}{\bar{w}}_{k}(2k-K)^4 - K^2 = 2K(K-1)\,. \end{aligned}$$

(13)

The left-hand sides of these two equations can be interpreted as expectation and variance, respectively, of a discrete random variable taking values $(2k-K)^2$, $k=L,\ldots ,K-L$. An upper bound for the variance is given in Muilwijk (1966) (see also Bhatia and Davis 2000). This yields for the variance

$$\begin{aligned} \sum _{k=L}^{K-L}{\bar{w}}_{k}(2k-K)^4 - K^2 \le ((2L-K)^2 - K)(K - R_{K})\,, \end{aligned}$$

where $R_{K}=0$ for K even and $R_{K} = 1$ for K odd. Since $B_{K}<L$ it follows that

$$\begin{aligned} ((2L-K)^2 - K)(K - R_{K}) < ((2B_{K}-K)^2 - K)(K - R_{K}) = 2K(K-1)\,, \end{aligned}$$

which is in contradiction to (13). Hence $a_{4}$ and consequently the leading coefficient has to be positive. $\square$

Proof (Proof of Theorem 2)

Under the assumptions of Theorem 2, the orbitwise sensitivity function ${\tilde{\psi }}$ of the optimal design ${\bar{\xi }}^*$ is a polynomial of degree four with positive leading term by Lemma 6 and 7. Then, in view of the fundamental theorem of algebra, the equality ${{\tilde{\psi }}}(k)=p$ can only have at most four distinct roots. Because of the symmetry of the sensitivity function with respect to K / 2 (cf. Lemma 6) the optimal design has thus to be concentrated on at most two symmetric orbits. In order to fulfill the condition ${{\tilde{\psi }}}(k)\le p$ for all $k=L,\ldots ,K-L$, imposed by the equivalence theorem on the optimal design ${\bar{\xi }}^*$, these symmetric orbits can only be the outmost orbit ${\mathcal {O}}_{L}\cup {\mathcal {O}}_{K-L}$ on the boundaries and the central orbit ${\mathcal {O}}_{K/2}$ for K even and ${\mathcal {O}}_{(K-1)/2}\cup {\mathcal {O}}_{(K+1)/2}$ for K odd, respectively.

On the other hand the nonsingularity condition of Lemma 1 requires that the optimal design ${\bar{\xi }}^*$ has to be supported by at least two symmetric orbits and, hence, ${\bar{\xi }}^*$ is of the form specified in the Theorem. Finally only the weights have to be optimized given the two symmetric orbits. $\square$

Appendix B: Tables

Table 1 Symmetric invariant D-optimal designs for wide bounds from Theorem 1

Table 2 Symmetric invariant D-optimal designs for narrow bounds from Theorem 2