Screening for chronic diseases: optimizing lead time through balancing prescribed frequency and individual adherence

Abstract

Screening for chronic diseases, such as cancer, is an important public health priority, but traditionally only the frequency or rate of screening has received attention. In this work, we study the importance of adhering to recommended screening policies and develop new methodology to better optimize screening policies when adherence is imperfect. We consider a progressive disease model with four states (healthy, undetectable preclinical, detectable preclinical, clinical), and overlay this with a stochastic screening–behavior model using the theory of renewal processes that allows us to capture imperfect adherence to screening programs in a transparent way. We show that decreased adherence leads to reduced efficacy of screening programs, quantified here using elements of the lead time distribution (i.e., the time between screening diagnosis and when diagnosis would have occurred clinically in the absence of screening). Under the assumption of an inverse relationship between prescribed screening frequency and individual adherence, we show that the optimal screening frequency generally decreases with increasing levels of non-adherence. We apply this model to an example in breast cancer screening, demonstrating how accounting for imperfect adherence affects the recommended screening frequency.

Appendix

1.1 Proof of Theorem 1

Recalling that $R = R(X+W),$ we can write

$$\begin{aligned} P( R \ge V, D = 1)= & {} P( R \ge V, t_0 < X+W+V \le t_{max}) \nonumber \\= & {} \int _0^{t_{max}} \!\! Q_1(v,t_0,t_{max}) g_V(v) dv. \end{aligned}$$

(21)

where $Q_1(v,t_0,t_{max}) = P( R(X+W) \ge v, t_0-v < X+W \le t_{max}-v \mid V = v).$

Focusing on the probability statement appearing inside the integrand:

$$\begin{aligned} Q_1(v,t_0,t_{max}) =&P( R(X+W) \ge v, X+W \le t_0, t_0-v< X+W, X+W \le t_{max}-v \mid V = v) \\&+ P( R(X+W) \ge v, X+W> t_0, t_0-v< X+W, X+W \le t_{max}-v \mid V = v). \\ =&P( R(X+W) \ge v, t_0-v < X+W, X+W \le \min \{ t_0,t_{max}-v \} \mid V = v) \\&+ P( R(X+W) \ge v, X+W > t_0, X+W \le t_{max}-v \mid V = v) \\ =&(I) + (II). \end{aligned}$$

The terms (I) and (II) appearing in the last expression are easier to simplify when handled separately. First, observe

$$\begin{aligned} (I) \,=\, P( R(X+W) \ge v,&\, t_0-v < X+W, X+W \le \min \{ t_0,t_{max}-v \} \mid V = v) \\&= \int _{t_0-v}^{\min \{ t_0,t_{max}-v \}} P( R(s) \ge v \mid X+W = s, V = v) g(s\mid v) ds \\&= \int _{t_0-v}^{\min \{ t_0,t_{max}-v \}} P( {\tilde{R}} \ge s - (t_0 - v) ) g(s\mid v) ds \end{aligned}$$

the last expression resulting from (1) since $s \le t_0$ and the fact that ${\tilde{R}} \bot (X,W,V).$ It now follows that

$$\begin{aligned} (I) = \int _{\max \{0,t_0-v\}}^{\max \{0,\min \{ t_0,t_{max}-v \}\}} P( {\tilde{R}} \ge s - (t_0 - v) ) \, g(s \mid v) \, ds, \end{aligned}$$

(22)

as $g(s \mid v) = 0$ for $s < 0$ and any $v > 0.$ Proceeding similarly, now observe

$$\begin{aligned} (II)= & {} P( R(X+W) \ge v, X+W> t_0, X+W \le t_{max}-v ) \nonumber \\= & {} I\{t_{max}-v > t_0\} \int _{t_0}^{t_{max}-v} P( R(s) \ge v \mid X+W = s, V = v) g(s\mid v) ds \nonumber \\= & {} I\{v \le t_{max}- t_0\} \int _{t_0}^{t_{max}-v} P( {\tilde{R}} \ge v ) g(s\mid v) ds \nonumber \\= & {} I\{v \le t_{max}- t_0\} \, P( {\tilde{R}} \ge v ) \cdot [ G(t_{max}-v \mid v) - G(t_{0} \mid v)]. \end{aligned}$$

(23)

Substituting (22) and (23) into (21) and simplifying respectively leads to expressions (B) and (A) in the statement of the theorem. $\square$

1.2 Proof of Theorem 2

Let

$$\begin{aligned} Q_2(v,t_0,t_{max}) = E\left[ I\{ t_0-v< X+W \le t_{max}-v, R < v \} (v-R) \mid V = v\right] \end{aligned}$$

and recall that

$$\begin{aligned} E[K(V - R)]= & {} \int _0^{t_{max}} \!\! Q_2(v,t_0,t_{max}) g_V(v) \, dv.~~ \end{aligned}$$

(24)

We begin with the expectation term appearing inside the integral:

$$\begin{aligned} Q_2(v,t_0,t_{max})&= \int _{t_0-v}^{t_{max}-v} E[ (v-R(s)) I\{R(s)< v\} | X+W = s, V = v] g(s \mid v) ds \nonumber \\&= \int _{t_0-v}^{t_{max}-v} E[ (v-R(s)) I\{R(s) < v\} ] g(s \mid v) ds, \end{aligned}$$

(25)

the last step following from (1) and the fact that ${\tilde{R}} \bot (X,W,V).$ For any positive random variable U with cumulative distribution function $J(a) = P( U \le a), a \ge 0,$ integration-by-parts gives for $v > 0$ that $E[ (v-U) I\{ U < v\}] = \int _0^v J(a) da.$ Consequently, we may write

$$\begin{aligned} E[ (v-R(s)) I\{R(s) < v\} ] = \int _0^v P( R(s) \le a ) da. \end{aligned}$$

Using (1) and defining ${\tilde{H}}(u) = \int _0^u P( {\tilde{R}} \le a ) da,$ it can be shown that

$$\begin{aligned} \int _0^v P( R(s) \le a ) da= & {} I\{s > t_0\} {\tilde{H}}(v) + I\{t_0-v< s < t_0\} {\tilde{H}}(v-(t_0-s)).\nonumber \\ \end{aligned}$$

(26)

Substituting (26) into (25) and simplifying, we obtain

$$\begin{aligned} \int _{t_0-v}^{t_{max}-v} E[ (v- R(s))&I\{R(s)< v\} ] g(s \mid v) ds \nonumber \\ = I\{t_{max}-v&\, \ge t_0\} \int _{t_0}^{t_{max}-v} {\tilde{H}}(v) g(s \mid v) ds \nonumber \\ +&\int _{\max \{0,t_0-v\}}^{\max \{0,t_{max}-v\}} I\{t_0-v< s < t_0\} {\tilde{H}}(v-(t_0-s)) g(s \mid v) ds. \nonumber \\ = I\{t_{max}-v&\, \ge t_0\} {\tilde{H}}(v) \cdot [ G(t_{max}-v \mid v) - G(t_{0} \mid v) ] \end{aligned}$$

(27)

$$\begin{aligned} +&\int _{\max \{0,t_0-v\}}^{\min \{t_0,\max \{0,t_{max}-v\}\}} {\tilde{H}}(s-(t_0-v)) g(s \mid v) ds. \end{aligned}$$

(28)

Substituting (27) and (28) into (24) now leads to (C) and (D) as given in the statement of the Theorem. $\square$

1.3 Proof of Theorem 3

Consider the class $\mathcal {F}_{\mu ,\sigma ^2}$ of distributions on ${\mathbb {R}}^+$ with mean $\mu$ and variance $\sigma ^2$. Our proof follows the strategy of Roos et al. (2020).

Fig. 5

Quadratic majorizing function for the objective function in (29)

The primal problem is given by (14) with constraints (15). The corresponding dual problem is

$$\begin{aligned} \inf _{\lambda _0,\lambda _1,\lambda _2}&\lambda _0+\lambda _1\mu +\lambda _2(\mu ^2+\sigma ^2) \end{aligned}$$

(29)

$$\begin{aligned} \mathrm {s.t.}\,&\mu ^{-1}{{\,\mathrm{{\textit{I}}}\,}}(x\ge \tau ) (x-\tau )\le \lambda _0+\lambda _1 x+\lambda _2 x^2. \end{aligned}$$

(30)

Defining $\Phi (x) = \mu \left( \lambda _0+\lambda _1 x+\lambda _2 x^2\right)$, we seek the coefficients that lead to the tightest majorizer for ${{\,\mathrm{{\textit{I}}}\,}}(x\ge \tau ) (x-\tau )$. The de Leeuw and Lange (2009) majorizer for this problem is given by

$$\begin{aligned} \frac{1}{4x_0}(x-\tau +x_0)^2 = \frac{1}{4x_0}x^2 - \left( \frac{\tau }{2x_0}-\frac{1}{2} \right) x + \frac{\tau ^2}{4x_0}+\frac{x_0}{4}-\frac{\tau }{2}, \end{aligned}$$

for some $x_0>0$. This implies that a feasible solution is

$$\begin{aligned} \lambda _0 = \mu ^{-1}\left( \frac{\tau ^2}{4x_0}+\frac{x_0}{4}-\frac{\tau }{2}\right) ,\quad \lambda _1 = \mu ^{-1}\left( \frac{1}{2}-\frac{\tau }{2x_0}\right) ,\quad \lambda _2 = \mu ^{-1}\frac{1}{4x_0}. \end{aligned}$$

(31)

The problem now becomes a univariate minimization problem of (29) in $x_0$, with unique positive solution $x_0^* = \sqrt{\sigma ^2+(\mu -\tau )^2}$. We know that the primal solution $F^*$ will be supported on two or fewer points (Roos et al. 2020) since our dual solution $\Phi ^*$ touches the constraint at two points. These two points are $\tau - \sqrt{\sigma ^2+(\mu -\tau )^2},\tau +\sqrt{\sigma ^2+(\mu -\tau )^2}$. We now need to distinguish two cases, since $\tau - \sqrt{\sigma ^2+(\mu -\tau )^2}<0$ when $\tau <(\mu ^2+\sigma ^2)/(2\mu )$; see Fig. 5 for illustrations of each case.

• Case 1: $\tau \le (\mu ^2+\sigma ^2)/(2\mu )$. Our solution from (31) no longer applies in this case because it gives one negative support point, which makes the corresponding distribution inadmissible under our primal formulation. We make use of a result from Scarf (1958): if Z has a two-point distribution on $\{z_1,z_2\},z_1<z_2$ with mean $\mu$ and variance $\sigma ^2$, then this is a one-parameter family of distributions indexed by $z_1$ with

  $$\begin{aligned} z_2&=\mu +\frac{\sigma ^2}{\mu -z_1}\\ P(Z=z_1)&=1-P(Z=z_2)\\&=\frac{\sigma ^2}{(\mu -z_1)^2+\sigma ^2} \end{aligned}$$

  and we know that the solution has to be supported at $z_1=0$, which means that $z_2=\mu +\frac{\sigma ^2}{\mu }$ and $P(Z=z_1)= \frac{\sigma ^2}{\mu ^2+\sigma ^2}$. Therefore, we seek a parabola that goes through (0, 0) and is tangent to the line $x-\tau$ at $x=\mu +\sigma ^2/\mu$. These conditions define a system of equations that will enable us to find the solution to the dual problem in this case:

  $$\begin{aligned} \lambda _2 (\mu +\sigma ^2/\mu )^2 + \lambda _1 (\mu +\sigma ^2/\mu ) + \lambda _0&= \mu +\sigma ^2/\mu -\tau \\ 2\lambda _2(\mu +\sigma ^2/\mu )+\lambda _1&=1 \\ \lambda _2 0^2 + \lambda _1 0 + \lambda _0&= 0 \end{aligned}$$

  From the final equation, we see that $\lambda _0^*=0$. From the second equation, we determine that $\lambda _1 = 1-2\lambda _2(\mu +\sigma ^2/\mu )$, which substituting into first equation gives

  $$\begin{aligned} \lambda _2 (\mu +\sigma ^2/\mu )^2+\left\{ 1-2\lambda _2(\mu +\sigma ^2/\mu ) \right\} (\mu +\sigma ^2/\mu )&= \mu +\sigma ^2/\mu -\tau \\ -\lambda _2 (\mu +\sigma ^2/\mu )^2&= -\tau \\ \lambda _2&= \frac{\tau }{(\mu +\sigma ^2/\mu )^2}. \end{aligned}$$

  Thus, we have

  $$\begin{aligned} \lambda _1 = 1-\frac{2\tau }{\mu +\sigma ^2/\mu }. \end{aligned}$$

  Returning to the definition of the dual problem in (29) and (30), we see that the solution for $\tau \le (\mu ^2+\sigma ^2)/(2\mu )$ is

  $$\begin{aligned} \lambda _0^*&=0 \\ \lambda _1^*&=\frac{1}{\mu }\left( 1-\frac{2\tau }{\mu +\sigma ^2/\mu } \right) \\ \lambda _2^*&=\frac{\tau }{\mu (\mu +\sigma ^2/\mu )^2}. \end{aligned}$$

  This implies an objective function value of

  $$\begin{aligned} \lambda _0^*+\lambda _1^*\mu +\lambda _2^*(\mu ^2+\sigma ^2)&= \left( 1-\frac{2\tau }{\mu +\sigma ^2/\mu } \right) + \frac{\tau }{\mu (\mu +\sigma ^2/\mu )^2}(\mu ^2+\sigma ^2) \nonumber \\&=\left( 1-\frac{2\mu \tau }{\mu ^2+\sigma ^2} \right) + \frac{\mu \tau }{\mu ^2+\sigma ^2} \nonumber \\&= \frac{\mu ^2+\sigma ^2-\mu \tau }{\mu ^2+\sigma ^2} = \frac{\mu (\mu -\tau )+\sigma ^2}{\mu ^2+\sigma ^2}. \end{aligned}$$

  (32)

• Case 2: $\tau >(\mu ^2+\sigma ^2)/(2\mu )$. In this case, both $\tau -\sqrt{\sigma ^2+(\mu -\tau )^2}$ and $\tau +\sqrt{\sigma ^2+(\mu -\tau )^2}$ are positive: this is a requirement for our feasible solution to the primal problem, since our random variable $T_1$ is supported on ${\mathbb {R}}^+$. Now, we substitute $x_0^*=\sqrt{\sigma ^2+(\mu -\tau )^2}$ into (31), and then plug the resulting $\lambda _0^*,\lambda _1^*,\lambda _2^*,$ in to the dual objective function (29) to obtain

  $$\begin{aligned} \lambda _0^*+\lambda _1^*\mu +\lambda _2^*(\mu ^2+\sigma ^2)&=\mu ^{-1}\left( \frac{\tau ^2}{4x_0^*}+\frac{x_0^*}{4}-\frac{\tau }{2}\right) + \left( \frac{1}{2}-\frac{\tau }{2x_0^*}\right) +\mu ^{-1}\frac{1}{4x_0^*}(\mu ^2+\sigma ^2) \nonumber \\&= \frac{\sigma ^2+(x_0^*+\mu -\tau )^2}{4\mu x_0^*} = \frac{\sigma ^2+(\sqrt{\sigma ^2+(\mu -\tau )^2}+\mu -\tau )^2}{4\mu \sqrt{\sigma ^2+(\mu -\tau )^2}} \nonumber \\&= \frac{\mu -\tau +\sqrt{\sigma ^2+(\mu -\tau )^2}}{2\mu }. \end{aligned}$$

  (33)

A feasible solution for the dual problem represents an upper bound for the primal problem’s solution (Roos et al. 2020); that is

$$\begin{aligned} \sup _{F\in \mathcal {F}_{\mu ,\beta }} \int \mu ^{-1} (x-\tau )_+\,dF(x) \le \lambda _0^*+\lambda _1^*\mu +\lambda _2^*(\mu ^2+\sigma ^2). \end{aligned}$$

We now show that we in fact have strong duality by calculating the objective function value for the primal problem using our solution to the dual problem. We know that the primal solution $F^*$ will be supported on two or fewer points (Roos et al. 2020) since our dual solution $\Phi ^*$ touches the constraint at two points. We proceed again by cases.

• Case 1: $\tau \le (\mu ^2+\sigma ^2)/(2\mu )$. The two support points in this case are as given above, $z_1=0$ and $z_2=\mu +\sigma ^2/\mu$ with mass $\sigma ^2/(\mu ^2+\sigma ^2)$ and $\mu ^2/(\mu ^2+\sigma ^2)$ respectively. The primal objective function value is

  $$\begin{aligned} \mu ^{-1}\int (x-\tau )_+\, dF(x)&= \mu ^{-1}(0-\tau )_+ \frac{\sigma ^2}{\mu ^2+\sigma ^2} + \mu ^{-1}\left( \mu +\frac{\sigma ^2}{\mu } -\tau \right) _+\frac{\mu ^2}{\mu ^2+\sigma ^2} \\&= \left( \frac{\mu ^2+\sigma ^2}{\mu }-\tau \right) _+\frac{\mu }{\mu ^2+\sigma ^2} = 1-\frac{\mu \tau }{\mu ^2+\sigma ^2} \\&= \frac{\mu (\mu -\tau )+\sigma ^2}{\mu ^2+\sigma ^2} \end{aligned}$$

  as long as

  $$\begin{aligned} \frac{\mu ^2+\sigma ^2}{\mu }\ge \tau \end{aligned}$$

  which is implied by the condition defining this case.

• Case 2: $\tau >(\mu ^2+\sigma ^2)/(2\mu )$. The two support points are $z_1=\tau - \sqrt{\sigma ^2+(\mu -\tau )^2}$ and $z_2=\tau +\sqrt{\sigma ^2+(\mu -\tau )^2}$; denote the mass at each of these points by $p_1,p_2$ respectively. The first primal constraint is equivalent simply to $p_1+p_2=1$, so we seek $p_1$ such that

  $$\begin{aligned} \mu&= p_1\left( \tau - \sqrt{\sigma ^2+(\mu -\tau )^2}\right) +(1-p_1)\left( \tau + \sqrt{\sigma ^2+(\mu -\tau )^2}\right) \end{aligned}$$

  so that

  $$\begin{aligned} p_1 = \frac{\tau -\mu +\sqrt{\sigma ^2+(\mu -\tau )^2}}{2 \sqrt{\sigma ^2+(\mu -\tau )^2}}. \end{aligned}$$

  It may be shown that the variance of the resulting distribution indeed equals $\sigma ^2$. Now we calculate the primal objective function value in this case:

  $$\begin{aligned} \mu ^{-1}\int (x-\tau )_+\, dF(x)&= \mu ^{-1}\left( -\sqrt{\sigma ^2+(\mu -\tau )^2}\right) _+ p_1 + \mu ^{-1}\left( \sqrt{\sigma ^2+(\mu -\tau )^2}\right) _+(1-p_1) \\&= \mu ^{-1}\sqrt{\sigma ^2+(\mu -\tau )^2}\left( 1-\frac{\tau -\mu +\sqrt{\sigma ^2 +(\mu -\tau )^2}}{2 \sqrt{\sigma ^2+(\mu -\tau )^2}}\right) \\&= \mu ^{-1}\left( \sqrt{\sigma ^2+(\mu -\tau )^2}-\frac{\tau -\mu +\sqrt{\sigma ^2 +(\mu -\tau )^2}}{2 }\right) \\&= \frac{\tau -\mu +\sqrt{\sigma ^2+(\mu -\tau )^2}}{2\mu }. \end{aligned}$$

In both cases, the primal and dual objective function values are equal, showing we have strong duality and completing the proof. $\square$

Table 2 Probability of no benefit $P(L=0|D=1)$, with $t_0=40$ years and $t_{max}=80$ years

Table 3 Mean lead time $E(L|D=1)$ in years, with $t_0=40$ years and $t_{max}=80$ years

Table 4 Optimal screening frequencies given a frequency–adherence curve defined by (19), with $\beta _1=0.25$. Here, $t_0=40$ years and $t_{max}=80$ years. Columns labeled G assume a gamma distribution for the inter-testing times, while UB represents upper bounds and LB represents lower bounds. Note that $\mu \ge 0.5$ by our predetermined maximum frequency under consideration, so for some entries in this table, the “optimal” frequency in fact corresponds to $\mu \le 0.5$

1.4 Numerical results evaluating bounds

Tables 2 and 3 show results of the numerical studies described in detail in Sect. 5.1. Specifically, for high levels of adherence, we see that the bound (16) performs quite well, as expected by Remark 5. In addition, similarly to Wu et al. (2007), we observe that increasing frequency (equivalently, decreasing $\mu$) contributes to improved outcomes of the hypothetical screening program, with increases in mean lead time and decreases in probability of no benefit.

1.5 Numerical results for $\beta _1=0.25$

Fig. 6

Features of the lead time distribution under the frequency–adherence curve defined by (19). Here, $\beta _1=0.25$; different values of $\beta _2$, representing changing sensitivity of adherence to prescribed frequency, are shown by different lines within each panel. The top two panels show the probability of no benefit $P(L=0|D=1)$, while the bottom two show the mean lead time $E(L|D=1)$ in years. The condition $D=1$ is defined by $t_0=40$ years and $t_{max}=80$ years. Left-hand panels show the bounds for the indicated quantities, assuming only first two moments of the inter-test time distribution are known, while the right-hand panels make the additional assumption that the inter-test times have the gamma distribution