Skip to main content
Log in

Power, Selection Bias and Predictive Performance of the Population Pharmacokinetic Covariate Model

  • Published:
Journal of Pharmacokinetics and Pharmacodynamics Aims and scope Submit manuscript

Abstract

Identification and quantification of covariate relations is often an important part of population pharmacokinetic/pharmacodynamic (PK/PD) modelling. The covariate model is regularly built in a stepwise manner. With such methods, selection bias may be a problem if only statistically significant covariates are accepted into the model. Competition between multiple covariates may further increase selection bias, especially when there is a moderate to high correlation between the covariates. This can also result in a loss of power to find the true covariates. The aim of this simulation study was to investigate the effect on power, selection bias and predictive performance of the covariate model, when altering study design and system-related quantities. Data sets with 20–1000 subjects were investigated. Five covariates were created by sampling from a multivariate standard normal distribution. The true covariate was set up to have no, low, moderate and high correlation to the other four covariates, respectively. Data sets, in which each individual had two or three PK observations, were simulated using a one-compartment i.v. bolus model. The true covariate influenced clearance according to one of several magnitudes. Different magnitudes of residual error and inter-individual variability in the structural model parameters were also introduced to the simulation model. A total of 7400 replicate data sets were simulated independently for each combination of the above conditions. Models with one of the five simulated covariates influencing clearance and the model without any covariate were fitted to the data. The probability of selecting ( according to a pre-specified P-value ) the different covariates, along with the estimated covariate coefficient, was recorded. The results show that selection bias is very high for small data sets ( ≤ 50 subjects ) simulated with a weak covariate effect. If selected under these circumstances, the covariate coefficient is on average estimated to be more than twice its true value, making the covariate model useless for predictive purposes. Surprisingly, even though competition from false covariates caused substantial loss in the power of selecting the true covariate, the already high selection bias increased only marginally. This means that the bias due to competition is negligible if statistical significance is also required for covariate selection. Bias and predictive performance are direct functions of power, only indirectly affected by study design and system-related quantities. Mainly because of selection bias, low-powered covariates can be expected to harm the predictive performance when selected. For the same reason these low-powered covariates may falsely appear to be clinically relevant when selected. If the aim of an analysis is predictive modelling, we do not recommend stepwise selection or significance testing of covariates to be performed on small or moderately sized data sets ( <50–100 subjects)

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E. N. Jonsson and M. O. Karlsson. Automated covariate model building within NONMEM. Pharm. Res. 15:1463-1468 (1998).

    Google Scholar 

  2. A. J. Miller. Selection of subsets of regression variables. J. R. Stat. Soc. Series A (General). 147:389-425 (1984).

    Google Scholar 

  3. U. Wahlby, E. N. Jonsson, and M. O. Karlsson. Comparison of stepwise covariate model building strategies in population pharmacokinetic-pharmacodynamic analysis. AAPS Pharm. Sci. 4:27(2002).

    Google Scholar 

  4. P. L. Bonate. The effect of collinearity on parameter estimates in nonlinear mixed effect models. Pharm. Res. 16:709-717 (1999).

    Google Scholar 

  5. R. O'Neill and G. B. Wetherill. The present state of multiple comparison methods. J. R. Stat. Soc. Series B (Methodological). 33:218-250 (1971).

    Google Scholar 

  6. E. W. Steyerberg, M. J. Eijkemans, and J. D. Habbema. Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. J. Clin. Epidemiol. 52:935-942 (1999).

    Google Scholar 

  7. G. E. P. Box. Use and abuse of regression. Technometrics. 8:629-629 (1966).

    Google Scholar 

  8. D. W. Cockroft and M. H. Gault. Prediction of creatinine clearance from serum creatinine. Nephron. 16:31-41 (1976).

    Google Scholar 

  9. R. Bruno, N. Vivler, J. C. Vergniol, S. L. De Phillips, G. Montay, and L. B. Sheiner. A population pharmacokinetic model for docetaxel (Taxotere): model building and validation. J. Pharmacokinet. Biopharm.. 24:153-172 (1996).

    Google Scholar 

  10. I. The MathWorks. MATLAB. The Math Works, Natick, MA.

  11. S. L. Beal. NONMEM Users Guides. NONMEM Project Group, University of California at San Francisco, San Francisco, CA, 1994.

    Google Scholar 

  12. U. Wahlby, E. N. Jonsson, and M. O. Karlsson. Assessment of actual significance levels for covariate effects in NONMEM. J. Pharmacokinet. Pharmacodyn. 28:231-252 (2001).

    Google Scholar 

  13. L. B. Sheiner and S. L. Beal. Some suggestions for measuring predictive performance. J. Pharmacokinet. Biopharm. 9:503-512 (1981).

    Google Scholar 

  14. J. S. U. Hjorth. Computer Intensive Statistical Methods. Chapman & Hall,London; 1994.

    Google Scholar 

  15. P. I. Lee. Design and power of a population pharmacokinetic study. Pharm. Res. 18:75-82 (2001).

    Google Scholar 

  16. E. N. Jonsson, J. R. Wade, and M. O. Karlsson. Comparison of some practical sampling strategies for population pharmacokinetic studies. J. Pharmacokinet. Biopharm. 24:245-263 (1996).

    Google Scholar 

  17. S. Retout, S. Duffull, and F. Mentre. Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs. Comput. Meth. Prog. Biomed. 65:141-151 (2001).

    Google Scholar 

  18. i. MathSoft.S-Plus 2000 Math Soft, Seattle, WA.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ribbing, J., Niclas Jonsson, E. Power, Selection Bias and Predictive Performance of the Population Pharmacokinetic Covariate Model. J Pharmacokinet Pharmacodyn 31, 109–134 (2004). https://doi.org/10.1023/B:JOPA.0000034404.86036.72

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:JOPA.0000034404.86036.72

Navigation