Highlight ArticleCUSUM-Logistic Regression analysis for the rapid detection of errors in clinical laboratory test results
Introduction
The periodic measurement of quality control (QC) material is the main practice used for monitoring the analytical performance of diagnostic tests [1]. A major drawback of this approach is that test performance is not monitored in the time periods between analyses of QC material. This can potentially result in the reporting of a large number of inaccurate test results until the problem is discovered, because QC material is often analyzed only once a day. The advent of advanced automation of chemistry analysers and their subsequent increase in sample throughput has further aggravated this problem and increased the need for the more timely monitoring of clinical laboratory tests.
Another common problem encountered in our current QC practices is that it typically depends on the use of non-commutable QC material. It is not uncommon to observe apparent shifts in test values of QC material but not in real specimens or, shifts in real specimens that are not reflected in QC material because of their different sample matrices [2], [3]. Another limitation is that the use of standard QC material does not assess all the steps in the analysis of a specimen [4]. For example, it does not detect pre-analytical problems related to specimen collection or processing or post-analytical problems related to the calculation and reporting of test results.
The use of patient sample based QC procedures is an alternative approach for detecting test errors [5]. In 1965, Hoffman et al. described the Average of Normals method in which the test results of a large number of patients falling within normal reference intervals are averaged and used to monitor potential changes in the testing process [6]. Later, Cembrowski et al. [7] used computer simulation to demonstrate the primary factors affecting error detection by this method. Besides the number of patient values used to calculate the mean, the ratio of the standard deviation of the truncated patient population to the precision of an analytical method was a major factor in the sensitivity of error prediction. They also showed that the truncation limits should be chosen so that they exclude outliers but still include the majority of the patient test results within the central test distribution [7]. Other improvements to the Average of Normals approach include the exponentially weighted moving average and other computational methods for establishing a mean of a moving window of patient test results [5], [8]. Although patient sample based QC procedures in theory provide a way to monitor analytical performance between QC runs, they are not widely used. This is largely due to the fact that for many less frequently ordered tests, the number of patient results that are needed to detect a clinically significant error is often greater than the number of patient samples that would typically be analyzed between QC runs for many clinical laboratories. This is also true for those tests with a wide reference range, which greatly limits the sensitivity of error detection by this method [5].
In this study, we describe a novel patient sample based QC procedure involving the use of CUSUM scoring and logistic regression, which we refer to as CUSUM-Logistic Regression (CSLR). In addition to monitoring the value of patient test results, it depends upon the inter-relationship between test results, as well as the time of day and day of the week that a test is performed. Using data from a standard clinical chemistry metabolic panel, we show that the CSLR approach is a relatively simple and sensitive method for using patient sample test results to monitor the performance of clinical laboratory tests between QC runs.
Section snippets
Clinical laboratory analysis
Laboratory test results from a commonly used Chem-14 metabolic chemistry panel (sodium (Na), potassium (K), chloride (Cl), urea (BUN), creatinine (Creat), bicarbonate (HCO3), alkaline phosphatase (ALP), alanine transaminase (ALT), aspartate transaminase (AST), glucose (Glu), albumin (Alb), calcium (Ca), total protein (TP), total bilirubin (TB)) were collected over a four year period. Samples were analyzed on the Synchron LX20 analyzer (Beckman Coulter, Atlanta GA 30326) at the Department of
Multiple regression model for predicting test results
Because of the homeostatic and pathophysiologic relationships between test analytes there is often a close correlation between many different laboratory test results. Using hierarchical based clustering, this test inter-relationship for the Chem-14 test panel can be observed by their cluster pattern (Fig. 1A). Similar relationships were observed in the correlation coefficients between test pairs (Fig. 1B). For example, Ca, Alb and TP formed a tight cluster, with all these tests positively
Discussion
QC procedures for monitoring the production of most manufactured goods are conceptually more straightforward. The desired specifications of any given product can be defined a priori and then be monitored in real time to determine if they are being met during the manufacturing process. This approach is obviously not applicable for clinical laboratory testing and we instead usually monitor QC material and not patient samples. To control costs and improve sample throughput, QC material is only
Acknowledgments
This research was supported by intramural research funds of the NIH Clinical Center.
References (11)
- et al.
Quality control for the clinical laboratory
Clin. Lab. Med.
(1983) Use of patient data for quality control
Clin. Lab. Med.
(1986)- et al.
Advantages of CUSUM techniques for quality control in clinical chemistry
Clin. Chim. Acta
(1980) - et al.
Commutability limitations influence quality control results with different reagent lots
Clin. Chem.
(2011) - et al.
Analytical biases with liquid quality control material
Am. J. Clin. Pathol.
(1983)
Cited by (0)
- 1
Present address: Department of Chemical Pathology, University of KwaZulu Natal and Inkosi Albert Luthuli Central Hospital, National Health Laboratory Service, Durban, South Africa.
- 2
Present address: Lancet Laboratories, Johannesburg, South Africa.