Elsevier

Clinical Biochemistry

Volume 49, Issue 3, February 2016, Pages 201-207
Clinical Biochemistry

Highlight Article
CUSUM-Logistic Regression analysis for the rapid detection of errors in clinical laboratory test results

https://doi.org/10.1016/j.clinbiochem.2015.10.019Get rights and content

Highlights

  • Laboratory test errors between QC runs can remain undetected for long periods.

  • A lengthy delay in detecting errors may lead to medical errors.

  • Traditional moving mean methods of using patient samples for QC are not timely.

  • We describe a rapid method of using patient samples for test error detection.

Abstract

Objective

The main drawback of the periodic analysis of quality control (QC) material is that test performance is not monitored in time periods between QC analyses, potentially leading to the reporting of faulty test results. The objective of this study was to develop a patient based QC procedure for the more timely detection of test errors.

Method

Results from a Chem-14 panel measured on the Beckman LX20 analyzer were used to develop the model. Each test result was predicted from the other 13 members of the panel by multiple regression, which resulted in correlation coefficients between the predicted and measured result of > 0.7 for 8 of the 14 tests. A logistic regression model, which utilized the measured test result, the predicted test result, the day of the week and time of day, was then developed for predicting test errors. The output of the logistic regression was tallied by a daily CUSUM approach and used to predict test errors, with a fixed specificity of 90%.

Results

The mean average run length (ARL) before error detection by CUSUM-Logistic Regression (CSLR) was 20 with a mean sensitivity of 97%, which was considerably shorter than the mean ARL of 53 (sensitivity 87.5%) for a simple prediction model that only used the measured result for error detection.

Conclusion

A CUSUM-Logistic Regression analysis of patient laboratory data can be an effective approach for the rapid and sensitive detection of clinical laboratory errors.

Introduction

The periodic measurement of quality control (QC) material is the main practice used for monitoring the analytical performance of diagnostic tests [1]. A major drawback of this approach is that test performance is not monitored in the time periods between analyses of QC material. This can potentially result in the reporting of a large number of inaccurate test results until the problem is discovered, because QC material is often analyzed only once a day. The advent of advanced automation of chemistry analysers and their subsequent increase in sample throughput has further aggravated this problem and increased the need for the more timely monitoring of clinical laboratory tests.

Another common problem encountered in our current QC practices is that it typically depends on the use of non-commutable QC material. It is not uncommon to observe apparent shifts in test values of QC material but not in real specimens or, shifts in real specimens that are not reflected in QC material because of their different sample matrices [2], [3]. Another limitation is that the use of standard QC material does not assess all the steps in the analysis of a specimen [4]. For example, it does not detect pre-analytical problems related to specimen collection or processing or post-analytical problems related to the calculation and reporting of test results.

The use of patient sample based QC procedures is an alternative approach for detecting test errors [5]. In 1965, Hoffman et al. described the Average of Normals method in which the test results of a large number of patients falling within normal reference intervals are averaged and used to monitor potential changes in the testing process [6]. Later, Cembrowski et al. [7] used computer simulation to demonstrate the primary factors affecting error detection by this method. Besides the number of patient values used to calculate the mean, the ratio of the standard deviation of the truncated patient population to the precision of an analytical method was a major factor in the sensitivity of error prediction. They also showed that the truncation limits should be chosen so that they exclude outliers but still include the majority of the patient test results within the central test distribution [7]. Other improvements to the Average of Normals approach include the exponentially weighted moving average and other computational methods for establishing a mean of a moving window of patient test results [5], [8]. Although patient sample based QC procedures in theory provide a way to monitor analytical performance between QC runs, they are not widely used. This is largely due to the fact that for many less frequently ordered tests, the number of patient results that are needed to detect a clinically significant error is often greater than the number of patient samples that would typically be analyzed between QC runs for many clinical laboratories. This is also true for those tests with a wide reference range, which greatly limits the sensitivity of error detection by this method [5].

In this study, we describe a novel patient sample based QC procedure involving the use of CUSUM scoring and logistic regression, which we refer to as CUSUM-Logistic Regression (CSLR). In addition to monitoring the value of patient test results, it depends upon the inter-relationship between test results, as well as the time of day and day of the week that a test is performed. Using data from a standard clinical chemistry metabolic panel, we show that the CSLR approach is a relatively simple and sensitive method for using patient sample test results to monitor the performance of clinical laboratory tests between QC runs.

Section snippets

Clinical laboratory analysis

Laboratory test results from a commonly used Chem-14 metabolic chemistry panel (sodium (Na), potassium (K), chloride (Cl), urea (BUN), creatinine (Creat), bicarbonate (HCO3), alkaline phosphatase (ALP), alanine transaminase (ALT), aspartate transaminase (AST), glucose (Glu), albumin (Alb), calcium (Ca), total protein (TP), total bilirubin (TB)) were collected over a four year period. Samples were analyzed on the Synchron LX20 analyzer (Beckman Coulter, Atlanta GA 30326) at the Department of

Multiple regression model for predicting test results

Because of the homeostatic and pathophysiologic relationships between test analytes there is often a close correlation between many different laboratory test results. Using hierarchical based clustering, this test inter-relationship for the Chem-14 test panel can be observed by their cluster pattern (Fig. 1A). Similar relationships were observed in the correlation coefficients between test pairs (Fig. 1B). For example, Ca, Alb and TP formed a tight cluster, with all these tests positively

Discussion

QC procedures for monitoring the production of most manufactured goods are conceptually more straightforward. The desired specifications of any given product can be defined a priori and then be monitored in real time to determine if they are being met during the manufacturing process. This approach is obviously not applicable for clinical laboratory testing and we instead usually monitor QC material and not patient samples. To control costs and improve sample throughput, QC material is only

Acknowledgments

This research was supported by intramural research funds of the NIH Clinical Center.

References (11)

There are more references available in the full text version of this article.

Cited by (0)

1

Present address: Department of Chemical Pathology, University of KwaZulu Natal and Inkosi Albert Luthuli Central Hospital, National Health Laboratory Service, Durban, South Africa.

2

Present address: Lancet Laboratories, Johannesburg, South Africa.

View full text