Accuracy of Portable Blood Glucose Monitoring

Transcription

1 CLINICAL CHEMISTRY Accuracy of Portable Blood Glucose Monitoring Effect of Glucose Level and Prandial State DINO A. VALLERA, M.D., MICHAEL G. BISSELL, M.D., PH.D., M.P.H., AND WILLIAM BARRON, M.D. Glucose was determined on capillary and venous blood samples from 274 adult diabetics by three different methodologies: the Glucoscan 2000 and Accu-Check II portable glucose meters (capillary) and the Kodak Ektachem 700 analyzer (venous). Both glucose meters correlated significantly with the Ektachem results. A significant positive bias was found for the Glucoscan compared with Ektachem, not found with the Accu-Check II. The Accu-Check performed better than the Glucoscan at venous plasma glucose levels < 1 g/l. The mean error of Glucoscan determinations was significantly greater and biased positive when the measurement was performed within 4 hours of eating, whereas no such effect was seen with Accu-Check error. Multiple regression analysis revealed that the Glucoscan measurement was independently influenced by both venous plasma glucose and prandial state, whereas the Accu-Check II measurement was not dependent on either variable. The within-run precision for both glucose meters were comparable. (Key Words: Portable glucose monitoring; Accuracy.) Am J Clin Pathol 1991;95: Portable capillary blood glucose meters have been used extensively in both inpatient and outpatient settings for many reasons, including patient and physician convenience and cost. 1,2 Although the accuracy and precision of this technology have been addressed by a number of previous investigators, 3,4 many of these studies have had considerable limitations. For example, sample sizes are often small, 5 " 7 meter accuracy frequently is addressed only through determination of the correlation coefficient between meter and reference laboratory methods, 5,8 " 12 and the impact of ambient plasma glucose levels on accuracy often is neglected. Furthermore, we are aware of only one study that has addressed the impact of prior feeding on the concordance between capillary blood glucose measured by a portable meter and venous plasma glucose de- termined by a laboratory reference methodology. 12 Consideration of this factor is important because there may be significant peripheral tissue glucose extraction in the postprandial state, 13 resulting in venous plasma levels that are considerably lower than capillary values. The present investigation was designed to address a number of these issues by performing a large number of capillary blood glucose determinations on two currently popular instruments, analyzing differences between portable meter and standard laboratory measurements as a function of both ambient plasma glucose level and the time since the patient's last meal. METHODS Study subjects included 274 adult diabetics on whom a blood glucose level had been requested by their primary From the Departments of Pathology, Medicine, and Obstetrics care and physician in the University of Chicago Health Service, Gynecology, University of Chicago Hospitals, Chicago, Illinois. a large internal medicine clinic with more than 50,000 Received September 18, 1989; accepted for publication July 24, outpatient visits per year. Participants included patients Dr. Vallera is currently with the Department of Pathology, Hinsdale on dietary therapy only, on oral hypoglycemic agents and Hospital, Hinsdale, Illinois; Dr. Bissell is currently with the Department on insulin. Each subject was asked when he had last eaten, of Clinical Pathology, City of Hope National Medical Center, Duarte, California. and the time was recorded to the nearest 30 minutes. Supported by Boehringer Mannheim Diagnostics, Indianapolis, Indiana. immediately into a tube containing potassium oxalate Blood was obtained from an antecubital vein and placed Address reprint requests to Dr. Bissell: Department of Clinical Pathology, City of Hope National Medical Center, 1500 East Duarte Road, (14.0 mg) and sodium fluoride (17.5 mg) (Becton Dickinson Vacutainer Systems, Rutherford, NJ). Plasma Duarte, California ob- 247

2 248 CLINICAL CHEMISTRY tained from the specimen was analyzed for glucose in the General Clinical Chemistry Laboratory of the University of Chicago Hospitals on the Ektachem 700 analyzer (Eastman Kodak Co., Rochester, NY), which uses a glucose oxidase-based reflection densitometric method. Within 4 minutes after venipuncture, the patient's finger was punctured with a spring-loaded lancet to obtain a drop of capillary whole blood. This was analyzed for glucose with either the Glucoscan 2000 with Glucoscan test strips (Lifescan, Inc., Mountain View, CA) or the Accu- Check II with Chemstrip bg test strips (Boehringer Mannheim Diagnostics, Indianapolis, IN). All determinations were performed by a single nurse who was trained in the operation and quality control of both instruments as directed by the manufacturers. One instrument of each type was used for the entire study. The precision of the Ektachem 700 was assessed by performing 20 repeated glucose determinations on a single sample over 20 consecutive minutes and by analyzing the same sample once daily on 20 consecutive days. Precision of each portable meter was evaluated by daily determinations of the glucose concentration of a standard test solution (provided by the manufacturer) for 20 days. In addition, the precision of the entire portable glucose meter methodology was assessed for each instrument by performing 20 repeated capillary blood glucose determinations on a single subject over a minute period. The statistical approach used to assess the agreement between each meter and the laboratory reference method is a modification of that suggested by Bland and Altman. 14 First, least squares linear regression analysis was used to assess correlations between meter and Ektachem determinations. Next, two-tailed paired /-tests were performed to evaluate the difference between meter and Ektachem determinations (referred to here as "meter bias" or "error") and to compare the magnitude of these two differences for the two meters. In addition, 95 confidence intervals (CI) were calculated for the mean bias for each meter. To determine whether meter bias was dependent on plasma glucose level, we performed a linear regression of the difference between meter and Ektachem measurements on the Ektachem value. To further clarify this relationship, data were stratified in 1 g/l increments according to the Ektachem value, and an analysis of variance was performed, followed by two-tailed two sample Mests to assess differences between the meters at various levels of venous plasma glucose. To analyze the impact of prior meal ingestion on monitor accuracy, meter and Ektachem values were grouped by the length of time that had passed since the last meal. Preliminary analysis separating data into four groups (0-2.0 hours, hours, hours, and >6.0 hours) indicated that results for groups 1 and 2 were similar, as were those for groups 3 and 4; therefore, data were combined into two groups, hours and >4.0 hours since the last meal, and two-tailed, paired and two-sample t- tests were performed to assess group differences. Finally, a multiple regression analysis was performed to assess the independent contributions of (1) the Ektachem value, and (2) the time since the last meal, on meter error. Data are reported as mean ± 1 SD. A P value of <0.05 was considered significant, except where multiple testing was performed, in which case the critical P value was adjusted by the Bonferroni method (both uncorrected and Bonferroni P values are presented). All calculations were performed on a microcomputer using SYSTAT software. Precision evaluation experiments were performed on both the Accu-Check II and the Ektachem according to the National Committee for Clinical Laboratory Standards (NCCLS) Guidelines EP5T. Briefly, manufacturers' supplied glucose control material was analyzed in duplicate, in two runs per day, for 20 days. Two analyte concentrations were tested for each method. This data was analyzed to provide an estimate of within-run and total imprecision. For the Accu-Check II, the determinations were performed by the same trained nurse who performed the patient determinations. The Ektachem study was performed by trained medical technologists in the General Clinical Chemistry Laboratory. The Glucoscan 2000 could not be included in this phase of the study because of the nature of its control materials. These were packaged in small plastic vials and, according to the manufacturer's instructions, are to be used within 24 hours of opening. This would necessitate the use of 20 different vials of control material for each analyte concentration to complete the study and would introduce another source of variability into the data. RESULTS A total of 277 determinations were made with the Glucoscan 2000, and 232 were performed on the Accu-Check II. There was a highly significant correlation between capillary (meter) and venous (Ektachem) glucose values for both instruments: Glucoscan: meter value = 0.84 X (Ektachem value) , r = 0.94 Accu-Check: meter value = 0.92 X (Ektachem value) , r = 0.94 Despite these results, further evaluation disclosed significant differences in performance of the two meters. Capillary glucose values measured with the Glucoscan averaged 1.91 ± 0.84 g/l (range = g/l; n = 277), whereas venous plasma glucose values on the same A.J.C.P. February 1991

3 VALLERA, BISSELL, AND BARRON 249 Accuracy of Portable Blood Glucose Monitoring subjects measured with the Ektachem averaged 1.85 ± 0.96 g/l (range = g/l), a highly significant difference (P = 0.001). The average absolute difference between the Glucoscan and Ektachem was ± g/l (95 confidence interval = ), and the average percentage difference was 8.8 ±21.6 (95 confidence interval = ). The observation that neither of these 95 confidence intervals includes 0 supports the given result (i.e., the mean Glucoscan 2000 and Ektachem values are significantly different from one another). Mean capillary glucose values determined with the Accu-Check II were 1.95 ± 0.99 g/l (range = g/l; n = 232), and paired venous samples measured on the Ektachem averaged 1.94 ± 1.01 g/l (range = g/l), a difference that was not significant (P = 0.93). The mean difference between the Accu-Check and Ektachem was ± 0.35 g/l (95 confidence interval = ), data which confirm the lack of difference between Accu-Check and Ektachem determinations. For the Glucoscan, 79.8 of determinations were within 20 of the correct value, whereas 87.9 of the Accu-Check measurements were in this range. Fourteen Glucoscan (5.0) and five Accu-Check (2.2) values deviated more than 50 from that determined on the Ektachem. Direct comparison of the average bias of the two meters indicated a highly significant difference when measured as a percentage of the Ektachem value. (Glucoscan: 8.8 ± 21.6; Accu-Check: 1.1 ± 21.8, P < 0.001), and a trend in the same direction when the bias was expressed in g/l (Glucoscan: ± g/l; Accu- Check: ± g/l, P = 0.038, Bonferroni critical value: P = 0.025). Next, we analyzed the effect of the venous plasma glucose level (Ektachem measurement) on the accuracy of each meter. For the Glucoscan there was a significant linear correlation between the meter error expressed in either absolute [(Glucoscan value) - (Ektachem value) = X (Ektachem value) , r = 0.486, P < 0.001] or percentage terms [ difference between Glucoscan and Ektachem = X (Ektachem) , r = 0.477, P < 0.001]. This effect was much less evident for the Accu- Check II, and was significant only for the error measured in absolute terms [difference (g/l) = X (Ektachem) , r = 0.229, P< 0.001; difference = X (Ektachem) + 4.8, r = 0.09, P = 0.18]. This effect is better appreciated when the data are grouped in 1-g/L intervals of the Ektachem value (Table 1). The average bias of the Glucoscan is more than 20 (95 confidence interval = 19-28) when the Ektachem value is less than 1 g/l, becomes minimal in the range 2-3 g/l, and is more than -10 when the venous plasma level is >4 g/l. In contrast, the error of the Accu-Check at the lowest glucose values averaged -6 ± 15 (95 confidence interval = -2 11; P < vs. Glucoscan error, Bonferroni critical P = 0.01) and varied from 6.3 to +7.8 in other ranges of venous glucose (= not significant vs. Glucoscan for each of these ranges). Thus, the Accu-Check performed better (i.e., had less bias) than the Glucoscan primarily when the venous plasma glucose was <lg/l. It also is of interest to note that all meter determinations with a difference from the Ektachem value of greater than 90 occurred in the most commonly encountered venous plasma glucose range (1-2 g/l) for both the Glucoscan (n = 4) and Accu-Check (n = 3). These seven determinations were made on four different subjects; on two of these, other meter measurements were within 20 of Ektachem values. When these seven outliers were excluded, the mean measurement error in the 1-2 g/l range was 8 ± 13 (95 confidence interval = 6-11) for the Glucoscan and 3 ± 16 (95 confidence interval = ) for the Accu-Check (P = 0.023, Bonferroni critical P = 0.01). Assessment of the effect of prior meal ingestion on measurement bias indicated further differences in meter performance (Table 2). The mean error of Glucoscan determinations was significantly greater and biased in a positive direction when the measurement was performed within 4 hours of eating, whereas recent food intake did TABLE 1. METER BIAS IN ABSOLUTE (mg/dl) AND PERCENTAGE TERMS AS A FUNCTION OF VENOUS PLASMA GLUCOSE LEVEL Venous Plasma Glucose (q/l) 3'L Glucoscan 2000 Bias H 9IL Accu-Check II Bias n < ^ ± * ± ± ± ± ±25* ±8-11 ± ± ± ± ± ±0.67-6± 15 8±30f 1 ± ± When four outliers { difference from Ektachem >90) were excluded, differences were 0.11 ± 0.20 g/l and 8 ± 13. t When three outliers were excluded, differences were 0.05 ± 0.25 g/l and 3 ± 16. Results are expressed as mean ± SD; n = number of paired determinations. Vol. 95 No. 2

4 250 CLINICAL CHEMISTRY TABLE 2. EFFECT OF PRIOR MEAL INGESTION ON METER BIAS GLUCOSCAN 2000 Time Since Last Meal Instrument Units 0-4 hours n >4 hours n Glucoscan Accu-Check g/l g/l ± ± ± 0.38* 2.1 ± ± 0.28* 4.5 ± 16.9f ± ± I * P < 0.02 for the comparison with the 0-4 hour time period. t P < for the comparison with the 0-4 hour time period. X P = for the comparison with Glucoscan. P < for the comparison with Glucoscan. Bonferroni critical P = Results are expressed as mean ± SD; n = number of paired determinations DIFFERENCE (X) not appear to have a significant effect on the Accu-Check meter error. Furthermore, the Accu-Check performed significantly better (i.e., less bias) compared with the Glucoscan only for measurements made less than 4 hours after the last meal. Finally, multiple regression analysis was performed to determine whether the level of venous plasma glucose and the time since the last meal had independent effects on meter bias. Regression coefficients (Table 3) demonstrate that the bias of the Glucoscan, in both g/l and percent, is independently influenced by both plasma glucose and prandial state. In contrast, the bias of the Accu- Check, as a percentage of the Ektachem value, was dependent upon neither of these variables, although the bias in g/l was significantly influenced, albeit to a smaller degree than for the Glucoscan, by the venous plasma glucose. For the Accu-Check II meter, the overall mean of all determinations using the high glucose control material was g/l. The within-run precision was g/l, whereas the between-run precision was g/l (coefficient of variation = 8.7). With the low concentration glucose control, the overall mean was g/l, with the within-run precision being g/l and the betweenrun precision being g/l (coefficient of variation = 6.4). TABLE 3. COEFFICIENT OF MULTIPLE REGRESSION ANALYSIS EVALUATING THE DEPENDENCE OF METER BIAS ON PLASMA VENOUS GLUCOSE AND TIME SINCE LAST MEAL Instrument Units Glucoscan Accu-Check Glucose Level g/l ± ±0.01 g/l ± Time Interval < ± < ± < ± ± Results are expressed as mean ± SE. "Time period since last meal" is categorized into "0-4 hours" or ">4 hours." ACCUCHECK II B DIFFERENCE () FIG. 1. A (upper). Percent differences between EKTACHEM and GLUCOSCAN 2000 values grouped according to the level of venous glucose determined by the EKTACHEM. Note that the ambient level of venous glucose has a significant effect on meter bias and that when venous glucose is less than 1 g/l, bias is approximately +20. B (lower). Percent differences between EKTACHEM and ACCUCHECK II values grouped according to the level of venous glucose determined by the Ektachem. There is no significant relationship between meter bias and venous plasma glucose levels. With the Ektachem, using the high concentration glucose control, the overall mean was 4.28 g/l, with the within-run precision being g/l, between-run precision being 4.08 g/l (coefficient of variation = 0.95). For the low glucose control, the overall mean was g/l, with within-run precision being g/l and between-run precision being g/l (coefficient of variation = 5.5). In the whole method precision study, both the Accu- Check II and Glucoscan 2000 were tested. Using the Glucoscan 2000, the first volunteer's mean blood glucose concentration was 0.94 g/l, with SD of g/l (coefficient of variation = 6.5). The second volunteer was tested with the Accu-Check II, yielding a mean blood glu- A.J.C.P. February 1991

5 VALLERA, BISSELL, AND BARRON 251 Accuracy of Portable Bh <d Glucose Monitoring cose concentration of g/l, with SD of g/l and coefficient of variation of 5.5. Repetitive sampling (n = 20) of the Ektachem high glucose control resulted in a mean concentration of g/l, with SD of g/l and coefficient of variation of For the low control, the mean concentration was g/l, with SD of g/l and coefficient of variation of DISCUSSION This study critically evaluated the performance of two popular blood glucose meters in a busy clinical setting and attempted to identify variables that might account for any observed bias. As reported by others, 815 we found that both meters' readings were highly correlated (r = 0.94) with the laboratory reference method; however, additional analysis disclosed that, on average, the Accu-Check II provided more accurate determinations, with a mean bias of only 1. i compared with 8.8 for the Glucoscan Yet more detailed evaluation disclosed that the greater bias of the latter instrument occurred primarily at lower levels of plasma glucose and when the patient had eaten within several hours of capillary glucose measurement. There have been numerous published reports that purportedly have investigated the accuracy of portable blood glucose monitors. 5 " 8,10,11,15 " 20 However, as discussed in detail by Bland and Altman, 14 many investigators inappropriately report only the correlation coefficient between measurements made by meter and some reference methodology. 5 " 12 Such a statistic fails to provide clinically relevant information {i.e., how much does the meter determination deviate from the true [laboratory reference] value, and does this deviation change at various levels of venous plasma glucose). Godine and colleagues 15 evaluated the accuracy of the Accu-Check meter when used by staff nurses and found a mean error of 8 ± 7, whereas Landon and colleagues, 20 studying the same meter, observed a mean deviation of approximately 15. Brooks and co-workers' 8 found a mean bias of 0.06 ± 0.30 g/l for the Accu-Check II and 0.08 ± 0.27 g/l for the Glucoscan 2000 (percentage data not available). Results of average bias for the two meters in the present study were in the ranges cited, with a mean error for the Accu-Check II of 1 ± 22 (0 ± 0.35 g/l) and 9 ± 22 (0.06 ± 0.33 g/l) for the Glucoscan Although the mean measurement error (as a of the venous glucose level) of both glucose meters was relatively small (<10; 95 confidence interval for mean bias: Glucoscan: ; Accu-Check II: ), this does not provide an estimate of a given capillary glucose determination, a parameter of considerable clinical importance. The latter is better estimated by the 95 limits of agreement (i.e., mean error ± 2 SD), which in the pres- ent study allow one to be 95 certain that a given glucoscan measurement was between -34 and +52 of the laboratory reference value and that an Accu-Check determination was between 43 and +45 of this value. When viewed in this light, the two meters do not appear, on average, to differ as much as suggested by comparison of the mean bias. Furthermore, the range of these limits make it clear that portable blood glucose meter results must be interpreted with caution, particularly in those ranges where a change in therapy may have substantial adverse effects. 21 Overall estimates of glucose meter bias as described may be misleading if this bias is significantly dependent on the level of venous plasma glucose (e.g., the bias may be large at one glucose level and minimal at another). Indeed, we found a significant linear relationship between Glucoscan 2000 meter error and venous plasma glucose, with highest errors at the lowest levels of plasma glucose, a phenomenon not observed for the Accu-Check II. Two previous studies of earlier models of the Accu-Check meter provided similar results. 6,20 When we stratified venous glucose levels (Table 1), it was discovered that the major difference between the two meters occurred at concentrations of <1 g/l, at which level the mean bias of the Glucoscan was +23 (95 limits of agreement: ), whereas the average error for the Accu-Check II was 6 (95 limits of agreement: ). These observations may have important clinical implications. For example, when rigid metabolic control is being attempted, such as is standard during diabetic pregnancy 10,22 (for which desired fasting and postprandial glucose levels are <1 and <1.2 g/l, respectively), Glucoscan 2000 results may lead to excessive insulin administration. On the other hand, use of Accu-Check II would be somewhat more likely to lead to unacceptably high blood glucose levels. Such conclusions highlight the need to understand glucose meter performance at levels of blood glucose that are likely to lead to changes in therapy. Another factor that might affect meter accuracy is the proximity of glucose measurement to the patient's last meal because significant forearm glucose uptake occurs in the postprandial state. 13 This could lead to capillary (meter) glucose levels that are significantly higher than venous (Ektachem) if food had been consumed shortly before measurement. 23 We are aware of only one study that addressed the impact of this phenomenon on the apparent bias of portable meter capillary glucose determinations. 12 In the fasting state, capillary determinations exceeded venous glucose levels by 14; however, 1 and 2 hours after a 100-g glucose load, capillary measurements were, respectively, 20 and 19 higher than venous determinations. Surprisingly, we found a similar effect only with the Glucoscan 2000 and not with the Accu-Check Vol. 95 No. 2

6 252 CLINICAL CHEMISTRY II (Table 2). Furthermore, multiple regression analysis indicated that this was not simply due to the positive bias of the Glucoscan 2000 at certain levels of venous glucose. Thus, we have no explanation for this observation, but the data indicate that a greater allowance for error must be taken into account if the Glucoscan 2000 is used to measure capillary glucose in the postprandial state. In considering all of these statistics, one must balance factors such as availability, turnaround time, convenience, and portability against the difference in accuracy and precision in various testing systems. These factors may become more or less important in different clinical settings. For example, accuracy and precision may be more important in measurements used to adjust insulin doses in a "rigidly" controlled diabetic, whereas ease of operation and availability may have increased importance in less tightly controlled patients. REFERENCES 1. American Diabetes Association. Bedside blood glucose monitoring in hospitals. Diabetes Care 1986;9: Belsey R, Morrison JI, Whitlow KJ, Baer DM, Nelson S, Hardwick DF. Managing bedside glucose testing in the hospital. JAMA 1987;258: Rasaiah B. Self-monitoring of the blood glucose level: potential sources of inaccuracy. Can Med Assoc J 1985;132: Villeneuve ME, Murphy J, Mazze RS. Evaluating blood glucose monitors. Am J Nurs 1985;11: Gifford-Jorgensen RA, Borchert J, Hassanein L, Tilzer GA, Eaks GA, Moore WV. Comparison offiveglucose meters for self-monitoring of blood glucose by diabetic patients. Diabetes Care 1986;9: James K, Latham D, Marrero J, Barrett DA. Capillary blood glucose measured by the laboratory phlebotomy team. Laboratory Medicine 1985;16: Webb DJ, Lovesay JM, Ellis A, Knight AH. Blood glucose monitors: a laboratory and patient assessment. Br Med J 1980;1: Chiasson JL, Morrisset R, Hamet P. Precision and costs of techniques for self-monitoring of serum glucose levels. Can Med Assoc J 1984;130: Drucker RF, Williams DRR, Price CP. Quality assessment of blood glucose monitors in use outside the hospital laboratory. J Clin Pathol 1983;36: Marquette GP, Dillard T, Bietla S, Niebyl JR. The accuracy of visual and meter determinations of blood glucose with the use of Chemstrip bg. Am J Obstet Gynecol 1985;153: Shapiro B, Savage PJ, Lomatch D, et al. A comparison of accuracy and estimated cost of methods for home blood glucose monitoring. Diabetes Care 1981;4: Weiner CP, Faustich M, Burns J, Fraser L, Whitaker L, KJugman M. The relationship between capillary and venous glucose concentration during pregnancy. Am J Obstet Gynecol 1986; 155: Jackson RA, Peters N, Advani U, et al. Forearm glucose uptake during the oral glucose tolerance test in normal subjects. Diabetes 1973;22: Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: Godine JE, Hurxthal K, Nathan DM. Bedside capillary glucose measurement by staff nurses in a general hospital. Am J Med 1986;80: Aziz S, Hsiang YH. Comparative study of home blood glucose monitoring devices: Visidex, Chemstrip bg, Glucometer, and Accu- Check bg. Diabetes Care 1983;6: Barr B, Leichter SB, Taylor L. Bedside capillary glucose monitoring in the general hospital. Diabetes Care 1984;7: Brooks KE, Rawal N, Henderson AR. Laboratory assessment of three new monitors of blood glucose: Accu-Check II, Glucometer II, and Glucoscan Clin Chem 1986;32: Clements RS, Keane NA, Kirk KA, Boshell BR. Comparison of various methods for rapid glucose estimation. Diabetes Care 1981;4: Landon MB, Cembrowski GS, Gabbe SG. Capillary blood glucose screening for gestational diabetes: a preliminary investigation. Am J Obstet Gynecol 1986;155: Cohen FE, Sater B, Feingold KR. Potential danger of extending SMBG techniques to hospital wards. Diabetes Care 1986;9: : 22. Langer O, Mazze RS. Diabetes in pregnancy: evaluating self-monitoring performance and glycemic control with memory-based reflectance meters. Am J Obstet Gynecol 1986;155: Eriksson KF, Fex G, Trell E. Capillary-venous differences in blood glucose values during oral glucose tolerance test. Clin Chem 1983;29:993. A.J.C.P. February 1991