THE "AVERAGE OF NORMALS" METHOD OF QUALITY CONTROL

Transcription

1 THE AMERICAN JOURNAL OF CLINICAL PATHOLOGY Copyright 1965 by The Williams & Wilkins Co. Vol. 43, No. 2 Printed in U.S.A. THE "AVERAGE OF NORMALS" METHOD OF QUALITY CONTROL ROBERT G. HOFFMANN, PH.D., AND M. E. WAID, M.D. % Computing Center, University of Florida, Gainesville, and Department of Pathology, Wuesthoff Hospital, Rockledge, Florida A new method of laboratory quality control is described in this paper which is simple, sensitive, and requires only a little clerical work for its use. It is based on the use of certain patients' specimens, but it is practical for small laboratories because few tests are needed. It also has the desirable feature of providing information that is useful in other ways than for quality control. The routine work required for use of the method is so simple that it can be described in the following 2 sentences: 1. At the end of each day, average the patient's test values that fall within the normal range. 2. Plot this "average of normals" on a control chart. This is all there is to the procedure, and as will be demonstrated, it provides useful information. The basis of the method is described with 2 examples from clinical laboratories. BASIS OF THE "AVERAGE OF NORMALS" QUALITY CONTROL METHOD A biological basis of the method is described here. It can be seen by laboratory personnel simply by scanning their own records. Study of a "laboratory log" for virtually any test will reveal that: (1) a substantial portion of tests of a patient's specimens falls within the normal range, and (2) the normal range is about as stable as anything in medicine can be. These are the biological facts on which the method is based. The procedures described here are designed to make the best use of them for laboratory control and other purposes. In the examples about to be presented, Received, July 31, 1964; revision received, November 6; accepted for publication November 6. Dr. Hoffmann was formerly Research Assistant Professor, University of Florida. Dr. Waid is Consultant in Pathology, Wuesthoff Hospital, Rockledge, Florida. 134 an electronic computer (IBM 709) was used for establishing normal ranges, but it is not necessary for any user of the method to have access to one. Computing can be done for a laboratory, however, regardless of its size or location, if it is desired. (For further information about computing services, write to the authors.) Figure 1 illustrates frequency distributions for 600 consecutively determined patients' tests for blood urea nitrogen (BUN) and fasting blood glucose (FBG). The test results were simply copied from laboratory records. Note that everything said to this point can be verified by study of the distributions in Figure 1. A large proportion of tests do fall within the normal range. The normal ranges shown were actually computed from the distributions of patients' values, 1 ' 2 as indicated by the Gaussian curves indicated as dashed lines on the charts. Laboratories need not go through this process, but it is important that the midpoint of the normal range used coincide with a reasonable estimate of the true highest point (mode) of the distribution of patients' values. A study of the distributions in the figures should make this clear. For instance, the midpoint of the normal range for glucose (Fig. 1) is 77.5 mg. and the range is 58 to 97 mg. From the statistical point of view, the method is based on the most frequently reported tests. Satisfactory results for this laboratory would not be obtained if a normal range of 70 to 120 mg. were used. COMPUTING THE AVERAGES OF NORMALS The daily routine of the control process consists of computing the averages of patients' values which fall within the normal range and plotting the averages on a chart. For instance, suppose the following results were obtained from a morning group of fasting blood glucose tests. Postprandial

2 Feb THE "AVERAGE OF NORMALS" METHOD B.U.N Mean Gaussian S.D. 4.2 mg/iooml.a> XI E Z IUU mg/ioo ml GLUCOSE A t \ i > / \ " /. NORMAL \ 1 IV V RANGE C / \ 1 1 ^ ^ r i i Gaussian Mean S.D mg/ioo ml -^*ft-p»» i i i i ma/ion ml FIG. 1. Frequency distribution of BUN test values of 600 patients (upper). Frequency distribution of fasting blood glucose test values of 000 patients (lower). In both charts the dashed curve is Gaussian, and the normal range was determined by adding and subtracting 2 standard deviations to the mean. or glucose tolerance tests are not included and would be ignored because thej r have different normal values In this laboratory, the normal range is 58 to 97 mg., so values of 55 or less and 100 or greater will be ignored. Keeping exactly to the normal range is not necessary; just keep close to it. It should be emphasized that these "cut off" points of 55 and 100 need be established only once, but after they are established they should be rigidly adhered to when averages of normals are computed. The dash following each test above indicates it is to be used in the average. There are 10 tests within the normal range, and their average is 80.5 mg. This average would be plotted on a control chart similar to those in Figure 2. Figure 2 illustrates "average of normals" control charts for glucose from 2 laboratories. A total of 600 patients' tests were obtained from each laboratory, and 10 tests were averaged for each point on each chart. The dashed lines are 95 per cent confidence limits. The solid horizontal line in the center of the chart is the midpoint of the normal range. The upper chart indicates a fairly stable testing procedure and the lower chart an unstable testing procedure. The purpose of a control chart is to detect

3 136 HOFFMANN AND WAID Vol.43 0> O 0 3 O <D tj> O k_ a> > < 90 Hh 80 7ft 70 GLUCOSE (stable procedure) r GLUCOSE (unstable procedure) O 0) k- <D < BO -M»_._»-. 75 L Date of Average FIG. 2. "Average of normals" control charts for fasting blood glucose. Each point on each chart is the average of 10 patients' tests which fall within the normal range. The dashed lines are 95 per cent confidence limits. The upper chart indicates a fairly stable testing procedure, and the lower chart, instability. trouble at the time it occurs, or as soon as possible afterwards. Averages should be plotted on the chart immediately after testing a patient's specimens. If a point falls outside the limits, an investigation should be made before reporting the patients' test results. With 95 per cent confidence limits, however, an occasional point outside the limits is to be expected with a stable procedure. Even normal values vary. But 2 successive points outside the same limit are an almost certain indication of trouble. Sources of the trouble encountered by the unstable laboratory (Fig. 2, lower) are known and will be discussed later. The point to be stressed here is that these charts give useful, not misleading, information. It was stated earlier that this control method can be used even by small laboratories. The averages in Figure 2 were computed by an electronic computer, using 10 tests for each average, and data were obtained from laboratory records. On a day-today basis, the number of normal tests will vary and, of course, will depend on the size of the laboratory. The confidence limits are determined partially by the number of tests included in each average, so "variable" confidence limits are needed. The simplest way to obtain them is to have them computed with an electronic computer,* but satisfactory limits can probably be obtained from * Limits are computed using the "within groups of normals" as the variation source for the standard deviation, from which the standard deviation of the average of normals is computed.

4 Feb THE "AVERAGE OF NORMALS" METHOD 137 the following procedure: 1. Estimate the "standard deviation of normals" by dividing the normal range by 4. This should be a realistic normal range verified by hand tabulating a frequency distribution of a patient's values, similar to those in Figure Determine the "standard deviation of averages of normals" by dividing the standard deviation obtained in step 1, above, by the square root of the number of tests in an average of normals group. 3. Multiply the value obtained in step 2 by Add the result obtained in step 3 to the midpoint of the normal range. This will yield the upper 95 per cent confidence limit. Subtracting from the midpoint of the normal range yields the lower confidence limit. For instance, suppose a laboratory is testing 5 to 10 "normal" specimens each day (or every 2 or 3 days). Its normal range for fasting blood glucose is 65 to 115 mg., so the midpoint of the normal range is 90 mg. Then: 1. (115-65)/4 = 12.5 mg. 2. (a) 12.5/V5 = 5.59 for 5 tests in an average (b) 12.5/VlO = 3.95 for 10 tests in an average 3. (a) (5.59)(1.96) = 11.0 mg. (b) (3.95) (1.96) = 7.7 mg. 4. (a) 90.0 ± 11.0 Upper = Lower = 79.0 (b) 90.0 =fc 7.7 Upper = 97.7 Lower = 82.3 The 2 sets of limits would be plotted on the control chart. Note that limits for 10 tests would be approximately 7 mg. narrower than for 5 tests. Common sense indicates that more variation is to be expected when few tests are available as is indicated by the limits. On any given day, the number of tests would be known, so no trouble would be encountered in estimating the proper limit if it was needed. As a matter of record, however, it might be also helpful to make a small chart which indicates the number of tests included in each average. If fewer than 5 tests are available on any given day, these values should be carried over and included in the next day's average. Figure 3 illustrates control charts for BUN, with 10 tests included in each average. Data from 2 laboratories are again illustrated, with the upper chart presenting the appearance of a stable laboratory testing procedure and the lower chart an unstable procedure. Sources of difficulties are known and will be described later. CHARTS AS A MEANS OF INFORMATION CONVEYING The charts in the figures of this paper are useful for purposes in addition to laboratory quality control. The frequency distributions in Figure 1 are concise means of conveying information about a given laboratory procedure. In several laboratories, these frequency distributions are displayed on a large board just outside the laboratory. Physicians may study them at any time without interrupting laboratory personnel. One pathologist makes a point of explaining their meaning to every new staff physician. They show, as only charts can, the complete situation with regard to a given test. The proportion of normal values reflects medical practice. The level at which normal values are reported reflects laboratory practice. Frequency distributions can be made even more easy to read by preparing them in "standard form." A standard form chart is a frequency distribution similar to those in Figure 1. Instead of being tabulated in arbitrary class intervals, such as the 2 mg. intervals used for BUN (Fig. 1), each class interval is 1 standard deviation of a normal subject's values; i.e., approximately onefourth of the normal range. An example of a standard form chart is illustrated in Figure 4, which includes results of testing 600 patients' specimens for chloride. The scale at the bottom of the frequency distribution is in meq. per 1., and each class interval is 1 standard deviation of "normals" computed from the Gaussian curve indicated by the dashed line. The scale at the top of the frequency distribution is a "normal quotient" scale, a normal quotient scale being one in which the midpoint of the normal range is always 100 and the normal range is 90 to 110 N/Q units. The shaded band is the normal

5 138 HOFFMANN AND WAID Vol. 43 B.U.N, (stable procedure) ID CD <U en o t- 0) > < r B.U.N, (unstable procedure) z> m <u en o k_ <u > < Date of Average FIG. 3. "Average of normals" control charts for BUN. Each point on each chart is the average of 10 patients' tests which fall within the normal range. The dashed lines are 95 per cent confidence limits. The upper chart indicates a stable testing procedure, and the lower chart, instability. range. Any group of medical measurements, where the distribution of healthy subjects is Gaussian, can be displayed by means of a standard form chart. Other examples and uses of normal quotient units may be found. 1-4 The control chart in the lower portion of Figure 4 is an average of normals chart for the 600 chloride values and includes 10 normal tests in each control point. No shift is apparent, but the testing procedure is a bit erratic. Too many points are below the lower control limit. The control charts illustrated in Figures 2, 3, and 4 are also useful in ways other than quality control. Inasmuch as they are based on the clinically normal range, they convey information about the normal range as it is continuously being estimated in the laboratory which tests the specimens. The amount of a shift in the testing procedure, if it occurs, can be estimated because the averages are in the same units as the original measurements. A control chart displayed immediately below its frequency distribution (Fig. 4) tells a story as almost nothing else can. The frequency distribution reveals how a test is used over a period of time, the level at which the laboratory is operating, and the normal range. The control chart indicates the stability of the testing procedure. With a stable procedure, the frequency distribution will remain the same. If the procedure has shifted, a new distribution should be prepared if the test is to be maintained at the new level. This is useful information

6 Feb THE "AVERAGE OF NORMALS" METHOD 139 Normal Quotient Scale 103 r CT llj E a> o k. a> > < »i ~» 5" Date of Average Fro. 4. "Standard form" frequency distribution of 600 chloride tests of patients' specimens (upper). The scale at the bottom of the chart is meq. per 1., and the scale at the top is a normal quotient scale. The shaded band represents the normal range. Average of normals control chart for chloride (lower). Each point is the average of 10 normal values. The procedure seems to be stable but erratic. for everyone concerned with the clinical laboratory. Charts are presented and discussed here from the quality control point of view, so "all" patients' results are included in each chart. Similar charts can be prepared for age, sex, diagnosis, treatment, or other specific pertinent factors. This would best be done by pooling information from many laboratories. Technics are available for doing this, so it is only a matter of putting them to use. PROBLEMS DISCLOSED BY USING AVERAGES OF NORMALS CONTROL CHARTS The examples of BUN and glucose shown indicate unstable testing procedures for 2 laboratories. Sources of the difficulties will now be discussed. The rise in glucose values (Fig. 2, lower) was apparently caused by a particular lot of glucose oxidase reagent. Even though it had freshly arrived in the laboratory, rapid development of color even before the filtrate was added indicated deterioration. The

7 140 HOFFMANN AND WAID Vol. 48 problem was corrected by a new lot number of reagent, but not until quite a few tests were reported as erroneously high. Physicians complained about test values during this period. Glucose data from approximately 25 laboratories has been analyzed by the methods described here. Study of them indicates that glucose is a difficult test to control properly. BUN tests can have low values when detergent or other substances interfere with the enzyme action of urease. When Nessler's reagent is not prepared properly, the solution is cloudy, and erroneously high values are the result. Both of these troubles have been seen, but most laboratories maintain control over BUN better than their glucose control. A detailed study of 5 laboratories, using a few tests from each, has been made. In these laboratories, one of us actually visited the laboratory and discussed quality control problems with laboratory personnel after averages of normals charts had been prepared from their own records. In all laboratories studied, commercial reference standards were used daily. It seems as though things are still to be learned about their best use. In 1 laboratory, values from patients were adjusted to an erroneously low commercial reference standard. The company eventually called in the erroneous lot. The average of normals method is not afflicted with such difficulties. A new set of "normal values" is available every day that 5 or more of their specimens are tested APPLICABILITY OF THE METHOD The method of averages of normals has been studied using data from laboratories all over the country. Data from 36 different laboratories has been analyzed for BUN. Study of their distributions and average of normals control charts indicates the problem is much the same for any laboratory regardless of size or location. Considerable variation among normal ranges was seen, and midpoints varied from 10.5 to 18.9 mg. Data from at least 1 laboratory has also been obtained for the following procedures: Prothrombin time C0 2 Sodium Protein bound iodine Spinal fluid sugar Spinal fluid protein Bilirubin (total) White cell count Thymol turbidity Total lipids Uric acid Potassium Alkaline phosphatase Acid phosphatase Serum phosphate Serum glutamic-pyruvic transaminases Serum glutamic-oxalacetic transaminases Hemoglobin Lactic dehydrogenase Serum globulin Serum protein Serum albumin Cholesterol Ammonia Amylase Calcium Hematocrit Chloride Creatinine Lipase In every instance, a high proportion of "normal values" was seen when a frequency distribution was tabulated. Averages of normals control charts prepared from the same data also seem to be reasonable. There is little question as to the usefulness of the method for a wide range of tests. DISCUSSION Laboratory quality control is still a perplexing problem to many directors of clinical laboratories, despite the number of papers written about it. A good bibliography and review of the problem is reported by Young. 6 The control method described here takes a trivial amount of time each day and anyone can be trained to compute and plot the averages. In other words, the routine of the process is so simple and straightforward that it can be done mechanically. But this is not quality control. Quality control is the control of the laboratory testing procedures. The charts should be reviewed constantly by the laboratory's most expert personnel. Here is how this might best be done: 1. Persons expert in the use of adding or calculating machines should be given the responsibility of keeping up the charts. As long as the control points stay within the confidence limits, no "alarm" is sounded. When a point goes outside the limits, the matter should immediately be brought to the attention of the laboratory director or other responsible, knowledgeable person. 2. Periodically, the laboratory director

8 Feb THE "AVERAGE OF NORMALS" METHOD 141 should review the control charts regardless of whether or not any difficulties have been brought to his attention. An abrupt shift, for example, caused by use of a new lot of a reagent will be detected by a control point going outside the confidence limits. A gradual shift, such as the deterioration of a reagent or part of an instrument might best be seen with periodic review of the chart, although it eventually will be detected as points go outside the limits. Maintaining an accurate and precise testing procedure is quality control. The charts help achieve this goal and reflect the success (or failure) of the process. Periodic review of all control charts can lead to the correction of problems that otherwise might not be detected as quickly. For instance, it was noticed in one laboratory that values of sodium as well as potassium were a bit low. A quick check of the flame photometer revealed that soot had accumulated in the optical path. Intervals between periodic cleaning of the instrument had been too long. This is an example of real quality control detection and correction of a problem before it causes major trouble. Quality control in clinical laboratories is the control of measuring processes, and no measuring process is perfect. Variability is an inescapable part of it. This is what seems to give laboratory personnel the greatest difficulty with quality control methods. Statistics, which can be regarded as a body of methods for dealing with problems in which variability is encountered, are not given much attention in the training of laboratory personnel. Quality control methods, and the statistics on which they are based, should be helpful as a portion of the training programs. Most laboratory tests are used solely for the purpose of estimating a portion of the condition of a patient at a given time or over a short period of time. The test results are then discarded. In this paper, a glimpse is given of how some of this information presently being discarded can be put to use for many beneficial purposes in clinical pathology. REFERENCES 1. Hoffman, R. G.: Statistics in the practice of medicine. J. A. M. A. 185: S64-873, 19G3. 2. Hoffman, R. G., Waid, M. E., Todaro, E., and Alston, R.: Retrieving and processing medical measurement data. Rochester Conference on Data Acquisition and Processing in Biology and Medicine, in press, Hoffman, R. G., and Waid, M. E.: A new scale of normal values for physicians. GP, SO: , Hoffman, R. G., and Waid, M. E.: Keeping informed about the clinical laboratory. Clin. Med., in press, Young, D. M.: Experience with and thoughts on quality control. Canad. J. Med. Techn., 25: 2-9, 1963.