Impact of Integrating Rumke Statistics to Assist with Choosing Between Automated Hematology Analyzer Differentials vs Manual Differentials

Transcription

1 Impact of Integrating Rumke Statistics to Assist with Choosing Between Automated Hematology Analyzer Differentials vs Manual Differentials Laura Stephens, 1 Wendy Hintz-Prunty, 1 Hans-Inge Bengtsson, 2 James A. Proudfoot, 3 Sandip Pravin Patel, 4,5 and H. Elizabeth Broome 1,4 * Background: To optimize precision of nucleated blood cell counting, the clinical laboratory scientist should post the automated differential rather than the manual differential if the former is within the 95% CI of the latter, as determined by the Rumke statistic. The objective of this study was to determine the potential impact of real-time, computerassisted use of Rumke statistics for more judicious use of the automated vs digitally imaged, manual differential. Methods: Complete blood counts with automated differentials produced by a XE5000 hematology analyzer (Sysmex) were compared with both the DM96 (CellaVision AB) preclassification differentials and the posted reclassified manual differentials, using the Rumke 95% CIs as calculated using the Clopper-Pearson method. Results: A total of 44.7% of manual differentials had no statistical or clinical justification over the automated differential. In addition, 31.1% of manual differentials had statistical discrepancies between the instrument absolute neutrophil count (IANC) and manual absolute neutrophil count (ANC). Nineteen of these IANC/manual ANC discrepant samples had ANCs below 1500/μL, a decision level for cancer treatment. Holding the IANC when it is less than 2000/μL until after manual smear review would have prevented the posting of any IANC vs manual ANC discrepant results at the 1500/μL ANC decision threshold. Conclusions: A real-time operator alert concerning the statistical identity of imaging device differentials vs automated differentials could have reduced manual differentials by nearly 45%. Not posting unnecessary manual differentials for the cases with IANC/manual ANC discrepancies would have likely reduced clinical error/confusion. IMPACT STATEMENT This study evaluates the potential impact of real-time, computer-assisted statistical analysis comparing autodifferentials and differential cell counts using blood smear imaging systems such as the DM96. The findings support the conclusion that real-time operator alerts concerning the statistical identity of the imaging device differential vs the hematology analyzer differential could have reduced reclassification and posting of manual differentials by nearly 45%, saving costly labor. Also, not posting unnecessary manual differentials for cases with hematology analyzer ANC/manual ANC discrepancies would have reduced clinical error/confusion for laboratories that post instrument ANCs before smear review, an increasingly common practice in cancer treatment centers. 1 Department of Pathology, University of California, San Diego, San Diego, CA; 2 CellaVision AB, Ideon Science Park, Lund, Sweden; 3 Department of Clinical and Translational Research Institute, University of California, San Diego, San Diego, CA; 4 Moores Cancer Center, University of California, San Diego, San Diego, CA; 5 Department of Medicine, University of California, San Diego, San Diego, CA.... January : JALM 357

2 Rumke Statistics and Differential Choice Modern hematology analyzers reliably measure blood cell counts and the nucleated cell differential including nucleated red cells and immature granulocytes (1). Through algorithms using various measurements, these analyzers also alert the operator to the possible presence of abnormal cells. To investigate these flagged samples, a clinical laboratory scientist (CLS) 6 usually examines a smear made from the blood to determine if there are any abnormal cells. If any abnormal cells are present, the morphology of these cells on the stained smear may allow more precise classification and quantification using the manual differential instead of the automated differential from the hematology analyzer. This blood smear microscopic review process can be semi-automated with the use of slide-maker-stainers and imaging systems such as the DM96 (CellaVision AB). The DM96 scans blood smears at low power to identify nucleated cells and then takes digital images at high magnification. These images are arranged in a preclassification display using a neural network image recognition system (2). The CLS reviews these preclassified cells and decides whether the cells are classified appropriately. If not, the CLS can reclassify the cells through a drag and drop computer mechanism. Along with examining the nucleated cells, the DM96 provides images of appropriate areas of the blood smear to perform erythrocyte morphology and platelet estimates. These results together with either the manual, digitally imaged differential from the DM96 or the automated differential from the hematology analyzer can be posted directly from the DM96 depending on the clinical laboratory workflow (Fig. 1). Use of the DM96 increases manual differential efficiency (3). However, even with automated Fig. 1. UCSD Clinical Hematology Laboratory workflow for leukocyte differential posting. CBC plus autodifferential results from the Sysmex XE5000 are analyzed by the Sysmex WAM hematology middleware to determine which results are autoverified or held for review before posting. Most samples held for review have smears automatically made using the SP1000i slide-maker-stainer. Those smears are imaged using the CellaVision DM96, and the CLS decides whether to post the autodifferential or perform a manual differential after reviewing the DM96 images, previous results, and XE5000 results (flags and histograms). The manual differential is performed by drag and drop reclassification of the preclassified images in the DM96. The UCSD Clinical Laboratory procedure states that the CLS should choose the automated differential over the manual differential if the former is within the 95% CI (Rumke statistic) of the latter. This report evaluates how automating the calculation and presentation of those statistics would affect the posting of manual differentials over automated differentials. imaging and neural network preclassification, the operator must decide whether to spend time reclassifying cells that are not preclassified correctly and whether to post the manual differential vs the automated differential. One reason to choose the *Address correspondence to this author at: Moores UCSD Cancer Center, 3855 Health Sciences Dr., La Jolla, CA Fax ; ebroome@ucsd.edu. DOI: /jalm American Association for Clinical Chemistry 6 Nonstandard abbreviations: CLS, clinical laboratory scientist; IANC, instrument absolute neutrophil count; ANC, absolute neutrophil count; CBC, complete blood count; UCSD, University of California, San Diego; WBC, white blood cell; NRBC, nucleated red blood cell JALM :04 January 2017

3 Rumke Statistics and Differential Choice ARTICLES automated differentials over the manual differentials is that the automated differentials are more precise, since they usually are based on over cells vs 100 cells for the manual differential. A more important reason to limit the posting of unnecessary manual differentials is to reduce the potential for discrepancy between automated instrument absolute neutrophil count (IANC) and absolute neutrophil count (ANC) from the manual differential when the IANC is released with the blood counts before posting of the complete differential. This practice is increasingly common in many laboratories serving cancer treatment centers (4 6). For these reasons, our institutional hematology laboratory procedure states the CLS should post the automated differential rather than the manual differential if the former is within the 95% CI of the latter, as determined by the Rumke statistic (7). However, without automated availability, applying this statistic in real-time laboratory practice is impractical. We hypothesized that real-time use of the Rumke statistic would reduce both unnecessary reclassification of cells in the CellaVision DM96 and unnecessary posting of reclassified manual differentials over automated differentials, thereby reducing discrepancies between the IANCs and the manual ANCs. MATERIALS AND METHODS All samples were from patients who had complete blood count (CBC) plus differentials ordered by clinicians at the University of California, San Diego (UCSD), Moores Cancer Center, a tertiary care academic cancer center serving various oncology clinics, a marrow transplant clinic, and an infusion center. These samples were collected over a 1-week period in September 2015 and analyzed at the UCSD Health System La Jolla Clinical Laboratory, with the workflow shown in Fig. 1. Patients had an average age of 58 years (range years) with a male-to-female ratio of 1.2:1. Data analyzed included CBCs with automated differentials produced by a Sysmex XE5000 (Sysmex), DM96 (CellaVision AB) preclassification differentials, and the posted manual differentials. The posted manual differentials were done using the DM96 with an average of 113 cells ± 11 (SD) with a range of Two outliers not included in the SD and range had cell counts of 178 and 229 due to counting duplicate slides and excessive smudge cells, respectively. Four slides had counts below 100 (9, 16, 76, and 78) due to low white blood cells (WBCs) (<0.1, 0.1, 0.2, and /μl, respectively). All blood smears were made using the SP1000i (Sysmex) automated smear-maker-stainer. The Institutional Review Board in accordance with the ethical standards established by UCSD approved procedures for deidentified patient data collection. Data analysis was performed using Microsoft Excel software. Rumke 95% CIs were calculated using the Clopper-Pearson method, reexpressing the binomial probabilities as quantiles from a β distribution with parameters, which are a function of the sample size, number of successes, and significance level. The open source statistical programming language R ( was used to automate the calculation of these intervals. The CIs for the percentage of manual reclassified differentials that were unnecessary (Fig. 2) were determined by treating the inclusion or exclusion into that 44.7% as a binary variable and computing the intervals based on a normal approximation. Additional analysis was also performed on samples with ANCs of approximately 1500/μL, a threshold for most cancer treatment decisions that depend on neutrophil counts. RESULTS Of 1004 CBCs with differentials posted for patients at the UCSD Moores Cancer Center over a 1-week period, 211 (21.0%) had manual differentials performed and posted. Of the samples with autodifferentials posted, 75% were autoverified using rules validated in WAM, the Sysmex... January : JALM 359

4 Rumke Statistics and Differential Choice Fig. 2. Samples whose autodifferentials (autodiffs) were within and outside of the 95% CI of the DM96 preclassification differentials (Diffs). Shaded boxes represent cases where a manual smear review was not necessary/appropriate. Percentages represent cases out of the total 199 manual differentials analyzed in this study. hematology middleware. Supplemental Tables 1 and 2 list the UCSD criteria for autoverification and for smear review, respectively (see Tables 1 and 2 in the Data Supplement that accompanies the online version of this article at content/vol1/issue4). Table 1 lists the XE5000 flagging for these manual differentials. Twelve of these manual differentials actually were performed at satellite laboratories, leaving 199 manual differentials with complete data for our analyses (Fig. 2). Over 99% of the manual differentials at the UCSD Moores Cancer Center clinical laboratory during the study time frame were performed using the DM96 imaging system. Of the 199 manual differentials, 92 (46.2%) of the CellaVision preclassification differentials had 95% CIs that included the autodifferentials (Fig. 2). Although these hematology analyzer and DM96 preclassification results were not statistically different, manual differentials were still performed and posted. This resulted in reclassified manual differentials that for 84 (42.2% of manual differentials) had 95% CI that still contained the autodifferentials. Eight of the 199 (4.0%) manual differentials had DM96 preclassification results with 95% CIs that contained the autodifferential but had reclassification posted manual results with 95% CIs that no longer contained the autodifferential. For these 8 samples, Table 2 shows the reasons for the autodifferential changing from within 95% CIs of the DM96 preclassification differential to outside of the manual reclassified differential. Of these 8 cases, expert review (by H. Elizabeth Broome, MD) revealed that 3 required manual differentials due to obvious issues, JALM :04 January 2017

5 Rumke Statistics and Differential Choice ARTICLES Table 1. Sysmex XE5000 instrument flags for the 199 samples with manual differentials analyzed in this study. WBCs Platelets RBCs Instrument flag Number of samples flagged WBC Abn a scattergram 13 NRBC Abn scattergram 1 Immature Gran? 10 Blast? 34 Atypical lympho? 6 Left shift? 35 Abn lympho/l_blasts? 18 RBC lyse resistance? 1 Abn scattergram 0 Clumps? 2 Abn distribution 43 RBC Abn distribution 5 Dimorphic population 5 RBC agglutination 0 Fragments? 24 RET Abn scattergram 1 a Abn, abnormal; Gran, granulocyte; lympho, lymphocyte; RBC, red blood cell; RET, reticulocyte. specifically basophils preclassified as monocytes, lymphocytes preclassified as blasts, neutrophils preclassified as monocytes, and nucleated red blood cells (NRBCs) preclassified as lymphocytes. The remaining 5 of 8 cases did not require posting of manual differentials, since expert review revealed no clear reason not to post the autodifferential after smear review. If the CLS had the Rumke 95% CI available in the CellaVision preclassification screen and had not noted any questionable cells, then the autodifferential could have been accepted without further delay. Thus, use of CellaVision preclassification screening could have avoided 44.7% of the reclassifications, that is, 84 cases (42.2% of the 199 manual differentials) whose manual differentials had 95% CI that still contained the autodifferentials, plus 5 cases (2.5%) that did not require posting of manual differentials per expert review (Fig. 2). A total of 107 of the 199 manual differentials (53.8%) had DM96 preclassification differentials with 95% CIs that did not include the autodifferential; 97 of these 107 manual differentials (90.7%; 48.7% of the total 199 manual differentials) had 95% CIs that included the DM96 preclassification differential, revealing superior performance of the DM96 preclassification differential compared to the autodifferential. Ten samples (107 minus 97) had manual differential 95% CIs that did not include the autodifferential or the DM96 preclassification differential. Two of these 10 had a very low WBC count (<200/μL) with very few cells identified by the DM96. Seven of the 10 had highly abnormal blood smears that would challenge a hematology expert including blasts, dysplastic hyposegmented neutrophils, large granular lymphocytes with prominent granules, prolymphocytes, and immature monocytes associated with recovery from cytotoxic treatment. Only 3 of the 10 samples had cells that the DM96 preclassified incorrectly for unclear reasons, and 2 of those 3 were from the same patient and showed somewhat hypogranular neutrophils that the DM96 preclassified as monocytes. Sixty-two of the 199 manual differentials (31.1%) had IANC-ANC statistical discrepancies, defined as the autodifferential neutrophils outside of the 95% CI of the sum of the manual bands and segmental neutrophils. Of these 62 cases with IANC-ANC discrepancies, 18 (29.0%) had ANCs between 100 and 1500/μL, a decision level for cancer treatments but above when smears have too few cells for a differential. Among these 18 cases, the IANC was less than the ANC for all but 1 sample, and that sample was from a patient with chronic lymphocytic leukemia and a WBC count of over 70000/μL with 98% lymphocytes by manual differential. With this type of sample, a small change in neutrophil percentage greatly affects the absolute count. No XE5000 flags were predictive of an IANC-ANC discrepancy until the IANC-ANC discrepancy was over 50%, a threshold previously reported to be predicted by the blast, monocytosis, and NRBC present flags (5). Three of the 18 samples with IANC-ANC discrepancies and an IANC <1500/μL had over... January : JALM 361

6 Rumke Statistics and Differential Choice Table 2. Reasons for the autodifferential changing from within the 95% CI of the DM96 preclassification differential to outside of the manual reclassified differential. Case Discrepant category after CLS reclassification Expert review findings Manual differential indicated? 1 Monocytes 5 blasts reclassified as lymphocytes; 2 monocytes and 1 unclassified reclassified as basophils. 2 Neutrophils and monocytes 3 Neutrophils and lymphocytes 1 artifact; 8 monocytes reclassified as neutrophils. 7 lymphocytes reclassified as NRBC. Yes 4 Neutrophils After reclassification, neutrophils increased from 52% to 58%. 5 Neutrophils After reclassification, neutrophils increased from 44% to 47%. 6 Neutrophils After reclassification, neutrophils increased from 90% to 91%. 7 Neutrophils After reclassification, neutrophils increased from 39% to 41%. 8 Neutrophils, lymphocytes After reclassification, neutrophils increased from 20% to 33% and lymphocytes decreased from 80% to 67%. Yes, final differential had no monocytes and 4% basophils. Yes, not clear why DM96 preclassified neutrophils as monocytes. No No No No No. WBC <100/μL; only 57 cells in the differential count. 50% difference, and all 3 had blast, monocytosis, and NRBC present flags. A more clinically relevant threshold of significance for the IANC-ANC discrepancy than the somewhat arbitrary 50% difference would be if the discrepancy affects the anticancer therapy dosing decisions. There are 2 types of dosing errors that may result from IANC-ANC discrepancies: (a) withholding/reducing the dose of therapy unnecessarily and (b) giving treatment inappropriately when the ANC is below 1500/μL. Using a cutoff of IANC <2000/μL for holding the IANC reporting until manual smear review would have resulted in 0 inappropriate infusions and potentially avoided 8 unnecessary dose reductions/therapeutic holds for patients who had manual differentials. A total of 138 of the 1004 (13.7%) CBCs plus differentials ordered over the week had IANC <2000/μL. DISCUSSION The CellaVision DM96 instrument reliably captures images of nucleated cells on blood smears and accurately preclassifies the nucleated cells using a neural network and image recognition (3, 8). The instrument also images areas of the smear for red cell morphology assessment and platelet count estimation. These capabilities streamline the workflow for CLS reviewing blood smears. However, the CLS still must decide whether to accept the autodifferential from the hematology analyzer or to reclassify cells and post a manual differential. Information important to making this decision is whether the autodifferential is within the 95% CI of the preclassified differential generated by the CellaVision DM96 or the 95% CI of the reclassified manual differential. Currently, that information is not readily available. Our findings support providing these 95% CIs to help make the best decision. Automated, real-time operator alerts concerning the statistical identity of autodifferentials and DM96 preclassified/reclassified differentials could have reduced manual differentials by 44.7%. Such alerts could have saved time and reduced clinical error or confusion, particularly for samples with an ANC below JALM :04 January 2017

7 Rumke Statistics and Differential Choice ARTICLES 1500/μL that also had IANC vs manual ANC discrepancies. If there are no abnormal cells and there is no statistical difference between the manual and autodifferential, then for optimal precision, the CLS should post the autodifferential in preference to the manual differential (9, 10). The reference method for the neutrophil count is by counting 400 leukocytes by microscopic examination in 2 blood smears (11). However, in samples with a very low ANC, there are insufficient leukocytes on blood smears to count 100 cells, let alone 400 cells. This means that the manual count is unlikely to be as precise as the autodifferentials at very low neutrophil counts, since hematology analyzer autodifferentials are based on over nucleated cells. Another reason to post the IANC when there is no statistical or clinical reason not to post the IANC is to reduce the potential for discrepancy between the ANC and the IANC that clinical laboratories often post with the CBC before posting the differential (5, 12). This IANC-ANC discrepancy has the potential for unnecessarily withholding/reducing or for inappropriately giving anticancer doses that depend on a 1500/μL ANC threshold. Previous reports have recommended using blast, monocytosis, and NRBC present flags from the XE5000 to predict if the IANC-ANC discrepancy would be over 50% (5). That threshold is arbitrary. We propose that a better determination of significant difference would be if the IANC were outside of the 95% CI for the ANC. Using that measure of significant difference, we found that none of the analyzer flags was predictive. Therefore, based on the analyses presented in this report, we have changed our policy at the UCSD Clinical Laboratories for reporting the IANC with the CBC when the differential is held for smear review. Previously, we reported the IANC with the CBC regardless. Currently, we hold the IANC until after smear review if the IANC is <2000/μL. The preclassification accuracy of the DM96 is critical to the success of using the 95% CI in the laboratory information system to help the CLS decide between posting the manual or autodifferential. Our results support the conclusion that for samples requiring manual differentials, the DM96 preclassification differentials are closer to the posted manual differentials than the autodifferentials. For preclassification differentials with 95% CIs that did not include the autodifferential, 90.7% had reclassified manual differentials that did include the DM96 preclassification differential. What would be the best way to display the 95% CIs? Currently, 95% CIs are not available either in our hematology middleware or with the DM96 imaging system. Our vision is that the DM96 or a similar blood smear-imaging device would post the 95% CIs next to each of the nucleated cell categories on the instrument display. These 95% CIs could be updated as the CLS reclassifies various images through the drag and drop mechanism. Once the CLS accepts that the nucleated cells are all in their correct category, a glance at the 95% CIs would be of critical assistance in deciding whether to post the manual or automated differential. The main reasons to choose the manual differential would be if the automated differential is outside of the 95% CIs of the manual differential or if there are abnormal cells to report that are not listed in the automated differential. Otherwise, choosing the autodifferential over the manual differential will improve precision while reducing the potential for any discrepancy between the IANC previously posted with the blood counts before posting the complete differential.... January : JALM 363

8 Rumke Statistics and Differential Choice Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Authors Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Employment or Leadership: H.-I. Bengtsson, CellaVision AB. Consultant or Advisory Role: H.E. Broome, CellaVision AB. Stock Ownership: H.-I. Bengtsson, CellaVision AB. Honoraria: H.E. Broome, Sysmex. Research Funding: H.E. Broome, CellaVision AB; this work was supported by the University of California, San Diego Clinical Laboratories, as a quality assurance project. Expert Testimony: None declared. Patents: None declared. Role of Sponsor: No sponsor was declared. Acknowledgments: This work was presented in part as a poster in May 2015 at the International Society of Laboratory Medicine meeting in Chicago, IL. REFERENCES 1. Meintker L, Ringwald J, Rauh M, Krause SW. Comparison of automated differential blood cell counts from Abbott Sapphire. Siemens Advia 120, Beckman Coulter DxH 800, and Sysmex XE-2100 in normal and pathologic samples. Am J Clin Pathol 2013;139: Lee LH, Mansoor A, Wood B, Nelson H, Higa D, Naugler C. Performance of CellaVision DM96 in leukocyte classification. J Pathol Inform 2013;4: Briggs C, Longair I, Slavik M, Thwaite K, Mills R, Thavaraja V, et al. Can automated blood film analysis replace the manual differential? An evaluation of the CellaVision DM96 automated image analysis system. Int J Lab Hematol 2009;31: Amundsen EK, Urdal P, Hagve TA, Holthe MR, Henriksson CE. Absolute neutrophil counts from automated hematology instruments are accurate and precise even at very low levels. Am J Clin Pathol 2012;137: Sireci AN, Herlitz L, Lee K, Bautista JL, Kratz A. Validation and implementation of an algorithm for reporting the automated absolute neutrophil count from selected flagged specimens. Am J Clin Pathol 2010;134: Lantis KL, Harris RJ, Davis G, Renner N, Finn WG. Elimination of instrument-driven reflex manual differential leukocyte counts: optimization of manual blood smear review criteria in a high-volume automated hematology laboratory. Am J Clin Pathol 2003;119: Rumke C. The statistically expected variability in differential counting. In: Koepke J, ed. CAP Conference. Vol Aspen, CO: College of American Pathologists; p Kratz A, Bengtsson HI, Casey JE, Keefe JM, Beatrice GH, Grzybek DY, et al. Performance evaluation of the CellaVision DM96 system: WBC differentials by automated digital image analysis supported by an artificial neural network. Am J Clin Pathol 2005;124: Siekmeier R, Bierlich A, Jaross W. The white blood cell differential: three methods compared. Clin Chem Lab Med 2001;39: Ruzicka K, Veitl M, Thalhammer-Scherrer R, Schwarzinger I. The new hematology analyzer Sysmex XE-2100: performance evaluation of a novel white blood cell differential technology. Arch Pathol Lab Med 2001;125: Koepke JA, Van Assendelft OW, Brindza LJ, Davis BH, Fernandes BJ, Gewirtz AS, et al. Reference leukocyte (WBC) differential count (proportional) and evaluation of instrumental methods: approved standard. Vol. H20 A2: Clinical Laboratory Standards Institute, Jatoi A, Jaromin R, Jennings L, Nguyen PL. Using the absolute neutrophil count as a stand-alone test in a hematology/oncology clinic: an abbreviated test can be preferable. Clin Lab Manage Rev 1998;12: JALM :04 January 2017