The use of scanned, high-resolution, digital images of

Transcription

1 Validating Whole-Slide Imaging for Consultation Diagnoses in Surgical Pathology Thomas W. Bauer, MD, PhD; Renee J. Slaw, MBA Context. The interpretation of scanned whole-slide images (WSI) offers some theoretical advantages for longdistance, consultative diagnosis in surgical pathology. Few WSI validation studies have focused on difficult consultation cases. Objective. To test intraobserver variability of WSI interpretations in cases that had been submitted for consultation using the same hardware and software configuration selected by a client. Design. The 217 cases (approximately 20 nearly consecutive cases received in consultation for each of 11 subspecialty groups) were scanned, uploaded to an imagedistribution application, and interpreted by 26 pathologists who had reviewed the microscope slides an average of 47 days earlier. Independent pathologists identified and classified discrepancies between microscope slide and WSI diagnoses. Results. There were 2 major discrepancies (0.92%) and 8 minor discrepancies (3.7%). One major discrepancy reflected atypical versus nonatypical endometrial hyperplasia; the other related to reactive squamous changes versus carcinoma. Strengths of the study include the large sample size, the many pathologists involved, the degree of difficulty of the cases, and the duplication of scanning and software configuration projected to be used by a client. Although the average 43-day washout exceeds the 2-week interval recommended by an expert panel of the College of American Pathologists, an important limitation in this study was that pathologists commonly remember consultation cases for a long time. Conclusion. Our results help support the safety and efficacy of WSI for surgical pathology diagnoses. (Arch Pathol Lab Med. 2014;138: ; doi: /arpa OA) The use of scanned, high-resolution, digital images of microscope slides ( whole-slide images ; WSI) for the interpretation of consultation cases in surgical pathology offers several potential advantages over conventional workflow. This is especially true for consultation cases in which a client pathologist seeks an opinion from a consultant pathologist at a distant location. Microscope slides at the client site can be scanned relatively quickly, uploaded to a network application along with clinical information, and made available to the consultant pathologist almost instantly. The consultant pathologist can view the WSI and correspondence and submit a report in an electronic format, hopefully reducing turnaround time significantly compared with the use of mail services to deliver Accepted for publication January 24, Published as an Early Online Release May 19, Supplemental digital content is available for this article at www. archivesofpathology.org in the November 2014 table of contents. From the Department of Anatomic Pathology, Cleveland Clinic, Cleveland, Ohio. Dr Bauer was a member of the Medical Advisory Board of Aperio (now Leica Biosystems, Vista, California) during the time this study was performed, but did not receive any personal income from that activity. Aperio loaned Cleveland Clinic an instrument during this study, but no commercial funds were used to support the study. The authors have no relevant financial interest in the products or companies described in this article. Reprints: Thomas W. Bauer, MD, PhD, Department of Anatomic Pathology, Cleveland Clinic, 9500 Euclid Ave, Cleveland, OH ( bauert@ccf.org). microscope slides and printed correspondence. The use of WSI may facilitate distributing and sharing a case among multiple consultant pathologists and can aid in archiving and potential image analysis. There has been increasing acceptance of WSI for surgical pathology diagnoses, as reflected by the recent approval of scanning devices from Omnyx (Pittsburgh, Pennsylvania) 1 and Leica Biosystems (Vista, California) 2 as class II medical devices from Health Canada, along with increasing use elsewhere outside the United States. However, some pathologists and regulatory agencies are concerned about the potential for intraobserver variability between WSI and microscope slide diagnoses, especially when applied to difficult cases. We previously published 3 the results of a large, intraobserver variability ( validation ) study in which we compared diagnoses obtained by interpreting WSI versus microscope slides of previously interpreted primary surgical pathology cases after a minimum 1-year washout period. That study, designed to test the null hypothesis that the true difference in major discrepancies between WSI and microscope slide review is greater than 4%, 3(p518) entailed the review of 607 primary cases of 2 pathologists, and showed that diagnostic review by WSI was not inferior to microscope slide review (P,.001). The study reflected the cases routinely encountered by one pathologist, with mainly orthopedic and gastrointestinal subspecialty interests, and one general surgical pathologist, with a practice similar to that of most community hospital based general surgical pathologists. By design, the study tested intraobserver Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw 1459

2 variability of primary diagnoses and was not enriched with unusually difficult cases. Shortly after our publication, an expert panel developed by the College of American Pathologists (CAP) Pathology and Laboratory Quality Center published guidelines for laboratories to consider when validating WSI for diagnosis. 4 Those guidelines suggested, among other things, that laboratories implementing WSI for diagnostic purposes should perform their own intraobserver variability (validation) studies, and that those studies should be appropriate for the intended clinical use of WSI in the laboratory. Validation studies should involve specimen preparations relevant to the intended use, and if a new intended use is contemplated, and this new use differs materially from the previously validated use, a separate validation for the new use should be performed. 4(p1712) The guidelines noted that the validation study should encompass the entire WSI system, and that revalidation is required whenever a significant change is made to any component of the WSI system. 4(p1712) The panel suggested that the validation process should include at least 60 cases for one application and another 20 cases for each additional application. A washout period of at least 2 weeks was recommended. Our original validation study had the advantages of a large sample size and a control set of cases rereviewed using microscope slides, but was focused on validating WSI for primary diagnosis of routine cases. It could be argued that many of those cases may have been fundamentally easier to interpret than cases received for consultation and that another validation study should be performed to validate WSI for our intended use of receiving digital images (only) for consultation diagnoses. As noted above, the CAP panel recommended that after an initial validation study is performed in a given laboratory, a second study with a sample set of 20 cases should be performed for each additional application. In addition, the CAP panel recommended a new validation study if a significant change is made to any component of the WSI system. 4(1712) Microscope slides for our original validation study had been scanned using a ScanScope XT (Aperio, Vista, California) WSI system, and the images were interpreted using the Aperio ImageScope viewing application. At least one of our clients intends to use a Leica Biosystems, Aperio CS scanner and intends to submit cases using the Leica Biosystems epathaccess application. Both the scanner and viewing applications that our client intends to use are similar to the scanner and viewing applications we used in our original validation study, but with the CAP guidelines in mind, we elected to duplicate as closely as possible the configuration intended to be used by one or more of our consultation clients. Furthermore, instead of limiting our validation to 20 cases, we targeted approximately 20 cases in each of the 11 subspecialty groups we plan to accept for consultation. This study design also provided a relatively large number of pathologists in our department an opportunity to gain experience with actual consultation cases and the image distribution application before implementing the service for patient care. MATERIALS AND METHODS The Cleveland Clinic Department of Anatomic Pathology (Cleveland, Ohio) is a large, academic department composed of more than 40 staff pathologists along with additional residents and fellows. There are 16 subspecialties, and most pathologists limit most of their practice to one or two subspecialty areas. In 2012, the Table 1. Number of Cases and Parts Subspecialty Cases, No. (%) Parts, No. (%) Breast 21 (9.7) 23 (7.6) Cardiovascular 7 (3.2) 13 (4.3) Dermatopathology 22 (10.1) 22 (7.3) Head and neck 21 (9.7) 36 (12.0) Gastrointestinal 25 (11.5) 32 (10.6) Genitourinary 20 (9.2) 40 (13.3) Gynecologic 20 (9.2) 32 (10.6) Hepatobiliary 19 (8.8) 22 (7.3) Neuropathology 20 (9.2) 21 (7.0) Musculoskeletal 22 (10.1) 32 (10.6) Pulmonary 20 (9.2) 28 (9.3) Total 217 (100) 301 (100) department interpreted approximately primary pathology cases, and 4855 cases that were initially diagnosed elsewhere but were reviewed at Cleveland Clinic before a patient was treated at our institution. In addition, 7671 cases were submitted to our department from outside pathologists seeking a consultation diagnosis. The latter type of case was the subject of this study. Sample Size Determination Before starting our previous intraobserver variability study, we consulted an independent statistician to help determine the sample size necessary to demonstrate noninferiority of WSI versus microscope slide interpretation. Based in part on the literature, we assumed the major discrepancy rate between the original diagnosis by microscope slide and the diagnosis by microscope slide review would be 3%. 3 We set the noninferiority margin for WSI review at 4%. A 1-sided, binomial test was used for comparison at a significance level of.05. The power to be achieved was 80%, and the significance level was.05. Based on those assumptions, the statistician calculated that 450 cases (225 per group) would need to be reviewed to establish noninferiority. Before starting the present consultation validation study, we requested an independent statistician recalculate the sample size, based not on data extrapolated from published literature but on actual data available from our first validation study. That revisedpower study calculated that 140 cases might be required to demonstrate noninferiority. Overall Experimental Design At least one of our likely consultation clients intends to use a Leica Biosystems, Aperio CS scanner and intended to transmit images and case information to Cleveland Clinic via the epathaccess (Leica Biosystems) application. For this validation study, a Leica Biosystems, Aperio CS scanner was installed on loan in our department, courtesy of the manufacturer. Two experienced histotechnology/scanning technologists and an administrator/ research coordinator were trained in its use and in the workflow required by the image distribution application (epathaccess, Leica Biosystems). Each pathologist in the department has access to a personal computer with a 24-inch monitor (Dell, Round Rock, Texas). Installation was also facilitated by a dedicated imaging information technology specialist in our section of e-pathology. Approximately 20 consultation cases that were received in 10 of our 11 subspecialty groups (91%) and 7 cases that were received in the 11th subspecialty group (9%; Table 1) were first interpreted for patient care using standard workflow and evaluation of microscope slides. Reports reflecting those interpretations were forwarded to the client pathologists and were used for patient care. Microscope slides from those cases were then scanned at 320 magnification using the Aperio CS scanner and were uploaded to the epathaccess application (Figure 1) along with correspondence from the client pathologists. The cases were then reaccessioned into our laboratory information system with a new reference number, and the clinical information that was originally provided to our pathologists was entered. The working draft from our 1460 Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw

3 Figure 1. Diagram of the workflow for whole-slide imaging in this study, made slightly complicated by the lack of an interface between our laboratory information system (LIS) and the image distribution application. Briefly, consultation cases were scanned and uploaded to epathaccess (Leica Biosystems, San Diego, California) in a way that would duplicate the activities of a client intending to submit a case to Cleveland Clinic (CC) for consultation. The case was then received in our section of e-pathology, assessed for adequacy, assigned a unique surgical electronic consult (SEC) accession number, and distributed to a pathologist who would interpret the case, dictate a report, and sign out the report in our CoPath LIS (Cerner, Kansas City, Missouri). A report was then printed from our LIS, uploaded into epathaccess, and then finalized by the pathologist. laboratory information system was also scanned and uploaded to the application. Our intention was to supply the consultation pathologist with the same clinical information that was available when the microscope slides of the case were originally reviewed. The original microscope slides, along with printed copies of the microscope slide interpretation, were returned to the client pathologists several days after scanning. Although we intended to upload 20 consecutive cases from each subspecialty, the practical logistics of scanning the cases and, at the same time, returning microscope slides promptly to clients meant that, for the busiest subspecialties, the cases accrued into the study were not necessarily consecutive. The decision to enter a case into the study versus to return it directly to the client was based strictly on the backlog of cases waiting to be scanned, not on the diagnosis or complexity of the case. After a washout period, which in most cases exceeded the 2 weeks recommended by the CAP panel, pathologists were notified by that a case was ready to review. Throughout the study, the cases were distributed within each subspecialty based simply on who was assigned to review outside consult cases for the day (rather than being assigned to pathologists based on their own preference for digital versus glass). The pathologists viewed the scanned images and dictated interpretations, again paralleling our microscope slide workflow. In addition, if immunohistochemically stained slides had been included during the initial workup of the case, those slides were also scanned, but the images were not forwarded to the pathologists unless requested. Transcribed reports were uploaded to epathaccess and were verified by the pathologist who then changed the case status to complete, which released the consultation results back to the client pathologist. After approximately 20 cases from each subspecialty had been completed, the diagnoses obtained by each method (microscope slides and WSI) were reviewed by one pathologist (T.W.B.), and potentially discrepant cases were identified. Those cases were then passed on to the directors of each subspecialty service to determine whether the diagnoses were (1) concordant, (2) discordant with a minor discrepancy (one that would not be associated with a change in treatment), or (3) discordant with a major discrepancy (one that might be associated with a change in treatment). Finally, cases determined to be discordant were reassigned to the pathologist who interpreted the case, providing an opportunity for that pathologist to comment on image quality or other factors that may have contributed to the discrepant diagnoses. Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw 1461

4 Figure 2. Box plot illustrating the median, the upper and lower quartiles, and the outliers at washout intervals among each subspecialty group (QI Macros, KnowWare International Inc., Denver, Colorado). The overall mean washout interval was 43 days, and most cases were interpreted with a washout interval longer than 2 weeks. Abbreviations: CARDIO, cardiovascular subspecialties; DERM, dermatologic; ENT, ear, nose, and throat; GI, gastrointestinal; GU, genitourinary; GYN, gynecology; MSK, musculoskeletal; NEURO, neuropathology. As noted above, the original microscope slides were returned to client pathologists shortly after scanning, which is our routine for patient care, so it was not possible to directly compare a microscope slide rereview with a WSI review (as we did in our original validation study of primary diagnoses). Therefore, our discrepancy rates were compared with the results of our previous validation study as well as with published discrepancy rates. Our a priori power study was used to guide our sample-size determination, but without a matched microscope slide rereview control group, no formal statistical evaluation of our results was performed. Cases Versus Parts Calculating discrepancy rates in surgical pathology can be more complicated than simply counting cases. Although some cases consist of a single part, other cases, such as prostate biopsies, might contain 6 or more individual biopsies (parts). The interpretation of any one of those parts could influence patient care, depending on the findings in the remaining biopsies. On the other hand, patientcare decisions are often based on interpreting the combination of all parts together. For this and our previous study, we defined the size of our study conservatively based on cases (not parts). Final discrepancy results were calculated both for diagnoses of the case and for diagnoses of the individual parts. RESULTS A total of 217 cases were interpreted by 26 different pathologists, first by microscope slides and then by WSI. The distribution of those cases according to subspecialty is shown in Table 1. The number of days (washout time) between microscope slide diagnosis and WSI diagnosis varied, but the average was 43 days (Figure 2). Table 2. Discrepancies by Case Subspecialty Major, No. (%) Minor, No. (%) Breast 0 (0.0) 2 (25.0) Cardiovascular 0 (0.0) 0 (0.0) Dermatopathology 0 (0.0) 1 (12.5) Head and neck 1 (50.0) 1 (12.5) Gastrointestinal 0 (0.0) 1 (12.5) Genitourinary 0 (0.0) 0 (0.0) Gynecologic 1 (50.0) 1 (12.5) Hepatobiliary 0 (0.0) 1 (12.5) Neuropathology 0 (0.0) 0 (0.0) Musculoskeletal 0 (0.0) 1 (12.5) Pulmonary 0 (0.0) 0 (0.0) Total 2 (100.0) 8 (100.0) 1462 Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw

5 Table 3. Discrepancies by Part Subspecialty Major, No. (%) Minor, No. (%) Breast 0 (0.0) 2 (22.2) Cardiovascular 0 (0.0) 0 (0.0) Dermatopathology 0 (0.0) 1 (11.1) Head and neck 1 (50.0) 2 (22.2) Gastrointestinal 0 (0.0) 1 (11.1) Genitourinary 0 (0.0) 0 (0.0) Gynecologic 1 (50.0) 1 (11.1) Hepatobiliary 0 (0.0) 1 (11.1) Neuropathology 0 (0.0) 0 (0.0) Musculoskeletal 0 (0.0) 1 (11.1) Pulmonary 0 (0.0) 0 (0.0) Total 2 (100.0) 9 (100.0) The number of cases and the number of parts with discrepant diagnoses are listed in Tables 2 and 3. There were 2 of the 217 cases (0.92%) with major discrepancies, and 8 cases (3.7%) with minor discrepancies. Those 217 cases were composed of 301 parts, each containing an individual diagnosis. With respect to parts, there were 2 major discrepancies (0.66%) and 9 minor discrepancies (3.0%). One dermatopathology case was deferred because the electron micrograph images that had been available at the time of microscope slide review were not available at the time of WSI review. The microscope slide and WSI discrepant diagnoses are listed in Tables 4 and 5, and the entire data set of diagnoses is included as supplemental digital content (see supplemental material file at www. archivesofpathology.org in the November 2014 table of contents). There were 2 major case discrepancies. The first was interpreted as complex atypical endometrial hyperplasia on microscope slides, but complex, nonatypical endometrial hyperplasia by WSI (Figure 3). This was a difficult case, and the microscope slides had actually been shared at a subspecialty consensus conference at the time the case was originally interpreted. The second major discrepancy was a laryngeal biopsy that was originally interpreted as ulcerated squamous mucosa with atypia, possibly dysplasia. The corresponding WSI was interpreted as squamous carcinoma in situ, suspicious for invasion (Figure 4). This case was also difficult, and also had been shown around at the time the microscope slides were originally interpreted. There were 8 minor case discrepancies (Table 5). Most of these illustrate difficult interpretive problems for which occasional discrepancies commonly occur when using microscope slides for diagnosis. COMMENT Relatively few published studies document true intraobserver variability in surgical pathology, and the extent of intraobserver variability that should be considered acceptable is an open question. Similarly, there are few studies Table 4. Microscope Slide Diagnosis Complex atypical endometrial hyperplasia Ulcerated squamous mucosa with atypia, possibly dysplasia Cases With Major Discrepancies Whole-Slide Image Diagnosis Complex nonatypical endometrial hyperplasia Squamous carcinoma in situ suspicious for invasion Table 5. Microscope Slide Diagnosis Borderline ovarian tumor with areas of intraepithelial carcinoma Benign cyst, consistent with aneurysmal bone cyst Inflamed myofibroblastic proliferation Superficially invasive, welldifferentiated squamous carcinoma, excision recommended Superficial fragments of squamous carcinoma Invasive ductal carcinoma, nuclear grade 2 Phyllodes tumor, benign Hepatocellular carcinoma Cases With Minor Discrepancies Whole-Slide Image Diagnosis Atypical cyst consistent with mucinous borderline tumor Benign cyst, consistent with unicameral bone cyst Inflamed granulation tissue, negative for malignancy Vulvar intraepithelial neoplasia III, excision recommended Atypical mucosa, suspicious for squamous carcinoma Poorly differentiated carcinoma Phyllodes tumor (borderline versus benign?) Highly suspicious for hepatocellular carcinoma that document intraobserver variability between WSI and microscope slides. With the goal of validating WSI for interpreting frozen sections, Fallon and coworkers 5 reported a 96% correlation between WSI and frozen section microscope slide interpretation for benign versus malignant, borderline, or uncertain ovarian lesions. Similarly, Evans and colleagues 6 evaluated the use of telepathology and WSI for interpreting frozen sections in a Canadian health network and reported an accuracy rate of 98% and a deferral rate of 7.7% for a series of more than 350 interpretations by robotic microscope and 630 by WSI. Ramey et al 7 reported 89% accuracy when comparing a glass slide review of 67 frozen section cases with interpretation using WSI and a mobile viewing device. Focusing on the use of WSI for primary diagnosis, Molnar and colleagues 8 reported a final concordance rate of 95.1% when comparing WSI with microscope slide diagnoses of 61 routine gastric biopsies and 42 routine colon biopsies. Jukić and coworkers 9 tested intraobserver variability interpreting glass slides and digital images of 101 cases among 3 pathologists. They reported an overall significant discrepancy rate of 4.4% after excluding highly unusual or difficult cases and those that needed high magnification. The authors interpreted the results to support the integration of digital slides into pathology work flow. We previously completed an intraobserver variability (validation) study designed to test the noninferiority of WSI compared with microscope slide review for a series of previously diagnosed cases. 3 That study included 2 pathologists who reviewed 607 cases composed of 1025 individual parts. The number of cases we included in that study was based on an a priori power calculation performed by an independent statistician and substantially exceeded the minimum 60 cases recently recommended by the CAP working group. The results showed 1.65% major discrepancies for interpreting WSI, compared with 0.99% for interpreting microscope slides. There were 2.31% minor discrepancies interpreting WSI and 4.93% interpreting microscope slides in that study. Within the assumptions inherent in that study design, WSI was found not to be inferior to microscope slide review (P,.001) for primary diagnoses. That study was based on primary pathology cases, however, and was not enriched Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw 1463

6 Figure 3. The first major discrepancy was an endometrial lesion that was interpreted as complex, atypical, endometrial hyperplasia based on microscope slides but as complex, nonatypical, endometrial hyperplasia by whole-slide imaging (WSI) (hematoxylin-eosin, original magnification [from WS] 320); final magnifications 34.4 [A] and [B]). Figure 4. The second major discrepancy was a laryngeal lesion interpreted as ulcerated squamous mucosa with atypia, possibly dysplasia, on microscope slides but was interpreted as squamous carcinoma in situ, suspicious for invasion by whole-slide imaging (WSI) (hematoxylin-eosin, original magnification [from WSI] 320; final magnifications 35.4 [A] and [B]). with difficult cases as might be encountered if WSI was used for consultations. Few previous studies have focused on the use of WSI or telepathology for consultative diagnoses. Wilbur et al 10 compared the original diagnoses of 53 cases that had been submitted for consultation with the diagnoses obtained by 2 subspecialty pathologists who reviewed glass microscope slides and WSI for those cases. There was complete concordance between the client pathologist and both consultants in 45 cases (85%) and agreement between digital and microscope slide diagnosis in 48 cases (91%). However, in 5 cases (9%), the WSI interpretation was determined to be incorrect. Cases with tumors had better correlation (93%; 26 of 28) than nonneoplastic cases (88%; 22 of 25). Among other things, the authors noted that issues related to fine resolution and navigating ability at high resolution may have been associated with the discordant diagnoses by WSI and that case selection and experience should improve performance. In another study focusing on the use of WSI for consultation cases, 4 pathologists with a urologic subspecialty used routine microscopy and WSI to review a single core level from each of 50 diagnostically challenging needle biopsy specimens. 11(p68) Intraobserver agreement for Gleason grade and score was considered good to excellent for all pathologists, and the authors concluded that digital interpretation on prostatic needle biopsies was comparable to that of routine microscopy. The major and minor discrepancy rates of 0.9% and 3.7% for the current study compare favorably with the results of previously published studies and are nearly identical to the results of our own validation study for primary diagnosis. However, our study has several important limitations. First, because it is our standard practice to return microscope slides to clients after diagnosis, those slides were not available for a control arm of the study to test intraobserver variability for microscope slide interpretation. In fact, to our knowledge, our previous validation study that tested WSI for primary diagnosis is the only published study that included a control group of microscope slide reviews after a long washout period. The most obvious limitation to our current study, however, is that pathologists remember interesting and difficult cases for a long time. The CAP panel recently recommended a minimum 2-week washout period between microscope slide and WSI review for a validation test. Our average washout interval was 43 days, but there is little doubt that pathologists remembered some of these cases and would likely remember them for many months if not years. The extent to which cases were remembered is difficult to quantify, but we noted that the pathologists requested relatively few immunohistochemical stains at the time of WSI review, possibly because they remembered the results of available stains from their previous microscope slide review. Although some of the discrepancies in our current study were in cases with washout periods shorter 1464 Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw

7 than, or close to, the 14-day interval recommended by the CAP panel, when questioned, the pathologists involved with those cases admitted remembering the cases as being difficult and remembered discussing those difficult cases at subspecialty consensus conferences, but couldn t quite remember the final decision of the group. So for a few cases, the discrepancy was based as much on the inability to remember a group s conclusion as it was on variability in case interpretation. We suggest that, for the types of ofteninteresting cases received in consultation, a washout period longer than that recommended by the CAP panel should be considered. Of even more importance would be testing intraobserver variability for review of the original microscope slides as we did in our previous study. Our current study also excluded hematopathology and lymphoproliferative lesions because those cases often require high magnifications and numerous special stains. When discrepant cases were rereviewed, several pathologists with limited previous experience using WSI indicated that, in the future, they might request scans at 340 magnification, especially when it is important to quantify mitoses. Our study also had several strengths. First, to our knowledge, it is the largest WSI validation study to date that included only cases submitted for consultation (and, therefore, were relatively difficult cases). We also included a relatively large number of pathologists, not all of who had extensive experience interpreting WSI. The quality of the original microscope slide also has a very strong influence on the quality of the scanned image. Our previous validation study used only microscope slides prepared in our own laboratory, but the design of the current, consultation validation study duplicates the intended use of interpreting microscope slides of variable quality prepared elsewhere. As recommended by the CAP panel, we also tested the exact scanner, case transmission application, and viewer that at least one of our clients intends to use for seeking consultation diagnoses. This fulfilled one of the recommendations of the CAP expert panel, but we noted no performance variability compared with our previous experience that could be attributed to the different system. Therefore, unless image quality appears compromised, we do not currently advocate performing complete validation studies in the future if other clients wish to use other scanners or case transmission applications to seek our consultation diagnoses. Although the CAP panel suggested that it is impractical for laboratories to validate WSI for each organ system, some authors have recommended organspecific validation. 11 We included approximately 20 cases for each subspecialty area and did not detect a higher discrepancy rate for any given organ system. However, by combining the results of this study with our previous validation of primary diagnoses, we note that distinguishing degrees of squamous dysplasia and/or carcinoma and endometrial hyperplasia/dysplasia/carcinoma were associated with minor discrepancies in each study. Those areas are anticipated to have relatively high discrepancy rates for either WSI or routine microscopic review. As in our previous study, some pathologists suggested that low-grade inflammatory lesions (such as evaluating liver transplant biopsies) seemed to be more difficult to visualize by WSI than by routine microscopy, although this problem might be improved by more frequent use of 340 scans compared with 320. CONCLUSION The results of this study should be interpreted with caution, but they support the safety and efficacy of WSI for surgical pathology diagnosis. Based in part on our previous study, a washout interval exceeding 1 year should be a consideration for future validation studies. We thank the scanning technologists Alissa Ealy, HTL (ASCP), and Sol Crisostomo, HTL (ASCP), MT (AMT); the information technology specialist A. Scott Mackie, CET; the 26 pathologists who participated in the study; the Department of Pathology administrative assistants, who helped coordinate the distribution of cases to e-pathology; and to the clients who sent the cases for consultation. References 1. Omnyx. GE Healthcare s Omnyx Integrated Digital Pathology system awarded Health Canada clearance for routine diagnostic use. Omnyx Integrated Digital Pathology Web site. ge-healthcares-omnyx-integrated-digital-pathology-system-awarded-healthcanada-clearance-for-routine-diagnostic-use/4717. Accessed August 5, Leica Biosystems. Leica Biosystems Aperio epathology System Awarded Health Canada Clearance for Routine Diagnostic Use. Leica Biosystems News Web site. article/leica-biosystems-aperio-epathologytm-system-awarded-health-canadaclearance-for-routine-diagnostic/. Accessed August 5, Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med. 2013:137(4): doi: / arpa oa. 4. Pantanowitz L, Sinard JH, Henricks WH, et al. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Arch Pathol Lab Med. 2013;137(12): Fallon MA, Wilbur DC, Prasad M. Ovarian frozen section diagnosis: use of whole-slide imaging shows excellent correlation between virtual slide and original interpretations in a large series of cases. Arch Pathol Lab Med. 2010; 134(7): Evans AJ, Chetty R, Clarke BA, et al. Primary frozen section diagnosis by robotic microscopy and virtual slide telepathology: the University Health Network experience. Hum Pathol. 2009;40(8): Ramey J, Fung K, Hassell LA. Use of mobile high-resolution device for remote frozen section evaluation of whole slide images. J Pathol Inform. 2011; 2(1): doi: / Molnar B, Berczl L, Diczhazy C, et al. Digital slide and virtual microscopy based routine and telepathology evaluation of routine gastrointestinal biopsy specimens. J Clin Pathol. 2003;56(6): Jukić DM, Drogowski LM, Martina J, Parwani AV. Clinical examination and validation of primary diagnosis in anatomic pathology using whole slide digital images. Arch Pathol Lab Med. 2011;135(3): Wilbur DC, Madi K, Colvin RB, et al. Whole-slide imaging digital pathology as a platform for teleconsultation: a pilot study using paired subspecialist histocorrelations. Arch Pathol Lab Med. 2009;133(12): Rodriguez-Urrego PA, Cronin AM, Al-Ahmadie HA, et al. Interobserver and intraobserver reproducibility in digital and routine microscopic assessment of prostate needle biopsies. Hum Pathol. 2011;42(1): Arch Pathol Lab Med Vol 138, November 2014 Validating Whole-Slide Imaging Bauer & Slaw 1465