OFF-SITE SMARTPHONE VS. STANDARD WORKSTATION IN THE RADIOGRAPHIC DIAGNOSIS OF SMALL INTESTINAL MECHANICAL OBSTRUCTION IN DOGS AND CATS

Transcription

1 OFF-SITE SMARTPHONE VS. STANDARD WORKSTATION IN THE RADIOGRAPHIC DIAGNOSIS OF SMALL INTESTINAL MECHANICAL OBSTRUCTION IN DOGS AND CATS PETER G. NOEL, ANTHONY J. FISCHETTI, GEORGE E. MOORE, ALEXANDRE B. LE ROUX Off-site consultations by board-certified veterinary radiologists benefit residents and emergency clinicians by providing immediate feedback and potentially improving patient outcome. Smartphone devices and compressed images transmitted by or text greatly facilitate availability of these off-site consultations. Criticism of a smartphone interface for off-site consultation is mostly directed at image degradation relative to the standard radiographic viewing room and monitors. The purpose of this retrospective, cross-sectional, methods comparison study was to compare the accuracy of abdominal radiographs in two imaging interfaces (Joint Photographic Experts Group, off-site, smartphone vs. Digital Imaging and Communications in Medicine, on-site, standard workstation) for the diagnosis of small intestinal mechanical obstruction in vomiting dogs and cats. Two board-certified radiologists graded randomized abdominal radiographs using a five-point Likert scale for the presence of mechanical obstruction in 100 dogs or cats presenting for vomiting. The area under the receiver operator characteristic curves for both imaging interfaces was high. The accuracy of the smartphone and traditional workstation was not statistically significantly different for either reviewer (P = and P = 0.536). Correlation coefficients were and for each reviewer when the same radiographic study was viewed in different formats. Accuracy differences between radiologists were potentially related to years of experience. We conclude that off-site expert consultation with a smartphone provides an acceptable interface for accurate diagnosis of small intestinal mechanical obstruction in dogs and cat. C 2016 American College of Veterinary Radiology. Key words: abdominal radiographs, consultation, JPEG, Off-site, small intestinal mechanical obstruction, smartphone. Introduction SMALL INTESTINAL MECHANICAL OBSTRUCTION is a potentially life-threatening condition requiring prompt and accurate diagnosis. 1 Compared to radiographs, abdominal ultrasound is a more accurate test to diagnose mechanical obstruction but can be limited by high cost, lower availability, and a steep learning curve. 2 Abdominal radiographs are more amenable to off-site consultation, especially when operator experience in ultrasound is variable and transmission of movie clips can be slow. At our teaching hospital, abdominal radiographs made after-hours or From the Animal Medical Center, Department of Diagnostic Imaging, New York, NY, (Noel, Fischetti, Le Roux) and Purdue University College of Veterinary Medicine, West Lafayette, IN, (Moore). Supported by the American College of Veterinary Radiology 2015 Resident Research Grant Award. Portions of this study were presented at the 2015 ACVR Annual Scientific Meeting, Minneapolis, MN, October 8, Address correspondence and reprint requests to Peter G. Noel, at the above address. peter.noel@amcny.org Received December 15, 2015; accepted for publication May 9, doi: /vru on weekends provide the basis for the initial consultation between the off-site radiologist and on-site resident. Numerous studies have validated the use of portable, handheld devices in the radiographic and computed tomographic diagnosis of emergent and nonemergent diseases in people, including aortic aneurysm, pulmonary thromboembolism, and tuberculosis. 3 7 A recent study determined that CT images interpreted on an off-site smartphone for patients with an inconclusive diagnosis of appendicitis did not differ significantly from the in-house reports. 6 While some of these studies used Digital Imaging and Communications in Medicine (DICOM), others used compressed images. 3 7 The Joint Photographic Experts Group (JPEG) file format is a form of irreversible (lossy) image compression that is widely available and allows rapid transmission of large files. The degree to which compression degrades image quality depends in large part on the compression ratio. 8 An investigation in people assessing three specific abnormalities on thoracic radiographs (pulmonary nodules, pneumothorax, and interstitial lung disease) found no statistically significant difference between 10:1 compressed Vet Radiol Ultrasound, Vol. 57, No. 5, 2016, pp

2 458 NOELETAL JPEG images and the uncompressed radiographs. 9 Two independent literature reviews funded by Canada Health Infoway concluded that irreversible compression is a clinically acceptable option for the compression of medical images. 8 The American College of Radiology (ACR) and American College of Veterinary Radiology (ACVR) share a similar stance on image compression. The ACVR states that several methods, including both reversible (lossless) and irreversible (lossy) compression techniques, may be used, provided the resulting images show no clinically significant loss of quality. 10,11 The purpose of this study was to compare the accuracy of abdominal radiographs in two imaging interfaces for the diagnosis of small intestinal mechanical obstruction in dogs and cats presented with acute vomiting. The two imaging interfaces were the JPEG-format, off-site, smartphone (Apple Inc., iphone 6, Cupertino, CA) and the DICOM-format, on-site, traditional picture archiving and communication system (PACS) workstation (Coactiv Medical Solutions, Ridgefield, CT). It is hypothesized that there will be no statistically significant difference in accuracy between these two interfaces. Materials and Methods This retrospective, cross-sectional, methods comparison study evaluated abdominal radiographs from dogs and cats confirmed with or without a small intestinal mechanical obstruction. The abdominal radiographs were made in either of the two X-ray suites (Quantum Medical Imaging, Quantum HF Radiographic Imaging System, Ronkonkoma, NY) at the Animal Medical Center. Both units had the same flat panel digital radiographic image detector (Canon U.S.A., Inc., Canon CXDI-50G, Melville, NY) with the same postprocessing algorithms. The technique varied depending on the thickness of the animal, with the kvp ranging from and the mas from 5 8. The Animal Medical Center s medical records were searched from January 1, 2010 through January 1, 2014, until 60 obstructed patients and 40 nonobstructed patients (overall test sample prevalence of 60%) were obtained. Inclusion criteria comprised either dogs or cats presenting with acute onset vomiting with at least two-view diagnostic quality orthogonal abdominal radiographs at the time of admission. Only twoview radiographs (right-lateral plus dorsal recumbent or left-lateral plus dorsal recumbent) were included. If threeviews were obtained at admission (right lateral, left lateral, and dorsal recumbent), the left-lateral view was used instead of the right lateral. The order of patient positioning was not standardized. Additional criteria included animals having an abdominal ultrasound and/or surgery within 24 h of presentation to confirm disease positive or negative obstructive status. Animals were excluded if they had chronic vomiting (>7 days) or delayed abdominal sonography or surgery greater than 24 h after presentation. Small intestinal mechanical obstruction was determined subjectively on ultrasound for patients with pathologic segmental intestinal dilation, intestinal plication, or an obstructing lesion such as a mass or foreign object. Abdominal ultrasound has a high accuracy for determining small intestinal mechanical obstruction justifying its use as a reference standard. 2 A second-year imaging resident (first author) reviewed the medical records and determined group status. Two board-certified veterinary radiologists, one with 10 years postresidency training (R1) and the other with 1 year (R2), interpreted all images. Radiologists were blinded from details related to disease prevalence, signalment, patient demographics, and medical history, aside from the presenting complaint of acute vomiting. Two separate imaging interfaces were evaluated using two-view abdominal radiographs. The first was the standard PACS workstation used to interpret DICOMformat abdominal radiographs. These were reviewed on two high-resolution monitors (NDS Dome E3cHB, Portrait , and Landscape , 23 inch, 750:1 contrast ratio, defaults brightness 500 cd/m 2,pixel pitch 0.23 mm) contained in a dimly lit radiographic interpretation room. 12 Windowing and leveling was performed as needed for interpretation. The second interface was JPEG-format radiographs, compressed (compression < 10:1) from the processed images without any changes to window/level, and transmitted by to an off-site smartphone (iphone 6, Apple; 4.7-inch HD display, resolution of at 326 pixels per inch, 1400:1 contrast ratio, default brightness 504 cd/m 2, pixel pitch 0.08 mm). 13 The simulated off-site viewing environment consisted of a high trafficked area on the main floor of the hospital with high ambient lighting. The cases selected for this study were viewed by either R1 or R2 at the initial clinical presentation, but no case was viewed within 4 months of study initiation. Two separate viewing sessions were scheduled, separated by a 4-month washout period to limit recall bias. 14 A total of 200 abdominal radiographic exams were interpreted for the study, 100 for each of the two viewing sessions. The first session consisted of 11 sets of abdominal radiographs (1 day had 12) evaluated daily by each board-certified veterinary radiologist for 9 days total (equaling 100 abdominal radiographic exams for each reviewer). Each daily set of abdominal radiographic exams consisted of randomized interfaces (smartphone vs. standard workstation) and outcomes (obstructed vs. nonobstructed). The randomization of interfaces for each daily review session was intended to limit the possibility of second look bias. 15 The time of each daily interpretation was also randomized. After the 4-month washout period, the second viewing session was

3 VOL. 57, NO. 5 OFF-SITE SMARTPHONE VS. STANDARD WORKSTATION 459 performed. This session was identical to the first, except that the complimentary imaging interface for each case was reviewed. Each reviewer was given four test cases to familiarize with the testing format prior to study initiation. Each reviewer completed a five-point Likert scale for the presence of a small intestinal mechanical obstruction: grade 0 (definitely not obstructed), grade 1 (probably not obstructed), grade 2 (indeterminate), grade 3 (probably obstructed), and grade 4 (definitely obstructed). Receiver operating characteristics (ROCs) curves were generated for each reviewer-imaging interface combination and for each viewing session. The area under the curve (AUC) was calculated in order to determine accuracy. The AUCs were compared between interfaces and between reviewers (STATA SE, version 14.1, StataCorp, College Station, TX). Spearman s rank correlation coefficient was also performed to assess correlation between ordinal scores from imaging formats and for R1 and R2. Values of P < 0.5 were considered significant. Results The sample population consisted of 100 animals (29 cats, 71 dogs). Sixty patients (16 cats, 44 dogs) tested positive for a small intestinal mechanical obstruction and 40 patients (13 cats, 27 dogs) tested negative. The sample size selected had 80% power to detect 5% difference in accuracy in paired ROC curves with a 95% confidence level. Of the 60 test positive outcomes, 14 patients (23%) had identifiable foreign objects (e.g., corncob, fruit pit, plastic toy, and cloth/textile). Forty-six (76%) of the obstructed patients had a final diagnosis made by surgery and 14 (24%) by abdominal ultrasonography. Forty (100%) of the nonobstructed patients had a final diagnosis determined by abdominal ultrasonography. In the obstructed patients, the mean age was 4.1 years (range: years). Thirty of 60 (50%) were neutered males, eight of 60 (13.3%) were intact males, 20 of 60 (33.3%) were spayed females, and three of 60 (3.4%) were intact females. In the nonobstructed patients, the mean age was 7.4 years (range: years). Eighteen of 40 (45%) were neutered males, one of 40 (2.5%) were intact male, 17 of 40 (42.5%) were spayed females, and four of 40 (10%) were intact females. The following breeds were included: domestic short-haired cat (n = 13 obstructed, n = 11 nonobstructed); mixed-breed canine (8,5); American Bulldog (4,0); Labrador Retriever (3,3); Dachshund (3,0); Cavalier King Charles Spaniel (3,0); French Bulldog (2,1); German Shepherd (2,0); Chihuahua (2,2); Bernese Mountain Dog (2,0); Golden Retriever (2,2); Black Russian Terrier (1,0); Chesapeake Bay Retriever (1,0); English Setter (1,0); Maine Coon (1,0); Miniature Pinscher (1,1); Miniature Schnauzer (1,0); Shih Tzu (1,1); Standard Poodle (1,0); Weimaraner (1,0); West Highland White Terrier (1,0); Yorkshire Terrier (1,4); Domestic medium-haired cat (1,0); Basset Hound (1,0); Norwegian Forest Cat (1,0); English Bulldog (1,0); Domestic long-haired cat (0,2); Basenji (0,1); Coton du Tulear (0,1); Jack Russel Terrier (0,1); Keeshond (0,1); Maltese (0,1); Miniature Australian Shepherd (0,1); Miniature French Poodle (1,1); and Soft Coated Wheaten Terrier (0,1). The area under the ROC curve (AUC) for R1 was for DICOM/PACS, corresponding to an accuracy of 93.5% (95% CI: ). This reviewer had an AUC of for JPEG/Smartphone, corresponding to an accuracy of 91.3% (95% CI: ). The accuracy of the smartphone and standard workstation interfaces was not statistically significantly different for R1 (P = 0.384). These results are summarized in Fig. 1A. The area under the ROC curve for R2 was for DICOM/PACS, corresponding to an accuracy of 83.2% (95% CI: ). This reviewer had an AUC of for JPEG/Smartphone, corresponding to an accuracy of 85.6% (95% CI: ). The accuracy of the smartphone and standard workstation interfaces was not statistically significantly different for R2 (P = 0.536). These results are summarized in Fig. 1B. Radiologist accuracy of the smartphone interfaces was compared to their accuracy with the standard workstation of the same study after a 4-month washout period. The correlation coefficient when the same study in different interfaces was evaluated after 4 months for R1 and R2 was and 0.705, respectively. When all 200 cases from session 1 and session 2 were combined to generate an overall accuracy for each reviewer, there was a statistically significant difference between the 92.5% accuracy for R1 and the 84.0% accuracy for R2 (P-value = ). Discussion Findings from the current study supported the hypothesis that accuracy of off-site expert interpretation using a small-screen, handheld smartphone is comparable to accuracy of conventional, on-site, radiographic interpretation for diagnosing small intestinal mechanical obstruction in dogs and cats presented for acute vomiting. This clinical model was chosen as it represents a fairly common afterhours scenario between off-site staff radiologist and on-site resident or primary care veterinarian at our institution. Authors do not intend for the smartphone interface to be used as a replacement for routine viewing with a standard workstation or ultrasound examination when available. Display monitor quality and viewing environment are often regarded as important factors potentially impacting diagnostic accuracy Monitor luminance, bit depth, and screen size are considered the most influential factors in monitor performance. The two monitor systems utilized in this study had similar luminance, but differing

4 460 NOELETAL FIG. 1. (A) Results for receiver operating characteristic (ROC) curve analysis of diagnostic accuracy for R1 s interpretation of DICOM/PACS and JPEG/Smartphone abdominal radiograph interfaces. There was no statistically significant difference (P = 0.384) in the area under the curves (AUCs) between the two interfaces. (B) Results for ROC curve analysis of diagnostic accuracy for R2 s interpretation of DICOM/PACS and JPEG/Smartphone abdominal radiograph interfaces. There was no statistically significant difference (P = 0.536) in the AUCs between the two interfaces. contrast ratios and screen size. The smartphone has nearly twice the contrast ratio as the medical grade monitor, but has only a 4.7-inch display compared to 23 inches. Despite this difference, screen size did not affect accuracy in our study. Practical differences such as the need to zoom on a small screen and difficulty viewing two images at the same time made review of the images tedious but did not affect the final diagnosis. Results could potentially differ in cases where the clinical question requires side-by-side comparisons (e.g., response to treatment for heart failure, evaluating contrast uptake, etc.). Regarding the viewing environment, ambient lighting for standard workstations is usually low, adjustable, and indirect. 17 High ambient lighting ultimately reduces luminance and thus contrast ratio. 17 Perhaps, the high ambient lighting while reviewing on the smartphone was offset by its relatively high luminance and higher contrast ratio. The JPEG compressed images are of particular interest in this study as they allow rapid transmission of large files. This JPEG image compression and transmission of radiographs was strongly discouraged in the 2008 veterinary digital imaging supplement 18 but may have become less of a concern with improved digital imaging technology. 19 Prior studies in people have shown no reduction in diagnostic accuracy for compression ratios <10:1 for thoracic radiographs. 9 The ACR and ACVR technical standards both allow for image compression under the guidance of a radiologist. 10,11 In fact, the ACR technical standard makes no statement regarding the type or amount of compression acceptable. The standard uses the term diagnostically acceptable irreversible compression (DAIC) to refer to compression that, under the guidance of a qualified individual, does not affect a particular diagnostic task. A human teleradiology survey in 2005 found that the second most commonly transmitted file format, after DICOM, was bitmap/jpeg (38.1% of respondents) and that files were transmitted via 21.4% of the time. 20 The images used in this study had postprocessing algorithms that were optimized for immediate viewing and required little or no windowing or leveling on the standard workstation. Our finding that JPEG- and DICOMformatted images had the same image quality may not be generalizable for clinical situations where original digital radiographs are poorly exposed or have suboptimal postprocessing. Therefore, it remains the responsibility of the expert consulting radiologist to determine whether the transmitted JPEG is diagnostically acceptable. Consistent with previous reports, veterinary radiologists are accurate at the radiographic diagnosis of small intestinal mechanical obstruction. 2 Also not surprising, the more experienced radiologist had a greater overall accuracy compared to the less experienced radiologist. This increased accuracy was the result of greater specificity. Previous studies have found that recently qualified radiologists have a high sensitivity and lower specificity. 21 Specificity and thus overall accuracy tend to increase with experience, as supported by this study. Both radiologists had very good correlation when comparing the two interfaces for the same case after a 4-month washout period. Although the less experienced radiologist had lower correlation, this was not likely related to competence. Rather, the ordinal scale used for the Likert scale may have reduced correlation. For example, if a reviewer selected a 0 (definitely not obstructed) for a particular case in the DICOM-format and then 4 months later selected a 1 (probably not obstructed) for the same case in JPEG format, the correlation would be negatively affected. However, the clinical difference between a definitely not and probably not obstructed response would only be minimal.

5 VOL. 57, NO. 5 OFF-SITE SMARTPHONE VS. STANDARD WORKSTATION 461 It has been suggested in human radiology that adults prefer immediate feedback with regards to learning. 22 The consultations between an on-site resident and off-site staff radiologist allows for nearly instantaneous feedback on a particular case, and we believe, facilitates learning. Because of potential improvements in patient outcomes, off-site consultation could also be supported in the scenario of telemedicine and consultation with other nonresident clinicians. However, instant report generation may be difficult for the consulting off-site radiologist. The expectations of clients may be different from that of a resident-mentor relationship. Regardless of who is being consulted, this study was not aimed to replace the traditional workstation, but simply to support a timely, off-site alternative. A limitation of this study includes the potential impact of recall bias of an abdominal radiograph after a 4-month washout period. There is no information in veterinary medicine, limited information in people, and no consensus regarding the length of time required to limit recollection of a previous radiographic diagnosis when reviewing the same image. 14,23 Human studies have used weeks, months, and up to a year. 14,23 Both radiologists in our study were asked after the second reading session about case recall, and neither mentioned remembering any of the cases (including the recognizable foreign bodies). Thus, we believe recall bias had little impact on this study. Second, field vs. laboratory bias is inherent to studies with this design. 23 Both reviewers were aware that their response to each case had no clinical consequence. Attempts were made to limit other causes of field vs. laboratory bias. These included limiting daily reporting intensity to cases, randomly changing the time of day between days, and intermixing interfaces during each daily interpretation session. These deliberate changes were created to mimic clinic practice as closely as possible, and not fatigue the reader with a large number of cases. Computer vision syndrome, which can include eye discomfort and blurred vision, and other negative effects from prolonged soft-copy viewing under suboptimal conditions, was not likely a problem during this study as the daily interpretation sessions lasted <30 min. Last, the use of JPEG images created from optimized digital radiographs (ready-to-view window/level and exposure) may have overemphasized the benefits of JPEG format compared to DICOM format. Future studies could include small screen, off-site format evaluation of other emergent disease conditions, such as left-sided congestive heart failure using thoracic radiography. Another study design could assess small screen interpretations of digital radiographs made from different units and different postprocessing algorithms. Based on a reference standard of ultrasonographic or surgical diagnosis, off-site smartphone review of abdominal radiographs by a board-certified veterinary radiologist yielded comparable diagnostic accuracy to the traditional on-site workstation for diagnosing small intestinal mechanical obstruction in this sample of acutely vomiting dogs and cats. These results may permit efficient and productive interactions between on-site primary care veterinarians and off-site radiologists. 1. Fossum, TW. Surgery of the small intestine. Small animal surgery, 3rd ed. St. Louis: Mosby-Elsevier, 2007; Sharma A, Thompson MS, Scrivani PV, et al. Comparison of radiography and ultrasonography for diagnosing small-intestinal mechanical obstruction in vomiting dogs. Vet Radiol Ultrasound 2011;52: Panughpath SG, Kalyanpu A. Radiology and the mobile device: radiology in motion. Indian J Radiol Imaging 2012;22: Choudhri AF, Norton PT, Carr TM, et al. Diagnosis and treatment planning of acute aortic emergencies using a handheld DICOM viewer. Emerg Radiol 2013;20: Choudhri AF, Carr TM, Ho CP, et al. Handheld device review of abdominal CT for evaluation of acute appendicitis. J Digit Imaging 2012;25: Seong NJ, Kim B, Lee S, et al. Off-site smartphone reading of Ct images for patients with inconclusive diagnoses of appendicitis from on-call radiologists. AJR 2014;203: Johnson PT, Zimmerman SL, Health D, et al. The ipad as a mobile device for CT display interpretation: diagnostic accuracy for identification of pulmonary embolism. Emerg Radiol 2012;19: CAR standards for irreversible compression in digital diagnostic imaging within radiology -Canadian Association of Radiologists. Ottawa, 2011; Beall DP, Shelton DP, Kinsey TV, et al. Image compression and chest radiograph interpretation: image perception comparison between uncompressed chest radiographs and chest radiographs stored using 10:1 JPEG compression. J Digit Imaging 2000;12(Suppl 1): ACR technical standard for electronic practice of medical imaging American College of Radiology (ACR) Practice Guidelines. United States 2014;Res. 39. REFERENCES 11. American College of Veterinary Radiology. Available from: www. acvr.org. January 20, NDS Surgical Imaging. Available from: January 6, Apple. Available from: January 6, Hardesty LA, Ganott MA, Hakim CH, et al. Memory effect in observer performance studies of mammograms. Acad Radiol 2005;12: Gondy L. Designing user studies in informatics. London: Springer, 2011; Ludewig E, Boeltzig C, Gabler K, et al. Display quality of different monitors in feline digital radiography. Vet Radiol Ultrasound 2011;52: Puchalski S. Image display. Vet Radiol Ultrasound 2008;49:S9 S Poteet B. Veterinary teleradiology. Vet Radiol Ultrasound 2008;49:S33 S Geyer N, Reichle J, Valdes-Martinez A, et al. Radiographic appearance of confirmed pulmonary lymphoma in cats and dogs. Vet Radiol Ultrasound 2010;51: Lienemann B, Hodler J, Luetolf M, et al. Swiss teleradiology survey: present situation and future trends. Eur Radiol 2005;15: Lamb C, Parry A, Baines E, et al. Does changing the orientation of a thoracic radiograph aid diagnosis of rib fractures?. Vet Radiol Ultrasound 2011;52: Chhem RK, Hibbert KM, van Deven T. Radiology education: the scholarship of teaching and learning. New York: Springer Science & Business Media, 2008; Boone D, Halligan S, Mallett S, et al. Systematic review: bias in imaging studies- the effect of manipulating clinical context, recall bias and reporting intensity. Eur Radiol 2012;22: