ORIGINAL RESEARCH CONTRIBUTION. Abstract. Inna Elikashvili, DO, Ee Tein Tay, MD, and James W. Tsung, MD, MPH

Transcription

1 ORIGINAL RESEARCH CONTRIBUTION The Effect of Point-of-care Ultrasonography on Emergency Department Length of Stay and Computed Tomography Utilization in Children With Suspected Appendicitis Inna Elikashvili, DO, Ee Tein Tay, MD, and James W. Tsung, MD, MPH Abstract Objectives: The role of clinician-performed ultrasonography (US) for suspected appendicitis is unclear. Published data conclude that US has high specificity to rule in the diagnosis of appendicitis, with variable sensitivity to rule it out. Newer data suggest that point-of-care (POC) US may have similar test characteristics. Our objective was to evaluate the effect of POC US in children with suspected appendicitis and its effect on emergency department (ED) length of stay (LOS) and computed tomography (CT) utilization. Methods: This was a prospective observational convenience sample of children with suspected appendicitis requiring imaging evaluation that adhered to the Standards for the Reporting of Diagnostic accuracy studies (STARD) criteria. Outcomes were determined by operative or pathology report in those who had appendicitis, and 3-week phone follow-up in those patients who were nonoperative. Differences in ED LOS were analyzed by one-way analysis of variance (ANOVA) between patients who received dispositions after POC US, radiology US, or CT. Test performance characteristics were calculated for all imaging modalities. Results: Among 150 enrolled patients, 50 had appendicitis (33.3%). There were no missed cases of appendicitis in discharged patients at 3-week phone follow-up, nor negative laparotomies in those who went to the operating room. Those who had dispositions after POC US (n =25) had a significantly decreased mean ED LOS (154 minutes, 95% confidence interval [CI] = 115 to 193 minutes) compared with those requiring radiology US (288 minutes, 95% CI = 257 to 319 minutes) or CT scan (487 minutes; 95% CI = 434 to 540 minutes). Baseline CT rate was 44.2% (95% CI = 30.7% to 57.7%) prior to study start and decreased to 27.3% (95% CI = 20.17% to 34.43%) during the study. CTs were avoided in four patients with conclusive POC US results and inconclusive radiology US results. The sensitivity, specificity, positive likelihood ratio (LR+), and negative likelihood ratio (LR ) for POC US were 60% (95% CI = 46% to 72%), 94% (95% CI = 88% to 97%), 10 (95% CI = 4 to 23), and 0.4 (95% CI = 0.3 to 0.6). For radiology US they were 63% (95% CI = 48% to 75%), 99% (95% CI = 94% to 99%), 94 (95% CI = 6 to 1,500), and 0.4 (95% CI = 0.3 to 0.6); and for CT they were 83% (95% CI = 58% to 95%), 98% (95% CI = 85% to 99%), 45 (95% CI = 3 to 707), and 0.2 (95% CI = 0.05 to 0.5). Conclusions: It may be feasible to reduce ED LOS and avoid CT scan when using POC US to evaluate children with suspected appendicitis. Test characteristics for POC US have high specificity to rule in appendicitis, similar to radiology US. Addition of POC US prior to sequential radiology imaging was safe, without missed cases of appendicitis or negative laparotomies. ACADEMIC EMERGENCY MEDICINE 2014; 21: by the Society for Academic Emergency Medicine From the Departments of Emergency Medicine and Pediatrics, Divisions of Pediatric Emergency Medicine (IE, ETT, JWT) and Emergency Ultrasound (ETT, JWT), Mount Sinai School of Medicine, New York, NY. Received May 1, 2013; revisions received July 29, August 22, and August ; accepted September 11, Presented at the Pediatric Academic Societies Annual Meeting, Washington, DC, May 2013; and the Society for Academic Emergency Medicine Annual Meeting, Atlanta, GA, May The authors have no relevant financial information or potential conflicts of interest to disclose. Supervising Editor: Thomas Costantino, MD. Address for correspondence and reprints: James W. Tsung, MD, MPH; jtsung@gmail.com by the Society for Academic Emergency Medicine ISSN doi: /acem PII ISSN

2 164 Elikashvili et al. POINT-OF-CARE US FOR PEDIATRIC APPENDICITIS The utility of clinician-performed ultrasonography (US) for suspected appendicitis is unclear. 1 Published data conclude that US has high specificity to rule in the diagnosis of appendicitis, with variable sensitivity to rule it out. 2 Recent data suggest that pointof-care (POC) US by emergency physicians (EPs) may have similar test characteristics to radiology department US for appendicitis evaluation. 1,3 Radiation exposure in children, 4 as well as the extended times required for evaluation of abdominal pain contributing to emergency department (ED) overcrowding, 5 are drawbacks to computed tomography (CT) scan. To reduce CT rates, sequential imaging with US first, followed by CT if US is inconclusive, is supported by published data 6,7 and advocated by multiple guidelines. 7 9 It is unknown if the addition of POC US to sequential radiology imaging can decrease ED length of stay (LOS) and CT use in children with suspected appendicitis. Our objective was to determine the effect of POC US on ED LOS and CT use in children with suspected appendicitis. We also calculated test performance characteristics for POC US, radiology department US, and CT scan for appendicitis. probability of appendicitis before POC US (<1%, 1% 25%, 26% 50%, 51% 75%, 76% 99%, or >99%). Using a SonoSite M-turbo US machine (SonoSite Inc., Bothell, WA) with a 10- to 5-MHz linear transducer probe, focused US was performed by the pediatric emergency attending or fellow. The transducer was placed over the point of maximal tenderness or pain (as indicated by the patient), typically in the right lower quadrant. Both longitudinal and transverse views of the right lower quadrant were obtained (Figure 1), and video or still images in each orientation were recorded. A positive POC US result was defined by the presence of a noncompressible tubular structure, visualized from tip to cecum, measuring >6 mm in diameter (Figure 2). Criteria for a normal appendix included a compressible tubular structure, visualized in perpendicular orthogonal planes from tip to cecum, measuring 6 mm in diameter (Figure 3). Equivocal results were recorded when the enrolling physician could not identify either of METHODS Study Design This was a prospective, observational study adhering to the Standards for Reporting of Diagnostic Accuracy (STARD) criteria. 10 Written informed consent was obtained from the patient or parent/guardian, and written assent was obtained from patients aged 7 years or older. Approval for this study was granted by the hospital s institutional review board. Study Setting and Population The study was conducted from May 1, 2011, to October 1, 2012, in an urban pediatric ED. A convenience sample of patients presenting with suspected appendicitis requiring laboratory or radiographic evaluation was enrolled when a trained pediatric EP was available. Inclusion criteria included patients up to 21 years of age, with suspected appendicitis requiring laboratory or radiographic (radiology US or CT) evaluation, as determined by the attending and fellow pediatric EP. Patients were excluded if they arrived to the ED with established diagnoses of appendicitis or inflammatory bowel disease, prior abdominal CT or US imaging, or unstable vital signs or life-threatening illness requiring resuscitation. Study Protocol Before the start of the study, all participating pediatric attending EPs and fellows attended a 30-minute didactic session to learn how to use US to evaluate the right lower quadrant for appendicitis and to standardize the method in which POC US was performed. This was followed by a 30-minute hands-on practical session with live models. Before performing the POC US, pediatric EPs completed a structured data collection form and recorded history and physical examination findings. The enrolling clinician also determined and recorded the pretest Figure 1. Right lower quadrant POC US technique: probe orientation. The strategy for finding appendicitis is to place the probe on the area of maximal tenderness, sono-palpalpation/ graded compression to eliminate overlaying bowel gas; transverse and longitudinal planes should be captured. POC = pointof-care; US = ultrasound. TRANSVERSE IMAGE Psoas muscle Lateral Abdominal musculature Medial LONGITUDINAL IMAGE Figure 2. POC US image of appendicitis in short and long axis. Using the psoas muscle and iliac vessels as constant anatomic landmarks, looking for a noncompressible tubular structure around to these landmarks, measure (outer wall outer wall) diameter of appendix if visualized; appendicitis on POC US defined as noncompressible tubular structure, visualized tip to cecum, measuring >6 mm. POC = point-of-care; US = ultrasound.

3 ACADEMIC EMERGENCY MEDICINE February 2014, Vol. 21, No TRANSVERSE IMAGE Psoas muscle Abdominal musculature LONGITUDINAL IMAGE Figure 3. POC US image of a normal appendix in short and long axis. Normal appendix on POC US defined as compressible tubular structure, visualized tip to cecum, measuring <6 mm. the above. The enrolling physician recorded the US findings on data collection forms immediately after the procedure and before reviewing or knowing the results of any radiographic imaging studies. If the diagnosis of appendicitis was made by POC US, the pediatric EP also determined and documented the posttest probability of appendicitis after POC US (51% 59%, 60% 69%, 70% 79%, 80% 89%, 90% 99%, or >99%). Patient demographic information, laboratory, and radiology findings were obtained from electronic medical records, as well as triage time, time of radiology US, time of CT scan, and time of disposition. Time of imaging study was determined by the time stamp on the radiology image. Time of disposition was determined by the time stamp of admission order or discharge order in the electronic medical record. ED LOS was determined by triage time to time of disposition. After completion of POC US, patients received further radiology imaging based on enrolling physicians discretion. At our institution, radiology department US is performed by sonographer technicians and interpreted by attending pediatric radiologists during the day and performed and interpreted by radiology residents at night and on weekends. CT scans are interpreted by attending pediatric radiologists during the day and by attending radiologists on nights and weekends. This staffing pattern was unchanged during the period of our study and 3 years prior. We defined radiology US or CT results as positive or negative for appendicitis based on an attending radiologist report consistent with appendicitis or normal appendix identified, respectively. We defined radiology US or CT findings as equivocal based on an attending radiologist s report neither a normal nor an abnormal appendix was visualized. For patients without definite diagnoses after radiology imaging in the ED, clinical follow-up consisted of review of electronic medical records and structured clinical telephone follow-up to ascertain final outcomes. In these patients the clinical diagnosis of no appendicitis was confirmed by resolution of all clinical symptoms at least 3 weeks after the initial ED visit. Outcome Measures Computed tomography ordering rates for patients with suspected appendicitis were determined before 11 and during the study. The estimate of CT utilization prior to the study was obtained from a retrospective review of consecutive cases of children with suspected appendicitis requiring radiology imaging from our electronic ED medical records and was cross-referenced with our radiology department s imaging database. 11 As mentioned before, no changes in radiology US availability or staffing occurred that would bias CT utilization rates during or prior to the study. Outcomes were determined by operative or pathology report in those who had appendicitis and 3-week phone follow-up in those patients who were non-operative. We calculated test performance characteristics for POC US, radiology US, and CT scan. We also performed subgroup analysis of test performance characteristics stratified by novice ( 25 scans) and experienced (>25 scans) based on the minimum number of scans for credentialing based on the American College of Emergency Physicians emergency US guidelines. 12 Data Analysis Data were analyzed with SPSS Statistics (IBM, Armonk, NY) and test performance characteristics were calculated for sensitivity, specificity, positive likelihood ratios (LR+), and negative likelihood ratios (LR ), with 95% confidence intervals (CIs). For purposes of calculating test characteristics, US examinations that were nondiagnostic (neither appendicitis nor normal appendix visualized) were coded as negative results. Descriptive statistical analyses were used for categoric data. Differences in ED LOS were analyzed by analysis of variance (ANOVA) between patients who received dispositions after POC US, radiology US, or CT. Ignoring clustering by US operator and using the method of Arkin and Wachtel, 13 a sample size of 150 would be needed to obtain a 95% CI (standard deviation [SD] 5%) around a 92% specificity for appendicitis according to the study by Fox et al. 1 The still images and video clips were reviewed for operator error by trained pediatric emergency medicine sonologists (JWT, ETT). They were blinded to clinical findings, the enrolling physicians US interpretations, and radiographic imaging. POC US images and videos were reviewed to identify and classify errors. 14 Additionally, the time to perform the POC US was determined from the time stamps on the first and last US images recorded for each patient. RESULTS A total of 150 patients were enrolled, 56% of patients were female, and the mean (SD) age was 12 (5.2) years, with a range of 2 to 19 years. Patient demographic and clinical information is presented in Table 1. Study flow by imaging results and disposition is presented in Figures 4A and 4B, and a standard STARD flow chart is available in Data Supplement S1 (available as supporting information in the online version of this paper). Among 150 enrolled patients, the prevalence of appendicitis was 33.3% (n =50). Using ANOVA, we determined that patients with dispositions after POC US

4 166 Elikashvili et al. POINT-OF-CARE US FOR PEDIATRIC APPENDICITIS Table 1 Patient Demographic and Clinical Characteristics Sex, female 84 (56) Age (yr) 12 (5.2) History of fever 52 (34.7) Fever at triage >100.4 F 12 (8.2) Nausea 103 (68.7) Vomiting 76 (50.7) Maximal tenderness in right lower 116 (77.3) quadrant pain ANC > /L 77 (51.3) Appendicitis 50 (33.3) Time to perform POC US (min) 7 ( ) Perforated appendicitis, n=50 13 (26) N=150. Data are reported as number (%), mean (SD), or mean (95% CI). ANC = absolute neutrophil count; POC = point-of-care; US = ultrasound. (n =25) had a significantly decreased mean ED length of stay compared to those requiring sequential radiology imaging (Table 2). The baseline CT rates prior to study start were 44.2% (95% CI = 30.7% to 57.7%) 11 and 27.3% (95% CI = 20.17% to 34.43%) during our study (p = NS). We observed that four patients with equivocal radiology US results did not receive CT scans, contrary to our sequential imaging protocol (CT scan following inconclusive radiology US results). Three of these patients (with pretest probabilities of appendicitis at 51% to 75%) with positive POC US results went to the operating room, and one patient (with pre-test probability for appendicitis at 51% to 75%) with a normal appendix found on POC US was discharged home. The test performance characteristics of POC US, radiology US, and CT are presented in Table 3. There were no missed cases of appendicitis in discharged patients (a) N=150 POC US + 35 POC US equivcoal 104 POC US - 11 TP= 30 FP = 5 Neg= 85 Pos=19 TN=10 FN=1 FOLLOW-UP IMAGING AND DISPOSITION AFTER PoC US RUS only: 25 CT only: 4 RUS+CT: 1 RUS: 3 CT: 0 RUS+CT: 2 RUS only: 42 CT only: 6 RUS+CT: 19 RUS: 9 RCT: 2 RUS+CT: 7 RUS only: 8 no CTs RUS: 1 OR: 28 Admit->OR: 2 DC home: 0 Appendicitis: 30 OR: 0 Admit->OR: 0 DC home: 5 Appendicitis: 0 OR: 0 Admit-> DC: 19 DC home: 66 Appendicitis: 0 OR: 15 Admit->OR: 3 DC home: 1* (*recalled by radiology) Appendicitis: 19 Admit-: 1 (serial exams) DC home: 9 OR: 1 Appendicitis: 1 (b) OR 5 Admit 1 POCUS N=150 Home 19 OR Radiology US Home Admit OR 13 CT 41 Home 19 Admit/OR for other 9 Figure 4. (A) Flow chart of POC US results, follow-up imaging and final disposition. There were no missed cases of appendicitis in discharged patients at 3-week phone follow-up or negative laparotomies in those who went to the OR. (B) Flow chart by patient disposition. *Admitted patient with POC US equivocal result was diagnosed with pyelonephritis. One of five patients with equivocal POC US result (appendicitis incompletely visualized); four of five patients with positive POC US results. Two of 19 patients with normal appendix on POC US (negative); 17 of 19 with equivocal POC US results (either normal appendix not visualized or incompletely visualized). DC = discharged home; OR = operating room; POC = point-of-care; RUS = radiology-performed ultrasound; TN = true negative; TP = true positive; US = ultrasound.

5 ACADEMIC EMERGENCY MEDICINE February 2014, Vol. 21, No Table 2 Time to Disposition Imaging Modality n ED LOS (min) 95% CI A. Stratified by Imaging Modality POC US RUS CT ANOVA p < for both within groups and between groups B. Experienced vs. Novice All POC US Experienced Novice ANOVA = analysis of variance; LOS = length of stay; POC = point of care; RUS = radiology-performed ultrasound. at 3-week phone follow-up and no negative laparotomies in patient who went to the operating room. We stratified test performance characteristics by experienced versus novice sonologists and the results are presented in Table 4 and Figure 5. There were six errors (Figure 5). Only one was a false-negative error due to a novice operator misidentifying small bowel as a normal appendix. This was the sonologist s fifth enrolled patient. The remaining five errors were false positives. Three were misidentified structures: two were normal bowel and one a contracted gallbladder mistaken for appendicitis. The remaining two false-positive errors were 1) due to a borderline normal appendix diameter of 0.59 cm identified as positive and 2) operator inability to differentiate appendicitis from epiploic appendagitis on POC US. DISCUSSION Clinical policy statements by the American College of Emergency Physicians 8 and American College of Radiology 9 recommend that US scanning be considered as the initial imaging test to evaluate children with suspected appendicitis; CT scans contribute significantly to ED LOS. 5 Our study demonstrates that when POC US results contribute to a patient s disposition (discharge home or to the operating room), ED LOS is significantly reduced: by 46% compared to patients with conclusive radiology US results and 68% compared to patients requiring CT scan (Figure 2A). Computed tomography is also a significant source of ionizing radiation exposure in children. It has been estimated that the lifetime risk of fatal cancer from abdominal and pelvic CT in children ranges from 1 in to 1 in 3, CT scans. Thus the risk of fatal cancer from CT scan approaches the risk of death from negative appendectomy (1 in 715) 17 and likely exceeds it if counting nonfatal cancers, which doubles to triples cumulative cancer risk. 4 Sequential imaging protocols using radiology department US first and then CT scan when US is inconclusive have reduced CT rates. 6,7 Implementing sequential imaging protocols with radiology department US followed by CT at our institution decreased CT use from 75.8% to 44.2%. 11 The change from 44.2% to 27.3% that we saw during the study is encouraging, but was not statistically significant. To further avoid CTs, recent data by Bachur et al. 18 suggest a strategy of serial US in children with suspected appendicitis to increase sensitivity over time and thus decreasing nondiagnostic US examinations. The advantage of POC US is the ability to perform serial Table 3 Test Performance of POC US, radiology US, and CT Imaging Modality n Sensitivity,% (95% CI) Specificity,% (95% CI) LR+ (95% CI) LR (95% CI) POC US ( ) 94 ( ) 10.4 ( ) 0.42 ( ) Radiology US ( ) 99.3 ( ) 93.8 ( ) 0.38 ( ) CT ( ) 98.1 ( ) 45 ( ) 0.17 ( ) LR+ = positive likelihood ratio; LR = negative likelihood ratio; POC = point-of-care; US = ultrasound. Table 4 Test Performance Characteristics Experienced Versus Novice Sonologists Test Novice Operator, n=89 Experienced Operator, n=61 Overall, n=150 Radiology US, n=117 Sensitivity,% 51.4 ( ) 80 ( ) 60 ( ) 62.5 ( ) Specificity,% 92.6 ( ) 97.8 ( ) 94 ( ) 99.3 ( ) LR ( ) 36.8 ( ) 10.4 ( ) 93.8 ( ) LR 0.53 ( ) 0.20 ( ) 0.42 ( ) 0.38 ( ) Prevalence rate of appendicitis,% 39.3 ( ) 24.6 ( ) 33.3 ( ) 37.6 ( ) CT scan rate,% 26.9 ( ) 27.9 ( ) 27.3 ( ) 28.2 ( ) % of nondiagnostic scans 71 (62 80) 67 (55 79) 69 (62 76) 59 (50 68) LR+ = positive likelihood ratio; LR = negative likelihood ratio.

6 168 Elikashvili et al. POINT-OF-CARE US FOR PEDIATRIC APPENDICITIS * Figure 5. Diagnostic results for each POC US as performed by pediatric EPs compared with the reference standard. *All scans in this column analyzed in experienced group. 26th and 27th scans in this column were analyzed in the experienced group. All other scans were analyzed in the novice group. and repeated examinations that would make this strategy feasible and merits future investigation. The application of serial US in the evaluation of appendicitis can be included in medical student US education, as well as in emergency medicine resident 23 and pediatric emergency medicine fellowship 24 US training to broaden the base of potential users in the future. After an hour-long focused US training session, our clinicians were able to evaluate for appendicitis with high specificity similar to radiologists (specificity = 94%, 95% CI = 92% to 95%) 2,6,7,18,25 and other clinician-sonologists (specificity 90% to 96%) 1,3 already reported in the literature. Although we had a limited number of errors (Figure 5), they were operator-dependent (five false-positives and one false-negative) and are correctable with additional training and quality assurance. Based on expert review of video, the single false-negative was the result of a novice sonologist not following criteria for identifying a normal appendix. These criteria include identification of appendix from tip to base at the cecum measuring 6 mm throughout, with compressibility. We recommend that novice sonologists (fewer than 25 scans) refrain from making clinical decisions regarding normal appendix without further clinical or imaging confirmation to avoid false-negative errors. 26 Misdiagnosis-related errors in pediatric appendicitis may be minimized by understanding the relationship of operator experience to test performance characteristics. 14,26 Sensitivities between novice and experienced sonologists varied by 28.6% (range = 51.4% to 80%), compared to specificities, which differed by 5.2% (range = 92.6% to 97.8%; Table 4). This 5.5-fold (28.6/ 5.2) difference suggests that operator experience has a greater effect on sensitivity to rule out appendicitis compared to specificity to rule in appendicitis. LIMITATIONS We conducted this study in a single pediatric ED of an academic medical center. Thus our results may not be applicable to other acute care settings, such as freestanding children s hospitals with 24-hour/7-day-a-week radiology services dedicated to children. However, our POC US test performance characteristics are within the range of our own radiology department (Table 2), as well as published test performance characteristics by other children s hospital radiology departments. 6,25,27 We were also limited by a convenience sample of patients presenting to our pediatric ED. Thus the extent to which our sample may fairly represent the entire population of pediatric patients with suspected appendicitis cannot be known. Additionally, potentially eligible patients may not have been enrolled during peak volume time periods. However, our prevalence rate and patient characteristics of suspected appendicitis (Table 1) are similar to those of other published studies. 1 3,6,7,25,27 Measurement of the CT rate during the study may have been susceptible to the Hawthorne effect. We attempted to minimize this effect by keeping novice study sonologists blinded to this study outcome. Additionally, our institutional practice is to obtain a CT scan for definitive diagnosis when US is inconclusive rather than admit for serial abdominal exams. Our estimates for reduction of CT ordering before and during the study were statistically not significant due to a small sample size. This is despite the possibility of the prior CT use estimate being overinflated due to the retrospective method of determining this estimate. However, there were four instances in which CT scans were avoided in pediatric patients with nondiagnostic radiology US results and diagnostic POC US results for a

7 ACADEMIC EMERGENCY MEDICINE February 2014, Vol. 21, No four-patient absolute reduction in CT scans. In our opinion, the opportunity to avoid even a single CT scan in a child is clinically significant. Furthermore, greater reduction in CT utilization may be seen in institutions that have not instituted imaging protocols using US first prior to CT scan, whether performed at the point of care or by the radiology department. 28 Although US is an operator-dependent imaging modality, in our study we employed a mostly novice group of sonologists. They were trained to use a standardized scanning technique of the right lower quadrant and used standard criteria to identify appendicitis versus normal appendix. Our requirement for visualizing appendicitis or normal appendix from tip to cecum (Figure 2) resulted in a trade-off between accuracy and numbers of nondiagnostic POC US results. This became more of a problem for experienced sonologists when incompletely visualizing structures that were possibly appendicitis or, less commonly, normal appendix. To avoid false-negative results, we required that normal appendix be visualized from tip to cecum and also be visualized in perpendicular orthogonal planes on video clips (Figure 3). These factors may have contributed to the high number of nondiagnostic (equivocal) POC US results in our study (Table 4), relative to other data, 29 but makes this study applicable to other institutions looking to incorporate POC US for suspected appendicitis into their practice. Future research is necessary to investigate methods to increase POC US accuracy through more effective educational and training techniques. CONCLUSIONS It may be feasible to reduce ED length of stay and avoid a computed tomography scan when using point-of-care ultrasound to evaluate children with suspected appendicitis. Test characteristics for point-of-care ultrasound have high specificity to rule in appendicitis, similar to radiology ultrasound. Addition of point-of-care ultrasound prior to sequential radiology imaging was safe, without missed cases of appendicitis or negative laparotomies. The authors acknowledge and thank the following sonologists for patient recruitment and enrollment into the study: Lana Friedman, MD, Sylvia Garcia, MD, Brittany Pardue Jones, MD, Charles Murphy, MD, Bret Nelson, MD, RDMS, Audrey Paul, MD, PhD, Louis Spina, MD, Christopher Strother, MD, and Adam Vella, MD. We thank Dr. Swathi Nadlindla Doyle for her assistance with the initial data analysis. We thank Gary Yu, MPH, for statistical assistance. References 1. Fox JC, Solley M, Anderson CL, et al. Prospective evaluation of emergency physician performed bedside ultrasound to detect acute appendicitis. Eur J Emerg Med 2008;15: Doria A, Moineddin R, Kellenberger C, et al. US or CT for Diagnosis of appendicitis in children and adults? A meta-analysis. Radiology 2006;241: Burford JM, Dassinger MS, Smith SD. Surgeon-performed ultrasound as a diagnostic tool in appendicitis. J Pediatr Surg 2011;46: Brenner DJ, Hall EJ. Computed tomography-an increasing source of radiation exposure. N Engl J Med 2007;357: Qureshi A, Smith A, Wright F, et al. The impact of an acute care emergency surgical service on timely surgical decision-making and emergency department overcrowding. J Am Coll Surg 2011;213: Garcia Pena BM, Mandl KD, Kraus SJ, et al. Ultrasonography and limited computed tomography in the diagnosis and management of appendicitis in children. JAMA 1999;282: Santillanes G, Simms S, Gausche-Hill M, et al. Prospective evaluation of a clinical practice guideline for diagnosis of appendicitis in children. Acad Emerg Med 2012;19: Howell JM, Eddy OL, Lukens TW, et al. American College of Emergency Physicians. Clinical policy: critical issues in the evaluation and management of emergency department patients with suspected appendicitis. Ann Emerg Med 2010;55: Rosen MP, Ding A, Blake MA, et al. ACR appropriateness criteria right lower quadrant pain-suspected appendicitis. J Am Coll Radiol 2011;8: Bossuyt PM, Reitsma JB, Bruns DE, et al. Standards for Reporting of Diagnostic Accuracy. Toward complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Br Med J 2003;326: Bentley S, Nelson B, Zahn L. Ultrasound evaluation for appendicitis in children; can we safely reduce CT scan utilization [abstract]. Ann Emerg Med 2010;56(3 Suppl 1). 12. American College of Emergency Physicians. Emergency ultrasound guidelines. Ann Emerg Med 2009;53: Arkin CF, Wachtel MS. How many patients are necessary to assess test performance. JAMA 1990; 263: Newman-Toker DE, Pronovost PJ. Diagnostic errors the next frontier for patient safety. JAMA 2009;301: Brenner DJ, Elliston CD, Hall EJ, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. AJR Am J Roentgenol 2001;176: Larson DB, Rader SB, Forman HP, et al. Informing parents about CT radiation exposure in children: it s OK to tell them. AJR Am J Roentgenol 2007;189: Balthazar E, Rofsky N, Zucker R. Appendicitis: the impact of computed tomography imaging on negative appendectomy and perforation rates. Am J Gastroenterol 1998;93: Bachur RG, Dayan PS, Bajaj L, et al. The effect of abdominal pain duration on the accuracy of diagnostic imaging for pediatric appendicitis. Ann Emerg Med 2012;60: Hoppmann RA, Rao VV, Poston MB, et al. An integrated ultrasound curriculum (iusc) for medical students: 4-year experience. Crit Ultrasound J 2011;3: Rao S, van Holsbeeck L, Musial JL, et al. A pilot study of comprehensive ultrasound education at the

8 170 Elikashvili et al. POINT-OF-CARE US FOR PEDIATRIC APPENDICITIS Wayne State University School of Medicine: a pioneer year review. J Ultrasound Med 2008;27: Bahner DP, Royall NA. Advanced ultrasound training for fourth-year medical students: a novel training program at The Ohio State University College of Medicine. Acad Med 2013;88: Fox JC, Chiem AT, Rooney KP, et al. Web-based lectures, peer instruction and ultrasound-integrated medical education. Med Educ 2012;46: Akhtar S, Theodoro D, Gaspari R, et al. Resident training in emergency ultrasound: consensus recommendations from the 2008 Council of Emergency Medicine Residency Directors Conference. Acad Emerg Med 2009;16(Suppl 2):S Vieira RL, Hsu D, Nagler J, Chen L. Pediatric emergency medicine fellow training in ultrasound: consensus educational guidelines. Acad Emerg Med 2013;20: Mittal MK, Dayan PS, Macias CG, et al. Performance of ultrasound in the diagnosis of appendicitis in children in a multicenter cohort. Acad Emerg Med 2013;20: Tsung JW. Rules for the road: an evidence-based approach to understanding diagnostic test performance of point-of-care ultrasound for pediatric abdominal emergencies. Crit Ultrasound J 2010;3: Puylaert JB, Rutgers PH, Lalisang RI, et al. A prospective study of ultrasonography in the diagnosis of appendicitis. N Engl J Med 1987;317: Russell WS, Schuch AM, Hill JG, et al. Clinical practice guidelines for pediatric appendicitis evaluation can decrease computed tomography utilization while maintaining diagnostic accuracy. Pediatr Emerg Care 2013;29: Schuh S, Man C, Cheng A, Murphy A, et al. Predictors of non-diagnostic ultrasound scanning in children with suspected appendicitis. J Pediatr 2011;158: Supporting Information The following supporting information is available in the online version of this paper: Data Supplement S1. Study flow chart.