Levels of Evidence for Foot and Ankle Questions on the OITE: 15-Year Trends Cory T. Walsh, Louis C. Grandizio DO, John Parenti MD, Gerard J. Cush MD GEISINGER ORTHOPAEDIC INSTITUTE 1
AOFAS Disclosure Statement No conflict to disclose Levels of Evidence for Foot and Ankle Questions on the OITE: 15-Year Trends Cory T. Walsh MD My Disclosure is in the Final AOFAS Mobile App I have no conflicts with this presentation 2
BACKGROUND There has been a trend towards evidence based medicine (EBM) in the orthopaedic surgery literature. Levels of Evidence (LoE) for articles in peer reviewed journals have improved in the past decade. The OITE functions as a yearly indicator of orthopaedic resident knowledge and has been found to correlate with performance on ABOS Part I Examination There is a paucity of literature pertaining to LoE supporting OITE questions. 3
PURPOSE & METHODS PURPOSE Determine if LoE for primary journal articles referenced within FA content domain on OITE have increased over 15 year period Determine if both characteristics and taxonomy have changed during this same time period METHODS All questions in FA content from 1995-1997 and from 2010-2012 were analyzed. Omitted questions excluded from official OITE 4
METHODS Data collection from OITE score reports included: Year of publication, journal, source type (primary journal article, review article, ICL, textbook) For each primary journal we documented the LoE for each using AAOS Levels of Evidence Guidelines Taxonomy Using Buckwalter Classification questions were assigned T1, T2, or T3 T1 recall basic facts T2 interpret an imaging study or make a diagnosis T3 treatment and management provided clinical situation 5
BASELINE CHARACTERISTICS Overall 1995-97 2010-12 P-value Total number of included questions, N 132 78 54 -- Buckwalter Classification.0286 T1, N (%) 39 (29.5%) 29 (37%) 10 (18.5%) T2, N (%) 25 (19%) 15 (19%) 10 (18.5%) T3, N (%) 68 (51.5%) 34 (44%) 34 (63%) Number of references per question <.0001 1, N (%) 24 (18%) 24 (31%) 0 (0%) 2, N (%) 88 (67%) 46 (59%) 42 (78%) 3, N (%) 19 (14%) 7 (9%) 12 (22%) 4, N (%) 1 (1%) 1 (1%) 0 (0%) Types of reference per question At least 1 primary journal article, N (%) 64 (48%) 34 (44%) 29 (54%).1762 At least 1 journal review article, N (%) 55 (42%) 28 (36%) 27 (50%).1061 At least 1 textbook, N (%) 76 (58%) 51 (65%) 25 (46%).0291 At least 1 ICL, N (%) 3 (2%) 2 (3%) 1 (2%).7871 6
RESULTS 78 questions from 1995-1997, 54 questions from 2010-2012 FAI was the most frequently cited source overall TAXONOMY Change in taxonomy distribution was statistically significant 34/54 were level T3 7
RESULTS Levels of Evidence 6/54 (11%) of FA questions from 2010-2012 were Level I or II, while 3/78 (4%) from 1995-1997 were Level I or II (p level =.1035) 8
RESULTS Comparison of the LoE for questions in each time period 1995-97 OITE (n=78) 2010-12 OITE (n=54) P- value Questions in each time period that cited level I-II evidence, N.1035 (%) 3 (4%) 6 (11%) Yes 75 (96%) 48 (89%) No Questions in each time period that cited level I-III evidence, N 5 (6%) 8 (15%).1111 (%) 73 (94%) 46 (85%) Yes No 9
CONCLUSIONS There has been a trend towards creating OITE questions supported by higher LoE, which reflects the improved LoE in orthopaedic FA literature Increase in level 1 studies in comparing 1995-1997 to 2010-2012 Change in question taxonomy with increased emphasis on clinical management questions compared to years past FAI was more frequently cited source 10
CONCLUSION Summary Our results can be used to help improve resident self-study and suggest that reviewing recent FAI articles may aid OITE preparation. 11
References Barr CR, Cheng I, Chou LB, et al. Foot and ankle questions on the orthopaedic in-training examination n: analysis of content, reference, and performance. Orthopedics.2012;35(6),472. doi: 10.3928/01477447-20120525-28. Barske HL, Baumhauer J. Quality of research and level of evidence in foot and ankle publications. Foot Ankle Int, 2012;33(1):1-6. doi: 10.3113/FAI.2012.0001. Bernstein J, Kenniston JA, Nydick JA, et al. Levels of evidence are low for clinical management questions on the orthopaedic in-training examination. J Bone Joint Surg Am. 2010;92:508-511. doi: 10.2106/JBJS.I.00530. Buckwalter JA, Schumacher R, Albright JP, et al. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ. 1981;56:115-121. Dougherty PJ, Walter N, Schilling P, et al. Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clin Orthop and Relat Res. 2010; 468(10):2797-2802. doi: 10.1007/s11999-010-1327-3. Hanzlik S, Mahabir RC, Baynosa RC, et al. Levels of evidence in research published in the journal of bone and joint surgery (american volume) over the last thirty years. J Bone Joint Surg Am. 2009; 91:425-428. doi: 10.2106/JBJS.H.00108. Haughom BD, Goldstein Z, Hellman MD, et al. An Analysis of References Used for the Orthopaedic In-Training Examination: What are Their Levels of Evidence and Journal Impact Factors? Clin Orthop and Relat Res. 2014; 472:4024-4032. doi: 10.1007/s11999-014-3895-0. Mankin HJ. The orthopaedic in-training examination (OITE). Clin Orthop Relat Res. 1971;75:108-116. Miyamoto RG, Klein GR, Walsh M, et al. Orthopedic surgery residents study habits and performance on the orthopedic in-training examination. Am J Orthop. 2007; 36(12):e185-188. Razi Z, Abbassian A, Cro S, et al. Levels of evidence in foot and ankle surgery literature: progress from 2000 to 2010? J Bone Joint Surg Am. 2012;94:e112-1. doi: 10.2106/JBJS.K.01453. Sheibani-Rad S, Arnoczky SP, Walter NE. Analysis of the basic science section of the orthopaedic in-training examination. Orthopedics. 2012; 35(8), 1251-1255. doi: 10.3928/01477447-20120725-28. Srinivasan RC, Seybold JD, Kadakia AR, Analysis of the foot and ankle section of the Orthopaedic In-Training Examination. Foot Ankle Int. 2009;30(11):1060-1064. doi: 10.3113/FAI.2009.1060. Thordarson DB. Levels of evidence, evidence-based medicine, and Foot & Ankle International. Foot Ankle Int. 2008; 9(9):881-882. doi: 10.3113/FAI.2008.0881. 12