HOW TO OBJECTIVELY ASSESS THE PREVENTION DOMAIN. SCOTT L. BRUCE, EdD, ATC FOUNDER, CHATTANOOGA CONCUSSION PREVENTION INITIATIVE

Transcription

1 HOW TO OBJECTIVELY ASSESS THE PREVENTION DOMAIN SCOTT L. BRUCE, EdD, ATC FOUNDER, CHATTANOOGA CONCUSSION PREVENTION INITIATIVE June 25, 2014

2 2 OBJECTIVES Demonstrate how to formulate clinical prediction guides to help identify deficiencies or problem areas that lead to potential injuries Show how to apply the process to your specific practice setting Explain the process of how to utilize the data collected effectively Demonstrate how to formulate a clinical prediction rule (guide)

3 3 QUESTIONS How many injuries did you prevent this past year/season?

4 4 CLINICAL PREDICTION (GUIDES) RULES Clinical Prediction (Guides) Rules have the potential to improve outcomes, increase patient satisfaction & decrease costs of care in (our) practice. Childs et al. Phys Ther. 2006

5 5 CLINICAL PREDICTION GUIDES (CPG) AT is way behind other health related/ rehabilitation professions Literature search for clinical prediction rule & athletic training found two hits 1,2 CPR exist in carpal tunnel syndrome, 3 cervical pain, 4,5 knee dysfunction, 6,7 & low back pain 8,9 1. Wilkerson et al., J Athl Train. 2010; 2. Wilkerson et al., J Athl Train. 2012; 3. Wainner et al. Arch Phys Med Rehabil. 2008; 4. Cleland et al. Phys Ther. 2007; 5. Tseng et al. Man Ther 2006; 6. Lesher et al. J Orthop Sports Phys Ther. 2006; 7. Currier et al. Phys Ther. 2007; 8. Childs et al. Ann Intern Med. 2004; 9. Hicks et al. Arch Phys Med Rehabil. 2005

6 6 CLINICAL PREDICTION GUIDES (CPG) Literature search found no CPG for ACL injuries We need to start developing our own CPGs or other professions will dictate to us how we perform our practice

7 7 GOAL OF CLINICAL PREDICTION GUIDES Intent is to assist clinicians in: decision making establishing a prognosis implementing an intervention

8 8 GOAL OF CLINICAL PREDICTION GUIDES Desire to determine if a set of criteria (i.e., predictors) exist in your cohort to allow you to ID risk factors for injury Once ID ed then the AT can address those at risk patients to prevent injuries Caution should be exercised to generalize a CPG from another population to your population HS athletes are different from intercollegiate athletes who are different from elite/pro athletes Need large scale, multi-setting data sets to create appropriate norms for specific populations

9 9 RECOMMENDATIONS outcome events (injuries) per independent variable 1,2 If gathering data over multiple years, then each athlete is considered 1 subject each year Reason = Subjects level of risk may change from 1 yr to next 3,4 Body type, conditioning, strength, growth etc. 1. Childs et al. Phys Ther. 2006; 2. Wasson et al. NEJM Beynnon et al. AJSM. 2005; 4. McHugh et al. AJSM. 2007

10 10 POTENTIAL PROBLEMS Collinearity occurs when the predictor variables are very highly correlated (r 0.80) 1 Example: Using High vs. Low BP as a variable & diastolic BP (mmhg) as a 2 nd variable Too many predictors can be burdensome 5 or more predictors, then use logistic regression to narrow number of actual predictors 2,3 1. Mertler & Vannetta. Advanced and Multivariate Statistical Methods. 2005; 2. Childs et al. Phys Ther. 2006; 3. Cook. J Man Manip Ther. 2008

11 11 POTENTIAL PROBLEMS Interaction Effects - determines whether or not the odds ratios are constant, or homogeneous, over the strata 1 Occurs when the relationship between variables are linear, but the slopes of the lines differ Represented graphically: if the lines of 2 variables do not intersect = an absences of interaction effect if the lines of 2 variables do intersect = an interaction effect is present 1, 2 1. Hosmer & Lemeshow, Applied logisitic regression. 2000; 2. Portney & Watkins, Foundations of clinical research: Applications to practice 2000

12 12 INTERACTION EFFECTS No Interaction Effect Present Interaction Effect Present

13 13 POTENTIAL PROBLEMS Failure to maintain quality of the tests & measures used as predictors 1 Too few subjects in sample = low statistical power 2 FB is OK for 1 season Other sports need multiple seasons or teams 1. Cook. J Man Manip Ther. 2008; 2. Childs et al. Phys Ther. 2006

14 14 STEPS TO DEVELOP & TEST CPGS 1. Develop the clinical prediction guide 2. Validation of the CPG Apply your CPG to a similar population 3. Assessment of the impact of the CPG Financial impact Time lost Number of injuries Flynn et al. Spine Today s focus will be on the development of CPGs

15 15 SELECTION OF BEST PREDICTION MODEL Based on: Sensitivity Specificity LRs Odds Ratio Relative Risk

16 16 SELECTION OF BEST PREDICTION MODEL Each potential predictor variable evaluated separately (univariable analysis) Each cohort member s value relative to cut-point classified as positive (1) or negative (0) for prediction of criterion status ROC analysis used to establish cut-point for each variable

17 17 SELECTION OF BEST PREDICTION MODEL Combinations of predictor variables evaluated as a set Each cohort member s # of positive factors (sum of 1 values) determined for a set of 2 or more predictor variables ROC analysis used to identify the optimal number of positive factors for discrimination of criterion-positive from criterion-negative cases

18 18 ODDS RATIO Ratio of the odds for a specified outcome (injury) for one group in relation to the odds for another group If the odds are the same for both groups, OR = 1.0

19 19 ODDS RATIO Probability vs. Odds of Heads on coin flips: 1 (single coin flip) =.50 or 1/2 Odds for 1:1 Odds against 1:1 2 consecutive: (.5) 2 =.25 or 1/4 Odds for 1:3 Odds against 3:1 3 consecutive: (.5) 3 =.12 or 1/8 Odds for 1:7 Odds against 7:1 4 consecutive: (.5) 4 =.06 or 1/16 Odds for 1:15 Odds against 15:1 5 consecutive: (.5) 5 =.03 or 1/32 Odds for 1:31 Odds against 31:1 6 consecutive: (.5) 5 <.02 or 1/64 Odds for 1:63 Odds against 63:1

20 20 ODDS RATIO Odds and horse racing 2:1 horse or 50:1 horse? Betting to win 2:1 better than the long shot of 50:1 2:1 injury or 50:1 injury Betting to lose 50:1 says you are more likely to get injured than someone at 2:1

21 21 RELATIVE RISK The likelihood that someone who has been exposed to a risk factor will develop the injury as compared to someone who does not have the risk factor If the probability is the same for both groups, RR = 1.0

22 22 PROCESS FOR DEVELOPING CPGS What is it that you wish to examine for a CPG? ID cohort Operationally define what you desire to examine Ex: How do you define an injury? Brainstorm for all possible factors/variables

23 23 FACTORS AFFECTING INJURY RISK, RECOVERY RATE, AND/OR SUSCEPTIBILITY TO RE-INJURY Age Gender Sport / Social Role Injury History Joint Stability Strength Fatigue Resistance Aerobic Fitness Neuromuscular Control Agility Swelling Volume Neurocognitive Locations of Tenderness Capabilities ROM Restriction Psychosocial Co-morbidities Characteristics Body Composition Social Support

24 UTC S EXPERIENCES (PRE-SEASON ASSESSMENT OPTIONS) Foot & Ankle Assessment Measure Int. Knee Doc. Comm. (IKDC) Survey Oswestry Disability Questionnaire Body Mass Index Navicular Drop Q-Angle Hip ER & IR ROM Shoulder ER & IR ROM Sit & Reach 1.5 Mile Run Time Vertical Jump Hand-Held Dynamometer Force Triple Hop for Distance Unilateral Zig-Zag Hop Test Tuck Jump Assessment Balance Error Scoring System Cogsport Neurocognitive Test ImPACT Neurocognitive Test Back Extension Hold Side Bridge Hold Trunk Flexion Hold Wall Sit Hold Isokinetic Peak Torque Horizontal Trunk Hold

25 25 CPG DERIVATION The number and variety of predictors required to create a meaningful CPG will vary Typically 3 to 5 variables Only variables that are relatively easy to measure with adequate precision should be considered as predictors Must not be time-consuming, complex, or expensive UTC 3 fatigue resistance tests & 3 surveys = 10 mins.

26 26 PROCESS FOR DEVELOPING CPGs Pre-season (starting from scratch) Run baseline tests for variables Determine the median score for each variable Dichotomize cases and classify as zeros (0) & ones (1) 1 = at risk/high risk; 0 = low risk Ex: Wall Sit Hold (WSH) median = 63 sec. 1 = 63 sec.; 0 = > 63 sec. All 1 s are in need of a remedial program Proportion at risk = At risk / Low Risk

27 PROCESS FOR DEVELOPING CPGs (CONT.) Do you have a math phobia? Can you count? Can you do simple arithmetic? Add? Subtract? Multiply? Divide?

28 28 PROCESS FOR DEVELOPING CPGs (CONT.) Here is an example WSH Time Subjects 63 sec. 15 > 63 sec. 81 Proportion at risk = At risk / Low Risk or Proportion at risk = 63 sec. / > 63 sec. Proportion at risk = 15 / 81 Proportion at risk = 5.4 So an athlete that has a time of 63 sec. on the WSH has 5.4 X s greater odds of getting injured than the odds for someone w/ a time of > 63 sec

29 29 PROCESS FOR DEVELOPING CPGs (CONT.) Can do the same for the other variables Median score for fatigue resistance tests For surveys rank order scores, (best to worst) calculate the lower 25% scores (those with the worst scores) Remember for surveys: Oswestery Low Back Survey high score is bad FAAM & IKDC high score is good Those below this point are at risk

30 30 PROCESS FOR DEVELOPING CPGs (CONT.) Add up how many variables each individual is positive on Fatigue Resistance Tests Surveys Name/Test WSH TFH HTH ODI FAAM IKDC Total SLB BA JT GBW

31 31 PROCESS FOR DEVELOPING CPGs (CONT.) Re-run the same steps to obtain the proportion at risk + Factors Subjects 3 12 < 3 84 Proportion at risk 12/84 = 7.0 Someone w/ 3 factors has 7 X s greater odds to be injured than the odds for someone w/ < 3 factors + Factors Subjects 2 18 < 2 78 Proportion at risk 18/78 = 4.3 Someone w/ 2 factors has 4.3 X s greater odds to be injured than the odds for someone w/ < 2 factors

32 32 PROCESS FOR DEVELOPING CPGs (CONT.) Post-season analysis Determine what injuries are to be examined LE, UE, Knee injuries, Shoulder injuries Define injury NCAA Definition injured in athletics, assessed by ATC or dr., missed at least 1 practice Overuse injuries injured in athletics, assessed by ATC or dr. & rec d tx for 2 or more days for injury Re-run the same stats but now count injured vs. non-injured 1 = Injured; 0 = Not injured

33 33 PROCESS FOR DEVELOPING CPGs (CONT.) 1 = Injured; 0 = Not injured Subjects Injured 23 Not Injured 73 Reclassify, based on WSH results Injury No Injury 63 sec. TP FP > 63 sec. FN TN Total

34 34 PROCESS FOR DEVELOPING CPGs (CONT.) Reclassify, based on WSH results WSH Time Subjects 63 sec. 15 > 63 sec. 81 Subjects Injured 23 Not Injured 73 Injury No Injury Total 63 sec > 63 sec Total

35 LET S DO SOME SIMPLE MATH Sensitivity (Sn) = TP / TP + FN Specificity (Sp) = TN / TN + FP OR = (TP)(TN) / (FP)(FN) TP FN FP TN RR = TP/TP + FP FN / FN + TN

36 36 LET S DO SOME SIMPLE MATH WSH Injury No Injury Total < 63 sec sec Total Sensitivity: 0.56 Specificity: 0.97 Odds Ratio: Relative Risk: 7.02 Percent Correct = TP + TN / Total Subjects % correct = / 96 = 84 / 96 % correct = * 100 = 87.5%

37 37 CPG DERIVATION Factors that influence the accuracy of a prediction model: Number of subjects & prevalence of the condition within cohort Should be at least outcome events (injuries) per independent variable 1,2 Magnitude of effect associated with selected variables Reliability & validity of methods used for measurement 1 Childs et al. Phys Ther. 2006; 2 Wasson et al. NEJM. 1985

38 FOOTBALL CORE/LE INJURY ANALYSIS Core Endurance Tests Wall Sit Hold Unilateral (WSH) WSH Dominant (stronger) 1 st Non-Dominant (weaker) 2 nd o WSH Mean = 28; SD = 14 Horizontal Trunk Hold (HTH) Kneeling back extension o HTH Mean = 49.32; SD = No test-retest reliability data Trunk Flexion Hold (TFH) Arms overhead (palms forward) o TFH Mean = ; SD =

39 FOOTBALL CORE/LE INJURY ANALYSIS Joint Function Surveys: Oswestry Disability Index International Knee Documentation Committee (IKDC) Survey Foot and Ankle Ability Measure

40 COMBINED ANALYSIS WALL-SIT HOLD N=257 WSH 0.58 WSH 56 sec AUC = 0.55 Injury No Injury 56 sec > 56 sec Total Fisher s Exact One-Sided p = Sensitivity: 0.57 Specificity: 0.54 Odds Ratio = (59*83) / (71*44) 4897 / 3124 = % CI: Relative Risk: 59 / (59+71) / 44 / (44+83) / = % CI:

41 COMBINED ANALYSIS OSWESTRY DISABILITY INDEX N=257 Core + LE Strains & Sprains Injury No Injury ODI 4 pts ODI 2 pts AUC = points < 4 points Total Fisher s Exact One-Sided p =.003 Sensitivity: 0.36 Specificity: 0.81 Odds Ratio = (37*124) / (30*66) 4588 / 1980 = % CI: Relative Risk = 37 / (37+30) / 66 / (66+124) / = % CI:

42 COMBINED ANALYSIS GAMES AS STARTER N=257 Starter 1 Game Core + LE Strains & Sprains Injury No Injury 1 game None Total Fisher s Exact One-Sided p <.001 Sensitivity: 0.61 Specificity: 0.72 AUC =.66 Odds Ratio = (63*111) / (43*40) 6993 / 1720 = % CI: Relative Risk = 63 / (63+43) / 40 / (40+111) / = % CI:

43 COMBINED ANALYSIS 3-FACTOR PREDICTION MODEL N=257 1) Starter ( 1 game) 2) Hi ODI ( 4) 3) Lo WSH-Z ( 0.10) Core + LE Strains & Sprains (78) 3 2 Sensitivity: 0.52 Specificity: 0.80 Odds Ratio: 6.87 Sensitivity: 0.11 Specificity: 0.98 Odds Ratio: Sensitivity:.91 Specificity:.29 Odds Ratio: 4.18 AUC =.70 3-Factor Model Injury No Injury 2 Factors or 1 Factor Total Fisher s Exact One-Sided p <.001 Sensitivity: 0.58 Specificity: 0.84 Odds Ratio = (60*128) / (26*43) 7680 / 1118 = % CI: Relative Risk = 60 / (60+26) / 43 / (43+128) / = % CI:

44 COMBINED ANALYSIS N=257 Risk Factors Injury No Injury Incidence % % % % Total %

45 COMBINED ANALYSIS N=257 Starter 1 Game AND Hi ODI Injury No Injury Both Factors or 1 Factor Total Fisher s Exact One-Sided p <.001 Sensitivity: 0.22 Specificity: 0.94 Odds Ratio = (23*144) / (10*80) 3312 / 800 = % CI: Relative Risk = 23 / (23+10) / 80 / (80+144) / = % CI: Starter 1 Game AND Lo WSH Injury No Injury Both Factors or 1 Factor Total Fisher s Exact One-Sided p <.001 Sensitivity:.31 Specificity:.90 Odds Ratio = (32*139) / (15*71) / 1065 = % CI: Relative Risk = 32 (32+15) / 71 / (71+139) / = % CI:

46 COMBINED ANALYSIS N=257 46

47 COMBINED ANALYSIS N=257 Hi Risk = Starter 1 Game and EITHER Hi ODI OR Lo WSH Injury No Injury Hi Risk or 1 Factor Total Fisher s Exact One-Sided p <.001 Sensitivity: 0.43 Specificity: 0.86 Odds Ratio = (44*132) / (22*59) 5805 / 1298 = % CI: Relative Risk = 44 / (44+22) / 59 / (59+132) / = % CI:

48 COMBINED ANALYSIS N=257 Hi Risk = Starter 1 Game and BOTH Hi ODI AND Lo WSH Injury No Injury Hi Risk to 2 Factors Total Fisher s Exact One-Sided p =.003 Sensitivity: 0.11 Specificity: 0.98 Odds Ratio = (11*151) / (3*92) 1661 / 276 = % CI: Relative Risk: 11 / (11+3) / 92 / (92+151) / = % CI:

49 49 SO NOW WHAT? So if there are 16 ID ed at risk athletes pre-season & they are placed on a remedial program End of season review number of injuries to at risk athletes 4 injured, 12 not injured Argument can be made that you helped to prevent 12 injuries

50 IDEAS FOR REMEDIAL EXERCISE 50

51 Bridging on a physio-ball (Note the arms) Start with a simple bridge When stable, use leg mvts.

52 Another good starting pt is w/ quadruped exercises Then on a prone plank w/ variations Increase difficulty of quadruped ex Superman exercises

53 53 Seated physio-ball routine w/ increased difficulty Can do same exercises but incorpora te arm mvts

54 54 From seated to supine, similar progressions Note: Hips dropped, must keep hips up & body parallel to floor Note: Unstable object

55 UE Exercises 55

56 56

57 57 Resistive Walking Resistive Step & Step Resistive walking All 3 planes, control mvt away & towards anchor Walking Running uphill (only walk downhill)

58 Advanced Exercises

59 TWEAKOLOGY Mvt Tweaks Dimension Tweaks Influence Tweaks Plane Tweaks Time Tweaks Control Tweaks Range Tweaks Repetition Tweaks Load Tweaks Position Tweaks Distance Tweaks Equipment Tweaks Joint Tweaks Sequencing Tweaks Feedback Tweaks Gary Gray, 1995

60 60 BOTTOM LINE Our job is to help strengthen the athlete s weaknesses and prevent him/her from suffering an injury

61 61 THANK YOU! 423/ Jeremiah 29:11