I have a clinical question how could I research the answer. Evidence-Based Practice. Standards of Care are Good!

Transcription

1 Evidence-Based Practice Accessing/Understanding the Current Literature Ed Mulligan, PT, DPT, OCS, SCS, ATC Clinical Orthopedic Rehabilitation Education Evidence-Based Practice Accessing/Understanding the Current Literature Ed Mulligan, PT, DPT, OCS, SCS, ATC Clinical Orthopedic Rehabilitation Education I have a clinical question how could I research the answer Evidence-Based Practice Sackett s Definition The conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of patients Why is it important? integration of evidence into our clinical decisionmaking should enhance outcomes and reduce unwarranted variation in practice Standards of Care are Good! Best practice guidelines for clinical care should be a clinician s obligation to consider but not necessarily supersede the other tenets of evidence based care IMO, it s better to be consciously ineffective than to be unconsciously ineffective What are the Elements of Evidence Based Practice? VALUES Patient Expectations BEST EVIDENCE Literature Resources EXPERIENCE Provider Expertise 1

2 What evidence practice is NOT What evidence practice is NOT A cookbook approach Evidence is always tempered by the patient s unique circumstance and needs A replacement for experience and expertise Solely a reflection of RCTs and meta-analyses A cost-cutting measure Hopefully, it identifies both efficacy (does it work) and efficiency (how well it works) Guide to Physical Therapist Practice This is an example of expert consensus, not evidence based information What Evidence is Data/information which tends to prove or disprove a construct Data/information ti that t makes something more plain or clear and, the best evidence is free of bias Unfortunate evidence perspectives Copy from one, it's plagiarism; copy from two, it's research; copy from three and it s a fact of practice The plural of anecdotes is evidence (if enough people think it or say it it must be true) In God we trust everyone else must bring data The problem with expert opinion you are often bestowed guru status The Journal of Mental Manipulation s most recent issue reported on the clinical superiority of a new manual therapy technique referred to as tactile hallucination. Rehabilitation clinicians are flocking to continuing education programs to learn this exciting new technique which appears to be applicable for conditions ranging from hangnails to hernias. Only expert clinicians trained abroad are qualified to teach these techniques at resort locations during the work week. Many physicians are prescribing this therapeutic technique based on the testimonials from their patients. Scientific investigation of the technique have proven to be unnecessary because of the fantastic results reported through infomercials, reputable media outlets such as the advertisement section in the back of Runner s World, Daytime talk shows and People magazine. For more information on this exciting development in rehabilitation, visit the originator s web site ( or watch their infomercial airing on Channel 76 (WDUH) at 4:30 am on Saturday, April 1 st. Without apology I am not a Guru as evidenced by the John Child s guru litmus test on his Evidence in Motion blog o I am not a great salesman o Personable, charming, charismatic, and persuasive o I will not offer miraculous or cure-all approaches o I do not have any proprietary language o I m still trying to come up with an original idea o You are not my students (but my colleagues) o I do not have any techniques named after myself I m not that Mulligan o I try to apply EBP materials o What has been Proven, Not Proven, & Proven Not 2

3 What is the public s attitude about utilization of EBP? You just have to be very inquisitive, he said. The physical therapist should be able to explain the various treatment options. You should ask about the benefits and risks, and ask what is the evidence that it will work. And if the therapist can t give you good answers, he added, you might want to rethink your choice of therapist. EBP Attitudes and Utilization Our actions are so loud maybe the medical community and public can t hear the words we re saying The days of assuming financial risk for our outcomes are imminent Here s some anecdotal evidence regarding our attitudes What is the attitude towards EBP? Overflowing auditorium audience necessitating a repeat lecture the next day for Barefoot Running Less than 50 people in attendance in a nearly empty auditorium for a lecture titled Attitudes and Beliefs of Physical Therapists about Evidence-Based Practice: How Do We Facilitate the Translation of Evidence into Practice Behaviors? 90% necessary 90% don t think it causes unreasonable demand 80% useful 80% improves care 70% assists decision making Jette, et al, Phys Ther 2003 What type of therapist uses EBP? Characteristics that best predicted practitioners that employ EPB Desire for learning Higher academic degree Value practicality and non-conformity Variables that had a negative correlation with use of EBP Older age and increased years of practice Decreased % of time in direct patient care It s time to stop studying our habits and attitudes about EPB and time to start doing it. Gerald Brennan, CSM, 2011 Bridges PH, et al, BMC Health Services Research,

4 It starts with reading to find the evidence Journal Reading Habit Survey of 544 clinicians from a large corporate outpatient setting read an average of 2.44 journal articles/month Distribution was skewed to the left Measure of central tendency hovered around 1 Mode and median = 1 68% read 0 or 1 articles/month 7% read 6 or more articles/month Clinicians self rating of research expertise Clinicians self rating of research expertise On a scale of 0-5 with 0 representing no expertise and 5 representing utmost competency, how would you rate your current knowledge and exper % % % 2 Mean = % % Rating % % Novice Expert 5 0 Clinicians self rating of comfort with researching a clinical question to ensure an evidence-based perspective Clinicians self rating of comfort with researching a clinical question to ensure an evidence-based perspective On a scale of 0-5 with 0 representing no expertise and 5 representing utmost competency, how would you rate your comfort in researching a clini % % 3 1 Mean = % 15.7 % Rating % % Novice % % Expert 4

5 What is the propensity to using the evidence? year-old clinicians are: (as compared to > 50 YO) ~ 20x more likely to have learned the foundations of EBP in their entry-level professional program ~ 10-11x more likely to be familiar with on-line databases and trained in search strategies ~ 22x more likely to have had formal training in critical appraisal ~3-4x more likely to be confident in search and analysis skills Jette, et al, Phys Ther 2003 What is the biggest barrier to EBP? Jette, et al, Phys Ther 2003 Not just in the U.S. Biggest barriers to implementation of EBP Time required to stay current Access to the evidence Lack skills to find and understand the evidence 70% of clinician respondents read the research literature at least monthly 10, 15, 25%, respectively, searched PEDro, Cochrane and Medline or Cinahl databases frequently Iles, et al, Physiother Res Int, 2006 Here s one final problem with EBP Evidence is of no value if it is not found, consumed, integrated, and utilized in the assessment and treatment of our rpatients It has been estimated that it takes 15 years for empirical data to permeate and become common general practice Possible Solution Therapists are willing to integrate new information and change clinical practice if information is synthesized and presented to them Journal Clubs with CATs Study reported familiar problems with integrating evidence Lack of time Difficulty accessing material Inadequate research in specific areas Fruth SJ, et al, Physiother Theory Pract, 2010 Is it possible to promote EBP? 5 Pediatric PTs provided didactic training to identify and implement EBP Able to implement some (but notall) strategies and modest (self reported) improvements in EBP behaviors and demonstrate a positive attitude towards EBP Reported barriers were lack of time, other influences on clinical decision making, and lack of incentive Schreiber J, et al, Phys Ther,

6 It s time to make a change We re making progress on understanding what EBP is but that s not enough We need to start making sure our behavior reflects it Let s not let evidence based practice be an oxymoron in our profession True education occurs when the recipient finishes an article (or course of study) with an inquisitive perspective and goes back to the clinic with the questions. how will I implement the changes suggested how will I evaluate it s value a renewed commitment to consulting the evidence to guide their care Change is good you go first! Being old experienced means you have invested a lot of time and effort in perfecting your logic and rationale on interventions that have now been proven wrong How to Use Evidence Use the evidence like a lamp post for illumination not for support Use the evidence to not only support your paradigm but to challenge it To effectively apply evidence in practice a clinician must: Identify gaps in your knowledge Formulate clinically relevant questions (PICO) Conduct an efficient literature search Apply the rules of evidence (including a hierarchy of evidence) to determine the validity of studies Apply the literature findings appropriately to the patient s problem(s) Appreciate how the patient s values affect the balance between potential advantages and disadvantages of the available management options, and then appropriately involve the patient in the decision. 6

7 10 general areas where clinical questions arise 1. Clinical Findings 2. Etiology (causes) 3. Clinical Manifestations of the injury/disease 4. Differential Diagnosis 5. Diagnostic Tests 6. Prognosis (clinical course and complications) 7. Therapeutic Interventions (worth cost and effort?) 8. Prevention (modify risk factors) 9. Experience and Meaning (how patients experience treatment and effect on therapy) 10.Self-improvement (better, faster, smarter, etc) Deciding Which Question is the Most Important to Ask Step 1 Framing the Question Which question is the most important to the patient s well being? Which question is most feasible to answer within your time constraints Which question will you most likely encounter over and over in your practice? Which question interests you the most? Framing the Question in a Patient Specific Context Begins by asking a good question Deciding if we need background or foreground information Framing the Question A background indicates a need for General information What is this? How do you diagnose it? How do you treat it? Assessed by Review Articles Evidence-based textbooks 7

8 Framing the Question A foreground question indicates a need for Specific or complex clinical queries Patient centered questions involving the interpretation and consideration of risk vs. benefit for a specific diagnostic, prognostic, or intervention for a particular case Assessed by Pre assessed, critically reviewed studies from the literature as high on the evidence hierarchy as possible Systematic Review of RCTs Multiple RCTs RCT Systematic Review of Below Observational Cohort or Case Control Series Case Reports Expert Opinion Unsystematic Clinical Observation Another Example of Evidence Quality Hierarchy CEBM s Grading Scheme to Recommend Utilizing the Evidence in Practice Level of Evidence Description Evidence supported by high quality randomized controlled trials Evidenced supported by lower quality randomized controlled trials or prospective comparative studies Evidence supported by retrospective comparative or case-controlled designed studies Evidence obtained from case studies Consensus - Agreement by an expert panel, in the absence of scientific evidence in the literature, based on experience and/or assumptions in the literature Grade Rationale Consistent evidence from level I RCT studies A Strong recommendations for or against a perspective can be conclusive made with this grade B Evidence consistent with level II and/or III studies acceptable Fair level of confidence in making a recommendation or decision C suggestive D or I weak Conflicting evidence or evidence from level 4 studies Weak confidence in decision making Insufficient evidence to make a decision Orthopedic Section Practice Guidelines Neck Pain: Clinical Practice Guidelines from the Orthopedic Section of the APTA J Orthop Sports Phys Ther 2008; 38(9):A1-A34 GRADES OF RECOMMENDATION based on: A B C STRENGTH OF EVIDENCE Strong Evidence A preponderance evidence obtained from multiple high quality (conclusive) RCTs, prospective p, or diagnostic studies Moderate Evidence (acceptable) Weak Evidence A single high level RCT or multiple lesser quality RCTs A single lower quality RCT or evidence from multiple casecontrolled, retrospective, or case series studies D Conflicting Evidence Higher quality studies that disagree on findings/conclusions. E Theoretical/Foundational Evidence A preponderance of evidence is from animal or cadaveric studies, from conceptual models, or from basic science research F Expert Opinion Best practice based on the clinical experience of content experts 8

9 Summary Determinants for Grading Clinical Evidence Study Design determines Level of Evidence determines Strength of Recommendation Frame the Question as a Patient Problem or Population of Patients Who is the patient? Disease or injury Chief complaint Age Gender Race Health status Current Co-morbidities Previous History Intervention What do you want to do for the patient? Diagnose Identify the source of the problem Prognose Predict outcome Intervene Recommend a product, procedure, or treatment Comparison To what do you what to compare your diagnosis or intervention? What are the alternatives? Control Placebo Different Treatment Different Pathology Outcome What specific result do you want to accomplish, improve, or affect in some measurable way? Accurate diagnosis i Change in pain Change in patient s perception of status Change in functional outcome measure 9

10 PICO Acronym Diagnostic Example PICO Acronym Intervention Example PICO Element Patient/Problem Intervention Comparison Outcome Tips Ask who is this patient? How would I describe this patient to a colleague? What is the common condition or disease you re interested in? Balance precision with brevity. What do I want to do for this patient (treat, diagnose, prognose)? What kind of pathology may be present? Be as specific as possible If not compared to a control group, what would be a likely alternative to the treatment or alternate diagnosis are you considering? (placebo, different form of therapy, medication, surgery) What are the relevant outcomes? What do I hope to change? Is it measurable? Don t just say more effective quantify how the intervention will specifically be more effective. Example 58-year old recreational male tennis player Uncover likely source(s) of shoulder pain Does this patient have a SLAP lesion, RC tear, or SAIS? How will my intervention recommendation be affected by the diagnosis PICO Element Tips Example Ask who is this patient? How would I describe this 58-year old patient to a colleague? What is the common condition or recreational male Patient/Problem disease you re interested in? Balance precision with tennis player with brevity. SAIS. Intervention What do I want to do for this patient (treat, diagnose, prognose)? What kind of treatment am I considering? Be as specific as possible Nitro-dur patch Comparison Outcome If not compared to a control group, what would be a likely alternative to the treatment you re considering? (placebo, different form of therapy, medication, surgery) What are the relevant outcomes? What do I hope to change? Is it measurable? Don t just say more effective quantify how the intervention will specifically be more effective. Placebo control group Pain on a numerical pain rating scale or DASH functional outcome score Who would like to ask a clinically relevant question in a PICO format? Why is a PICO helpful? Focuses your question Facilitates your search Hones the results Makes you time efficient Execute Search Here s the evidence I need for decisionmaking Produces a couple of high quality articles that seem to address my PICO 10

11 Even easier. /ask/ask.php?from=tbld Results /ask/ask.php?from=tbld We have our question how can we find answers? Resources Database vs. Search Engine Database is a collection of stored articles Example: Medline Search Engine: Program designed to search the database Example: PubMed and Ovid search Medline database Where can I find evidence? Search PubMed 11

12 Here s what it looks like This database is housed with the NLM and is based on Medline (Index Medicus) Premier medical data base in the world 4600 refereed journals 12 million citations dating back to 1966 the FREE!!! internet version of Medline Note the links for help, an overview, or a tutorial PubMed On-line Tutorials Noteworthy Items Search limits Brackets indicate non-english Search terms Search executed by clicking go Search Results Page icon indicates abstract available Check box to save or print desired articles Boolean Search Terms Here s the URL address Boolean operators typed in caps can narrow the search AND (pathology AND rehabilitation) used when each citation ti contains all the search items OR (diagnosis AND sensitivity OR specificity) used when each citation contains at least one of the terms NOT (pathology NOT surgery) Cuff used to exclude a given category suffixes * - TRUNCATION (tend*) used for variable suffixes Rotator Using NOT will eliminate all results associated with that key word NOT Surgery 12

13 Example: Search Parameters using Clinical Queries Results: sorted by date of publication Search Terms: Subacromial Impingement Syndrome Category: Diagnosis Scope: narrow, specific search most recent Checked citations Now that you have your information how can you stay current? Click on hyperlink to go to abstract Indicates full-text reprint available Type in your search term(s) and click Save Search Register for an account or fill in your user name and password 13

14 Check yes that you want to receive alerts on this search topic and indicate when you d like to receive; then click OK. Review and Update your Saved Searches That s it! Alternate Search Sites: Results: can sort by date or relevance Another Alternate Search Site is 14

15 Results Other On-line Search Services Cumulative Index of Nursing and Allied Health Literature CINHAL Premier database for nursing and allied health 2800 journals containing 1,000,000 abstracts dating back to 1982 Just $20/year for unlimited on-line access or free for APTA members through Open Door portal Systematic Review Site COCHRANE Type in your search term 15

16 Physiotherapy EvidenceDatabasero PEDRO Type in your Search Terms PEDro Search Results search continued - PEDro Scale Scale considers the aspects of clinical trial quality: internal validity and whether the study contains sufficient statistical information to make it interpretable Eligibility criteria defined (no point value) Random allocation Concealed allocation Baseline comparable Blind Subjects Therapists Assessors Adequate Follow-up Intention to Treat Analysis Between Group Comparisons Point estimates of variability 16

17 APTA APTA Hooked Website "grassroots" effort to develop a database containing current research evidence on the effectiveness of physical therapy interventions the project was motivated by concern that clinicians lacked access to the knowledge available from current research, thus hindering evidence-based practice Website s Objectives Allow members to search a database of article extractions relevant to the field of physical therapy to build support for evidence-based practice Allow members to contribute extractions of the peer-reviewed literature to the database List useful web resources and other information consistent with evidence-based practice Disseminate clinical practice guidelines based on systematic reviews of the literature Can search by keyword, ICD-9 code, or clinical scenario 6300 Currently has about 5600 extractions Example Evidence in Practice Feature No longer available been replaced by The Bottom Line and Podcasts 17

18 Podcasts Updates on timely legislative and professional issues and in-depth educational presentations from national meetings Philadelphia Panel PRACTICE GUIDELINES October 2001 Listen to audio versions of journal articles, abstracts, discussions, debates, etc. Join a section of the APTA Orthopedic Section Journal of Orthopedic and Sports Physical Therapy Neurologic Section Journal of Neurologic Physical Therapy Sports Section - JOSPT, Journal of Sports Health, International Journal of Sports Physical Therapy Acute Care Section Journal of Acute Care Physical Therapy Cardiopulmonary Section Cardiopulmonary Physical Therapy Journal Geriatrics Section- GeriNotes, Journal of Geriatric Physical Therapy Oncology Section - Rehabilitation Oncology Pediatric Section Pediatric Physical Therapy Section on Health Policy and Administration - HPA Journal Aquatics Section Journal of Aquatic Physical Therapy Women's Health Section Journal of Women s Health Physical Therapy My local UTSW Med Center library link LIBRARY Your Local Medical Library As a CI supervising student affiliates Do you have access to the school s medical library or on-line resources? COLLEAGUES Sneak into a Library If you don t know? ASK 18

19 Using my sons username and password Other On-line Sources Stanford University Library s access to 800 journals and 1,000,000+ free articles l List of free medical journals /scharr/ir/netting/ Netting the Evidence Over 7000 scholarly journals Directory of Open Access journals Post a question on a bulletin board DISCUSSION BOARDS PUSH Have information pushed to you British Medical Journal Here is the URL link to register master.ca/indes.asp Have information pushed to you Ortho SuperSite fault.asp?page=view&rid=3919 and click on the icon at the bottom of the left navigation panel 19

20 Have information pushed to you One last Push Mechanism Click here to register and get alerts mcmusculoskeletdisord/alerts BMC Musculoskeletal Disorders Questions?? Other Ideas 20

21 Interpreting the Evidence 1. What were the results? 2. Are the results valid? 3. Can I apply the results to my patient and how will they influence the patient s outcome What were the results and do they help me decide what to do? How large was the treatment effect? Were all clinically important outcomes considered? Does the benefit outweigh the potential harm and costs? Were the results valid? Appraising the evidence Primary Assessment Were the subjects selected randomly? Were all subjects accounted for? Secondary Assessment Who/What was blinded? Were the groups similar in the beginning? Were the groups treated equally? If evidence is limited to investigate a clinical intervention, a diagnostic test, or a clinical outcome we conclude: Insufficient: to few published studies Inconclusive: published studies are available but they do not meet research design or analytic standards Silent: no studies in the literature address a relationship or interest Appraising the evidence Appraising the evidence When there is sufficient scientific evidence between a clinical intervention and outcome we conclude Supportive Quantitative information from properly designed trials support a significant relationship Suggestive Case reports or descriptive studies provide a directional relationship but do not permit a statistical assessment of significance Equivocal Qualitative data has not provided a clear direction There is insufficient information Aggregate comparable studies disagree Proven Not Proven Proven Not 21

22 Cognizant of the patient s expectations and your common sense and experience Research Designs Randomized Controlled Trial on a very elementary level Random assignment of subjects Unbiased distribution of confounding variables Prospective manipulation of variables Precise measurement of physiological variables Rigorous control of extraneous variables Strong internal validity Provides strong evidence of intervention efficacy Randomized Controlled Trial Quasi-Experimental Randomized Non-randomized assignment of subjects to groups (experimental and control) Population Intervention Group Control Group Review Results Review Results Compare Results Used when RCTs are impractical or unethical Evidence may be biased toward the experimental or treatment group But evidence of treatment effectiveness is at least a little better than absence of evidence 22

23 Epidemiological Designs The study of society s collective health Descriptive Epidemiology Patterns and trends Analytical epidemiology Establishing causation (etiology) Epidemiological Research Cohort Studies Follow a group of people with a common experience/injury/disease over a defined period of time Subject selection is defined by availability or defined characteristics Classification of Research Design Observational (Quasi-Experimental) Cohort Study Prospective (cause-to-effect hypothesis) Variability in subject exposure to risk factors Subjects not exposed to risk factor(s) are considered the controls Large number of subjects and long duration of study is usually required Because of no randomization, difficult blinding, and influence of confounding variables Classification of Research Design Observational (Quasi-Experimental) Cohort Study cont - Observation to determine differences in incidence of injury in relation to exposure to risk factor(s) Risk Ratios Established Incidence rate for those exposed compared to those not exposed to risk factor(s) over a defined period of time Cohort Study Classification of Research Design Observational (Quasi-Experimental) Treatment Group Non- Treatment comparison time Review Review Compare Groups Case Control Study (analytical epidemiology) Retrospective (effect-to-cause hypothesis) Subjects with condition (cases) compared to similar subjects without the conditions (controls) Subjects matched on as many factors as possible Study the variability in past exposure to risk factor(s) Explore what made the group of individuals different 23

24 Classification of Research Design Observational (Quasi-Experimental) Case-Control Study cont Determination of whether cases and controls differ in terms of past exposures to risk factor(s) Example might be higher BMI Odds ratios (estimate of risk ratio) calculated Prevalence rate (number of cases with condition) compared between cases and controls Injury Prevention/Treatment Effectiveness Number Needed to Treat (NNT) Numerical expression of the number of interventions necessary to prevent one adverse outcome Calculated based on the absolute risk ratio (ARR) (incidence rate for control group minus the incidence rate for the intervention group) NNT = 1/AAR Outcomes Research Somewhat similar to a prospective cohort study in which patients treated at a facility are evaluated based on their outcome Can also be conducted as a true experimental study if random assignment of subjects to different treatment protocols Classification of Research Design Observational Case Series Information on a series of patient have the same injury or illness No comparison or control group Case Report Information on a single subject Classification of Research Design Synthesis of Research Evidence Systematic Review Criteria for inclusion of studies established prospectively Objective quality rating of evidence relevant to a specific clinical question May or may not include a meta-analysis of findings for multiple studies (effect sizes) Meta-analysis is a quantitative comparison of multiple studies that have addressed the same clinical question with different method Clinical Prediction/Decision Rules A decision-making rule for clinicians that include 3 or more variables from History Physical Exam Special Tests 24

25 Clinical Prediction/Decision Rules CPRs used for Diagnosis of a condition Clinical guideline in lieu of a gold standard Prognosis for the recovery of function Estimation of likelihood for full recovery under specific conditions in a defined time frame Likely response to a particular intervention Patient characteristics linked to the choice of treatment Clinical Prediction/Decision Rules CPRs are derived from systematic clinical observations CPRs are validated by evaluating the sensitivity, specificity, and LRs of specific observations CPRs are useful in guiding decision-making rationale Clinical Prediction/Decision Rules 3 Stages of Maturation Development (identification of predictors) Validation (assessment of accuracy) Impact Analysis (utilization) Diagnostic Accuracy Sensitivity (SNOUT) Rule out conditions Specificity (SPIN) Rule in conditions Likelihood Ratios Shifts in probability 25