Evidence Based Medicine

Transcription

1 Hamadan University of medical sciences School of Public Health Department of Epidemiology Evidence Based Medicine Amin Doosti-Irani, PhD in Epidemiology 10 March

2 Outlines An introduction to evidence based medicine (EBM) Formulate a research question and PICO Teamwork Principle and concepts in the methodology of clinical articles Critical appraisal and quality assessment of clinical articles Teamwork Principles and concepts of diagnostic test Critical appraisal and quality assessment of diagnostic test articles Teamwork and discussion 10 March 2017 a_doostiirani@yahoo.com 2

3 An introduction to EBM The term "evidence based medicine" was coined at McMaster Medical School in Canada in the 1980s Definitions EBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. EMB is the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions. 10 March

4 An introduction to EBM Good doctors use both: individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannized by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. Systematic reviews provide strong evidence to underpin EBM. 10 March

5 Components of evidence-based medicine 10 March

6 Components of evidence-based medicine Clinical expertise Best available evidence Patient values & expectation Improved patient outcome 10 March

7 Five Ways Evidence-Based Medicine Adds Value to Health Systems 1.Helps clinicians stay current on standardized, evidence-based protocols. 2.Uses near real-time data to make care decisions. 3.Improves transparency, accountability, and value. 4.Improves quality of care. 5.Improves outcomes. 10 March

8 An introduction to EBM Evidence based medicine is not restricted to randomized trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions. 10 March

9 An introduction to EBM To find out about the accuracy of a diagnostic test, we need to find proper cross sectional studies, not a randomized trial. For a question about prognosis, we need proper follow up studies of patients and sometimes the evidence we need will come from the basic sciences such as genetics or immunology. 10 March 2017 a_doostiirani@yahoo.com 9

10 An introduction to EBM It is when asking questions about therapy that we should try to avoid the non-experimental approaches, since these routinely lead to false positive conclusions about efficacy. Because the randomized trial, and especially the systematic review of several randomized trials, is so much more likely to inform us and so much less likely to mislead us, it has become the "gold standard" for judging whether a treatment does more good than harm. 10 March 2017 a_doostiirani@yahoo.com 10

11 We need to EBM. Why? 10 March

12 Study designs study Qualitative Quantitative Observational Interventional Descriptive Analytic Randomized control trial Field trial Community trial Case series Ecologic Crosssectional* Case-control Cohort 10 March

13 Observational studies Type of study Other name Unit of study Observational studies Descriptive Case-report Patients Case-series Patients Analytic Ecologic Correlation Population Cross-sectional* Prevalence Individuals Case-control Case-reference Individuals Cohort Incidence (follow-up) Individuals 10 March

14 Interventional studies Type of study Other name Unit of study Interventional Experimental studies Randomized controlled trial Clinical trial Patients Field trial Community trial Healthy people Communities 10 March

15 Case series Key characteristics Description of patients Description of some new patients with unknown sever acute respiratory illness in 2002 that became known as SARS. 10 March

16 Ecologic studies The units of observation is geographically defined populations such as counties or regions within a country Mean values for both a given postulated risk factor and the outcome of interest The analysis of ecological data involves plotting the risk factor and outcome values for all observation units to assess whether a relationship is evident. 10 March 2017 a_doostiirani@yahoo.com 16

17 Example Ten year coronary death rates of the cohorts from the seven countries The higher average saturated fat intakes the higher death rates 10 March

18 Ecological fallacy Aggregate Data Individual Inference Ecological Fallacy 10 March

19 Cross-sectional studies In a cross-sectional study design, a sample of (or the total) reference population is examined at a given point in time. it consists of taking a "snapshot" of a cohort by recording information on disease outcomes and exposures at a single point in time 10 March 2017 a_doostiirani@yahoo.com 19

20 Participants Cross-sectional studies Only prevalence Prevalence+ measure of association Unexposed without disease Unexposed with disease Exposed without disease Exposed with disease 10 March

21 Cross-sectional studies Key characteristics Direct measurement of prevalence Cross-sectional studies are relevant to public health planning Disadvantages It is difficult to establish the temporal sequence It is usually not possible to determine incidence, the use of prevalence as a proxy of frequency may distort the exposuredisease relationship 10 March 2017 a_doostiirani@yahoo.com 21

22 Case-control studies a case-control study usually compares cases (diseased individuals) and controls (usually non diseased individuals) with respect to their level of exposure to a suspected risk factor. 10 March 2017 a_doostiirani@yahoo.com 22

23 Types of Case-Control Study Case-control designs Case-Based Within a Defined Cohort Nested casecontrol Casecrossover Case-cohort 10 March 2017 a_doostiirani@yahoo.com 23

24 Design of a case-control studies Exposed Cases Unexposed Participants Exposed Controls Unexposed 10 March 2017 a_doostiirani@yahoo.com 24

25 Design of a case-control studies Level of exposure (risk factor) is compared in: cases (diseased individuals) controls (usually non-diseased individuals) Dichotomous risk factors >> odds ratio. Continuous risk factor >> mean difference 10 March 2017 a_doostiirani@yahoo.com 25

26 Critical issues affecting the validity of case-control data. Cases and controls originate from the same source population different reference population having similar relevant characteristics Otherwise selection bias may ensue Losses may affect the comparability of cases and controls. 10 March

27 Critical issues affecting the validity of case-control data Selection bias may occur even if cases and controls are from the same "hypothetical" cohort; this happens when "losses" occurring before the study groups are selected affect their comparability. 10 March

28 Potential Biases in Case-control Studies Selection bias Sources of Cases Selection of Controls Information bias Problems of Recall Limitations in Recall Recall Bias 10 March

29 Cohort studies Participants Exposed Unexposed Disease No disease Disease No dosease 10 March

30 Cohort studies 10 March

31 Potential Biases in Cohort Studies Selection Biases Nonparticipation Nonresponse lost to follow-up Information Biases If the quality and extent of information obtained is different for exposed persons than for no exposed persons, If the person who decides whether disease has developed in each subject also knows whether that subject was exposed, if that person is aware of the hypothesis being tested 10 March

32 Participants Randomized controlled trial New treatment Improved Not improved Current treatment Improved Not improved 10 March

33 Randomized controlled trial Key characteristics of RCT Randomization To generate comparative groups To enable valid statistical tests Concealment Blinding Open label (unbinding) Single blinding (patients only) Double blinding (patients and investigators) Triple blinding (patients and investigators and Monitoring investigators) 10 March

34 Randomized controlled trial Protocol deviation Patient ineligible Wrong treatment Competing events Noncompliance Loss to follow up Missing data 10 March

35 Randomized controlled trial Data analysis Baseline data analysis Main analysis Protocol deviation Intention to treat per protocol approaches Adherers only Intention to Treat: 1&2 vs 3&4 Adherers only: 2 versus 4 As treated: 1&4 versus 2&3 As treated 10 March 2017 a_doostiirani@yahoo.com 35

36 Ways of Expressing the Results of Randomized Trials 10 March

37 Review Types of review article Narrative review SR of Qualitative studies Meta-synthesis Systematic review (SR) SR observational studies May led to a Metaanalysis SR of Quantitative studies Two interventions May led to a metaanalysis SR interventional studies More than two interventions May led to a network metaanalysis 10 March 2017 a_doostiirani@yahoo.com 37

38 Systematic review A systematic review is a research article that identifies relevant studies, appraises their quality and summarizes their results using a scientific methodology. 10 March 2017 a_doostiirani@yahoo.com 38

39 Steps of a systematic review Step 1 Framing questions Step 2 Identifying relevant literature Step 3 Assessing quality of the literature Step 4 Summarizing the evidence Step 5 Interpreting the findings 10 March 2017 a_doostiirani@yahoo.com 39

40 Meta-analysis The term meta-analysis is not synonymous with a systematic review. It is only a part of the systematic review. It is a statistical technique for combining the results of a number of individual studies to produce a summary result. Some publications called metaanalysis are not systematic reviews. 10 March 2017 a_doostiirani@yahoo.com 40

41 Network meta-analysis When several treatment options are available, a series of individual meta analyses provides only partial information A network meta-analysis is a simultaneous analysis of the data from all of these randomized trials. With a network meta analysis, the relative effectiveness of 2 treatments can be estimated even if no studies directly compare them. 10 March 2017 a_doostiirani@yahoo.com 41

42 Level of evidence 10 March

43 10 March

44 Steps of EBM Five steps in evidence based medicine Formulate a clear clinical question from a patient's Problem Search the literature for relevant clinical articles Evaluate (critically appraise) the evidence Implement useful findings in clinical practice Assess your performance 10 March 2017 a_doostiirani@yahoo.com 44

45 Steps of EBM Step 5 Step 1 Ask a clinical question Step 2 Acquire the best evidence Step 3 Appraise the evidence Step 4 Apply the evidence Assess your performance 10 March 2017 a_doostiirani@yahoo.com 45

46 Types of questions Question Type of study How common is the problem? Prevalence Cross-sectional What is the causes of disease and their modes of operation Etiology Case control or cohort study Is early detection worthwhile? Screening Cross-sectional What will happen if we do nothing? Prognosis Cohort studies Does this intervention help? Treatment Randomized control trial What are the harms of an intervention? Adverse events Randomized control trial Is one intervention more cost-effective than another? Is the diagnostic test accurate? Diagnosis Diagnostic validation studies, Crosssectional Costeffectiveness Economic evaluation What will be quality of life of the patients? Quality of life Both qualitative & quantitative (cross-sectional) studies 10 March

47 Step 1: Ask a clinical question Formulate a clear clinical question from a patient's Problem Question components (PICOS) Populations Interventions (or exposures) Comparison Outcomes Study design 10 March 2017 a_doostiirani@yahoo.com 47

48 Template for asking answerable clinical questions Population Intervention Comparison Outcome Study design List concepts here List concepts here List concepts here List concepts here List concepts here Your completed clinical question 10 March

49 Formulate a clear clinical question from a patient's Problem 10 March 2017 a_doostiirani@yahoo.com 49

50 Formulate a clear clinical question from a patient's Problem An example question about clinical effectiveness Free form question: Which of the many available antimicrobial products improve healing in patients with chronic wounds? 10 March 2017 a_doostiirani@yahoo.com 50

51 Formulate a clear clinical question from a patient's Problem An example question about clinical effectiveness Structured question The population The interventions The outcomes The study design In adults with various forms of chronic wounds in an ambulatory setting.. would systemic or topical antimicrobial preparations.. improve wound healing? A comparative study that allocates subjects with chronic wounds to alternative therapeutic interventions of interest and determines the effect of the interventions on wound healing (e.g. randomized controlled trial). 10 March 2017 a_doostiirani@yahoo.com 51

52 An example question about etiology Structured question Free form question: Is exposure to benzodiazepines in pregnancy associated with malformations in the newborn baby? 10 March

53 An example question about etiology 10 March

54 An example question about etiology The population The exposures The outcomes The study designs In pregnant women.. does exposure to benzodiazepines during early pregnancy.. cause malformations in the newborn baby? A study that recruits women in early pregnancy, assesses their exposure to benzodiazepines, follows them up and examines their newborn babies to compare the rates of malformations among women with exposure and those without (cohort study). A study that retrospectively compares exposure to benzodiazepines in early pregnancy among women who have given birth to a child with malformation with those women who gave birth to a healthy child (case-control study). 10 March 2017 a_doostiirani@yahoo.com 54

55 An example question about test accuracy Free form question: Among postmenopausal women with abnormal vaginal bleeding, does pelvic ultrasound scan exclude uterine cancer accurately? 10 March

56 Structured question 10 March

57 Structured question The population The test The reference standard The study design In postmenopausal women, within a community setting, with vaginal bleeding.. does a uterine ultrasound scan test accurately predict.. histological diagnosis of uterine cancer? A study that recruits women from a relevant population, uses the test (scan) and a reference standard investigation to confirm or refute the presence of cancer (histology), and determines the accuracy with which the test identifies cancer 10 March 2017 a_doostiirani@yahoo.com 57

58 An example question about qualitative research Free form question: How does the experience of endometriosis impact on women s lives? 10 March 2017 a_doostiirani@yahoo.com 58

59 Structured question 10 March

60 Structured question The population The intervention The outcomes The study design In women with a confirmed diagnosis of endometriosis.. how does observation or treatment.. affect pain, social relationships and self-image? A study that narrates subjective experiences 10 March 2017 a_doostiirani@yahoo.com 60

61 An example question about an educational intervention Free form question: How does use of portfolios affect student learning in undergraduate (medical and nursing) education? 10 March

62 Structured question 10 March

63 Structured question The population The intervention The outcomes The study design Among undergraduate nursing or medical students.. does a portfolio, defined as a collection of evidence of student learning, a learning journal or diary, or a combination of these two elements.. improve knowledge and skills? A study that evaluates the educational effects of portfolios. 10 March 2017 a_doostiirani@yahoo.com 63

64 Steps of EBM Step 5 Step 1 Ask a clinical question Step 2 Acquire the best evidence Step 3 Appraise the evidence Step 4 Apply the evidence Assess your performance 10 March 2017 a_doostiirani@yahoo.com 64

65 Step 2 Acquire the best evidence 10 March 2017 a_doostiirani@yahoo.com 65

66 Template for designing a search strategy Population Intervention Comparison Outcome Study design List concepts here List concepts here List concepts here List concepts here List concepts here #1:.. #5:.. #9:.. #13:.. #17:.. #2:.. #6:.. #10:.. #14:.. #18:.. #3:.. #7:.. #11:.. #15:.. #19:.. #4: #1 OR #2 OR #3 #8: #5 OR #6 OR #7 #12: #9 OR #10 OR #11 #16: #13 OR #14 OR #15 #20: #17 OR #18 OR #19 #21: (#4 AND #8 AND #12 AND #16 AND #20) 10 March 2017 a_doostiirani@yahoo.com 66

67 #1: Tuberculosis #2: Latent Tuberculosis Infection #3: #1 OR #2 #4: QuantiFERON #5: Interferon-gamma Release Test #6: Interferon-gamma Release Assay #7: Enzyme-Linked Immunospot Assay #8: IGRA #9: Tuberculin Test #10: Tuberculin #11: PPD-S #12: Skin Test #13: Mantoux tuberculin skin test #14: #4 OR #5 OR #6 OR #7 OR #8 OR #9 OR #10 OR #11 OR #12 OR #13 #15: Kappa #16: Kappa-value #17: Kappa-statistic #18: Agreement #19: expected agreement #20: interobserver agreement #21: inter rater agreement #22: Predictive value test #23: Positive predictive value #24: Negative predictive value #25: Sensitivity #26: Specificity #27: #15 OR #16 OR #17 OR #18 OR #19 OR #20 OR #21 OR #22 OR #23 OR #24 OR #25 OR #26 #28: Observational Study #29: Cross-Sectional Study #30: Cross-Sectional Analysis #31: Cross Sectional Survey #32: Cohort Study #33: Retrospective Study #34: Prospective Study #35:#28 OR #29 OR #30 OR #31 OR #32 OR #33 OR #34 #36: Animal #37: (#3 AND #14 AND #27 AND #35) NOT #35 10 March

68 Think about what kind of evidence you need to answer your question: 1. Levels of evidence: what type of study would give you the best quality evidence for your question? 2. Secondary sources: is there a quality and relevance-filtered summary of evidence on your question, such as in ACP Journal Club or Clinical Evidence? 3. Systematic reviews: is there a systematic review in the Cochrane Library? 4. Bibliographic databases: in which database would you find relevant studies? 10 March 2017 a_doostiirani@yahoo.com 68

69 Levels of evidence: what type of study would give you the best quality evidence for your question? Try these first TRIP Database EBM Online Clinical Evidence Cochrane Library These sources will give you the best return on your precious time. 10 March

70 10 March

71 10 March

72 10 March

73 Secondary sources: is there a quality and relevance-filtered summary of evidence on your question, such as in ACP Journal Club or Clinical Evidence? Type Description Source Critically appraised topics (CATs) Evidence-based summaries Appraisals of evidence in response to clinical questions Reviews of the evidence around a specific clinical topic CATCrawler Journal clubs Your and your colleagues own collection Bandolier, Clinical Evidence ( Structured abstracts Appraisals of important clinical papers EBM Online, ACP Journal clubs, evidence-based journals Health technology assessments Appraisals of the evidence for a specific intervention Cochrane Library UK NHS HTA Programe Systematic reviews Review of all the evidence around a specific topic Cochrane Library 10 March 2017 a_doostiirani@yahoo.com 73

74 Can I trust this secondary source? Only if you can answer yes to all of the following: There are no conflict of interest It clearly states what question it addresses There is an explicit and evidence-based methodology behind finding, producing and checking the information. The source is reviewed and updated regularly 10 March 2017 a_doostiirani@yahoo.com 74

75 Primary sources Bibliographic databases: Medline Web of sciences Cochran collaboration Embase Scopus 10 March

76 PubMed 10 March

77 Web of sciences 10 March

78 Scopus 10 March

79 Embase 10 March

80 Cochrane library 10 March

81 BMJ best practice 10 March

82 Steps of EBM Step 5 Step 1 Ask a clinical question Step 2 Acquire the best evidence Step 3 Appraise the evidence Step 4 Apply the evidence Assess your performance 10 March 2017 a_doostiirani@yahoo.com 82

83 Appraise the evidence Questions that clinicians can ask when apprising evidence Dose the evidence improve decision-making or outcomes? clinical quality or patient safety? efficiency and lower costs? the care experience? access to health care services? 10 March

84 Appraise the evidence Critical appraisal of Guidelines Systematic reviews Diagnosis articles Articles on harm/etiology Prognosis studies Therapy articles Qualitative studies 10 March

85 Critical appraisal of guidelines 1. Scope and purpose of the guideline Does the guideline address a clear issue? Are the target users of the guideline clearly defined? 2. Methods Was there a comprehensive search for the evidence? Are the criteria for data extraction clearly described? Are the methods used for formulating the recommendations clearly described? Are the health benefits, side effects and risks of the interventions considered in formulating recommendations? 10 March 2017 a_doostiirani@yahoo.com 85

86 Critical appraisal of guidelines 3. Applicability Are different options for diagnosis and/ or treatment of the condition clearly presented? Are the key recommendations identifiable? 4. Conflicts of interest Is the guideline editorially independent from the funding body? Are the conflicts of interest of the developing members recorded? How up to date is the guideline? 10 March

87 Appraising systematic reviews Is the systematic review valid? Is it a systematic review of high-quality studies which are relevant to your question? Does the methods section adequately describe: a comprehensive search for all the relevant studies? how the reviewers assessed the validity of each study? Are the studies consistent, both clinically and statistically? 10 March 2017 a_doostiirani@yahoo.com 87

88 Appraising systematic reviews Are the results important? Are they clinically significant? Can the results help you? How precise are the results? 10 March

89 Appraising systematic reviews Three key features of a systematic review a strenuous effort to locate all original reports on the topic of interest critical evaluation of the reports conclusions are drawn based on a synthesis of studies which meet pre-set quality criteria 10 March 2017 a_doostiirani@yahoo.com 89

90 Appraising systematic reviews Is it a systematic review of the right type of studies which are relevant to your question? Only if: The review addresses a clearly defined question The review includes studies which also look at this question, The studies are the right design to address this question 10 March 2017 a_doostiirani@yahoo.com 90

91 Appraising systematic reviews Does the methods section describe how all the relevant trials were found and assessed? Search strategy hand searching of journals and searching for unpublished literature. Were any obvious databases missed? check the reference lists of articles and of textbooks contact experts Did they use an appropriate search strategy: were important subject terms missed? 10 March

92 Appraising systematic reviews Did the authors assess the trials validity? Publication bias What criteria were used to extract data from the studies? Again, it s helpful to think in terms of patient, intervention, outcome: Who were the study participants and how is their disease status defined? What intervention/s were given, how and in what setting? How were outcomes assessed? 10 March 2017 a_doostiirani@yahoo.com 92

93 Appraising systematic reviews Are the studies consistent, both clinically and statistically? Clinical heterogeneity use your clinical knowledge to decide whether the groups of patients, interventions, and outcome measures were similar enough to merit combining their results Statistical heterogeneity Check the statistical tests for assessing the heterogeneity 10 March

94 Appraising systematic reviews Are the results important? Is this odds ratio or relative risk clinically important? How precise are the results? The statistical significance of the results Confidence intervals for all results 10 March

95 Appraising systematic reviews 10 March

96 10 March

97 10 March

98 Appraising diagnosis articles Is the study valid? 1. Was there a clearly defined question? 2. Was the presence or absence of the target disorder confirmed with a reference standard? (Gold Standard) Was this comparison independent from and blind to the study test results? 3. Was the test evaluated on an appropriate spectrum of patients? 4. Was the reference standard applied to all patients? 10 March 2017 a_doostiirani@yahoo.com 98

99 Appraising diagnosis articles Are the results important? What is meant by test accuracy? a. a true positive result b. a false positive result c. a false negative result d. a true negative result Ideally, we would like a test which produces a high proportion of a and d and a low proportion of b and c. 10 March 2017 a_doostiirani@yahoo.com 99

100 Appraising diagnosis articles Sensitivity and specificity Sensitivity the ability of the test to identify correctly those who have the disease Specificity the ability of the test to identify correctly those who do not have the disease Positive predictive value: If the test results are positive in this patient, what is the probability that this patient has the disease? Negative predictive value: If the test result is negative, what is the probability that this patient does not have the disease? 10 March

101 Appraising diagnosis articles Cochrane risk of bias assessment Patient selection Index test All test Reference standard Flow and timing 10 March

102 Appraising diagnosis articles Cochrane risk of bias assessment 10 March

103 Appraising diagnosis articles Cochrane risk of bias assessment 10 March

104 Appraising diagnosis articles Cochrane risk of bias assessment 10 March

105 Appraising diagnosis articles Cochrane risk of bias assessment 10 March

106 Appraising articles on harm/etiology Is the study valid? Was there a clearly defined question? Were there clearly defined, similar groups of patients? Were exposures and clinical outcomes measured the same way in both groups? Was the follow up complete and long enough? Does the suggested causative link make sense? 10 March 2017 a_doostiirani@yahoo.com 106

107 Appraising articles on harm/etiology Are the results important? the higher the risk or odds, the stronger the association Statistical significance 10 March

108 Appraising prognosis studies Is the study valid? Is the sample representative? Were they recruited at a common point in their illness? Did the study account for other important factors? Is the setting representative? Was follow up long enough for the clinical outcome? Was follow up complete? If follow up is less than 80% the study s validity is seriously undermined. Were outcomes measured blind? 10 March 2017 a_doostiirani@yahoo.com 108

109 Appraising prognosis studies Are the results important? What is the risk of the outcome over time? as a percentage of survival at a particular point in time; as a median survival (the length of time by which 50% of study patients have had the outcome); as a survival curve that depicts, at each point in time, the proportion (expressed as a percentage) of the original study sample who have not yet had a specified outcome. Survival curves provide the advantage that you can see how the patient s risk might develop over time. How precise are the estimates? 10 March 2017 a_doostiirani@yahoo.com 109

110 Appraising prognosis studies Other tools STROBE Ottawa Newcastle scale Critical Appraisal Skills Program (CASP) 10 March

111 10 March

112 Appraising therapy articles Is the study valid? Was there a clearly defined research question? Was the assignment of patients to treatments randomized and was the randomization list concealed? Were all patients accounted for at its conclusion? Was there an intention-to-treat analysis? Were research participants blind? Were the groups treated equally throughout? Did randomization produce comparable groups at the start of the trial? 10 March 2017 a_doostiirani@yahoo.com 112

113 Appraising therapy articles Are the results important? What is the benefit of the treatment? bias; chance variation between the two groups; the effect of the treatment. 10 March

114 Appraising therapy articles Quantifying the risk of benefit and harm Relative risk or risk ratio (RR) RR is the ratio of the risk in the experimental group divided by the risk in the control group. Absolute risk reduction (ARR) ARR is the difference between the event rates in the two groups. Relative risk reduction (RRR) Relative risk reduction is the ARR as a percentage of the control group risk 10 March 2017 a_doostiirani@yahoo.com 114

115 Appraising therapy articles Quantifying the risk of benefit and harm 10 March

116 Appraising therapy articles Quantifying the risk of benefit and harm Number needed to treat (NNT) Number needed to treat is the most useful measure of benefit, as it tells you the absolute number of patients who need to be treated to prevent one bad outcome. It is the inverse of the ARR: NNT = 1 ARR 10 March 2017 a_doostiirani@yahoo.com 116

117 Appraising therapy articles Cochran risk of bias tool 10 March

118 Appraising therapy articles Cochran risk of bias tool 10 March

119 Appraising therapy articles Other tools Jadad Scale Cochran tool CONSORT 10 March

120 10 March

121 Thanks for your kind attention 10 March