Why is ILCOR moving to GRADE?

Transcription

1 1 Why is ILCOR moving to GRADE? Associate Professor Peter Morley Director Medical Education Royal Melbourne Hospital University of Melbourne 10 min

2 2

3 3

4 4

5 5

6 6

7 7 Apart from international consensus Allows more clarity and consistency when move from evidence to recommendations

8 8 So is this really different to the C2010 approach?

9 9 Differences between processes 2010 ILCOR Identify PICO question Search for studies Clear inclusion exclusion criteria Identify studies for detailed review Identify key (critical) outcomes of relevance 2015 ILCOR (GRADE) Identify PICO question Search for studies Clear inclusion exclusion criteria Identify studies for detailed review Identify key (critical) outcomes of relevance

10 10 Differences between processes 2010 ILCOR Allocate study LOE: from LOE 1 to LOE 5 Allocate methodological quality for each study All studies (with outcomes) into grid according to direction for specific question 2015 ILCOR (GRADE) Allocate study type: RCTs or observational Assess risk of bias for each study All studies (with outcomes) into risk of bias table (including overall risk of bias for each study)

11 11 Assessment of methodological quality (risk of bias) of RCTs: similar 2010: ILCOR Quality RCTs (not clearly documented) Adequacy of randomization Allocation concealment Blinding Loss to follow up Intention to Treat (IT) analysis Groups treated equally? Groups similar at the start? 2015: GRADE Risk of bias RCTs (clearly documented) Adequacy of randomization Allocation concealment Blinding Loss to follow-up, Intention to Treat (IT) analysis Any other risks? Early stopping for positive benefit others

12 Allocation of quality Vienna 2012 vs downgrading for RCTs: similar : Quality for individual RCT Good studies most/all of the relevant quality items Fair studies have some of the relevant quality items Poor studies have few of the relevant quality items (but sufficient value to include for further review). Excluded insufficient relevant quality items to be included 2015: Overall risk of bias for individual RCT Low (?Good) Unclear (?Fair) High (?Poor)

13 13 Published limitations (extras at 5)

14 14 Remember our evidence table

15 15

16 Overall risk of bias for study: Low, Moderate or High Low risk of bias = most or all key criteria listed are met, and any violations are not crucial. Moderate risk of bias if have a crucial limitation in one criterion or some limitations in multiple criteria, sufficient to lower the confidence in the estimate of effect. High risk of bias if have a crucial limitation in one or more criteria, sufficient to substantially lower the confidence in the estimate of effect. 16

17 17 Differences between processes 2010 ILCOR Narrative description of key information from all studies 2015 ILCOR (GRADE) Narrative description of key information from all studies Plus specific information regarding evidence for each key outcome Evidence profile table(s) Summary of findings table(s)

18 18 Major difference in approach for GRADE Key outcome measures are decided for each PICO question, and allocated a numerical rating Critical 9, 8, 7 Important 6, 5, 4 Limited importance 3, 2, 1 Quality of evidence is assessed (across studies) for each key outcome

19 19 How is the evidence for each outcome collated? Evidence Profile table(s) Summary of Findings table(s) Narrative description Consensus on Science statement

20 20 Evidence Profile table(s) PICO question Outcome to which evidence applies No of studies that report data for that outcome Study Design (RCT, Observational etc) Risk of bias (limitations: no serious, serious, very serious) Inconsistency (limitations: no serious, serious, very serious) Indirectness (limitations: no serious, serious, very serious) Imprecision (limitations: no serious, serious, very serious) Other including publication bias (Undetected, strongly suspected) Overall Quality of evidence for outcome High, Moderate, Low, Very low

21 Evidence profile table 21

22 Classification across all studies 22 for each outcome Risk of Bias/Inconsistency/Indirectness/Imprecision No serious limitations: Most information is from studies at low risk of bias. Do not downgrade Serious limitations: Most information is from studies at moderate risk of bias. Rate down one level Very serious limitations: Most information is from studies at high risk of bias. Rate down two levels

23 Classification across all studies for 23 each outcome Publication Bias Undetected Strongly suspected Quality of Evidence across all included studies for outcome High Moderate Low Very Low

24 24

25 25

26 Quality of evidence for outcome (across all studies) Four final categories High Moderate Low Very low Start with high for RCTs Start with low for observational Can be modified according to a number of factors 26

27 27 RCTs can be downgraded all or most of the studies had sufficient problems to make estimate unreliable or uncertain design and execution (risk of bias) consistency of results directness of comparisons precision publication bias

28 Overall quality of evidence: Vienna for each key outcome across studies

29 29 What about even lower levels of evidence? Case series (LOE 4) Manikin/Models/Animals (LOE 5)

30 30 Case series Start at low/very low quality Can be upgraded

31 31

32 32

33 33 Case series Start at low quality Can be upgraded BUT not if would be downgraded first!

34 34

35 Manikin/Models/Animals (LOE 5) (NB. GRADE not created for these) 35 Starting point depends on methodology RCTs at high quality Observational at low quality BUT Even RCTs can be downgraded Very serious indirectness = -2 Can be explained in comments

36 Summary of Findings table: columns PICO Key Outcomes Illustrative comparative risks (categorical or continuous with 95% CI) Control group Intervention group Relative effect (95% CI) No of Participants (studies) Overall quality of the evidence (GRADE) Comments/Footnotes 36

37 Summary of findings table 37

38 38 Next = CoSTR Consensus on Science statement and Treatment Recommendations

39 Differences between processes Consensus on Science statement (based on listing of levels of evidence for specific outcomes) Treatment recommendation: using behavioral wording, based largely on quality of evidence 2015 Consensus on Science statement (based on overall quality of evidence for critical outcomes ) Treatment recommendation: strong or weak (with behavioral correlates), based on evidence quality (high, moderate, low, very low)

40 40 So how will our Consensus on Science statement and Treatment Recommendations differ? COS: Quality level of evidence (GRADE) TR: Strength of Recommendation (GRADE)

41 GRADE: Strength of Recommendation Vienna Strong: the desirable effects of an intervention clearly outweigh the undesirable effects, or clearly do not. Weak: the trade-offs are less certain either because of low quality evidence or because evidence suggests that desirable and undesirable effects are closely balanced.

42 42 Strong Recommendation For patients most people in your situation would want the recommended course of action and only a small proportion would not; request discussion if the intervention is not offered For clinicians most patients should receive the recommended course of action

43 43 Weak Recommendation For patients most people in your situation would want the recommended course of action, but many would not For clinicians you should recognize that different choices will be appropriate for different patients and that you must help each patient to arrive at a management decision consistent with her or his values and preferences

44 44