Appendix I: GRADE profiles

Transcription

1 Appendix I: GRADE profiles GRADE and Modified GRADE approaches used in this guideline Standard GRADE approach Modified GRADE approach Criteria Effectiveness evidence (Developed by GRADE working group) Diagnostic test accuracy evidence (Modified by Schunemann et al, based on QUADAS checklist) Starting point High if RCT design, Low if any other design High if RCT, or cross sectional design, Low if any other design Prognostic evidence (Developed by Internal Clinical Guidelines team, based on QUIPS checklist) High if prospective study designs, Low if retrospective study designs Appraised using relevant methodology checklists Appraised using relevant methodology checklists Appraised using relevant methodology checklists Indirectness Inconsistency Imprecision Indirect population, intervention, comparator, or outcome Indirect comparison e.g. no head to head comparisons (no A to B comparison but A to C and C to B are available) Inconsistency of point estimates Confidence intervals that don t overlap Statistical measures of heterogeneity are high (e.g. I ) Few participants, few events (<00) and wide confidence intervals (where clinical action would differ if the upper or the lower boundary of the confidence interval represented the truth) The optimal information size has not been met Differences in population studied and population where recommendations will be applied Differences in diagnostic expertise in the study and in those intended to use the test Diagnostic tests are not directly compared (each test is only compared to the reference standard) Unexplained in sensitivity, specificity or likelihood ratios Wide confidence intervals for test accuracy Differences in the population, prognostic factors or outcomes of the included evidence compared with those for whom the recommendation is intended Unexplained in point estimates Few participants, and wide confidence intervals (where clinical action would differ if the upper or the lower boundary of the confidence interval represented the truth) Upgrading Observational evidence can be upgraded when no downgrading has taken place AND one or more of the following is satisfied: There is evidence of a large effect Plausible es underestimate true effect There is evidence of a dose-response gradient Not applicable Not applicable Internal Clinical Guidelines, 0

2 I. Signs, symptoms and risk factors for gallstone disease (Question ) GRADE profile studies Risk factors Wegge (985) serious With gallstone disease Without gallstone disease (95% CI) very None serious Did not use all patients that were available. Only those presenting during office hours were included in the analysis Study does not report data from the multivariate analysis. Study was underpowered for the number of variables entered into the analysis Not reported I. Diagnosing gallstone disease (Question ) GRADE profile I.. Diagnosing gallbladder stones assessment patients studies Sensitivity Effect Specificity Ultrasound compared to surgery Ahmed (00) serious serious serious None (.00,.00) 0. (0., 0.7) No description of inclusion/exclusion criteria and no definitions provided for the population term used. Unclear if researchers were blinded when interpreting the results of the test. unclear if the population selected is appropriate to answer the research question Lower bound of one of the confidence intervals was at or below 0.50 I.. Diagnosing cholecystitis studies Internal Clinical Guidelines, 0 assessment patients Sensitivity Effect Specificity MRCP compared to Surgery Hakansson (000), Park (998) serious, serious none (0.70, 0.96) 0.89 (0.50, 0.99) Ultrasound compared to surgery serious serious serious none (0.8, 0.88 (0.6,

3 De Vargas (006), Hakansson (000), Park (998) MRI compared to surgery Altun (007) CT compared to surgery De Vargas (006) 0.9) 0.97) serious serious 5 serious 6 serious none 0.95 (0.7, 0.99) 0.69 (0., 0.88) serious serious 8 none 0.95 (0.5,.00) 0.88 (0.7, 0.99) In Hakansson (000) there is possible selection as more than half of eligible participants were excluded because they pr esented outside of office hours. In Park (998) it is unclear how participants were selected (random/consecutive/ self-selected). Patients were selected retrospectively, this could lead to selection. Unclear if the number included in the study was al l the available cases or whether it is a sample of the available cases. Lower bound of one of the confidence intervals was at or below 0.50 Individual study estimates vary widely 5 Reference standard and index test were performed a month apart. This is an inappropriate interval. 6 Study aims to differentiate between different types of cholecystitis. 8 Lower bound of both of the confidence intervals were at or below 0.50 I.. Diagnosing common bile duct stones studies MRCP compared to ERCP 8 Chan (996), Regan (996), Soto (000), Griffin (00), Kondo (005), Stiris (000), Sugiyama (998), Holzknecht (998) Ultrasound compared to ERCP 5 Jovanovic (0) Regan (996), Rickes (006), Sugiyama (997), Sugiyama (998) Karki (0) Internal Clinical Guidelines, 0 Endoscopic ultrasound compared to ERCP Kondo (005), Polkowski (999), Sugiyama (997) CT cholangiography compared to ERCP Kondo (005), Polkowski (999), Soto (000), Soto (999) assessment serious serious serious serious serious serious patients Sensitivity none (0.7, 0.9) none (0.5, 0.8) Effect Specificity 0.90 (0.8, 0.95) 0.88 (0.6, 0.97) MODERATE very serious 9 none serious none (0.87, 0.97) none (0.67, 0.9) 0.9 (0.,.00) 0.8 (0.7, 0.9) MODERATE

4 CT compared to ERCP Soto (000), Sugiyama (997), Tseng (008) serious 5 yes (0.69, 0.8) 0.90 (0.66, 0.97) MODERATE Predictive model (Model predicts CBDS by measuring alkaline phosphatase, total bilirubin, amylase, dilated common bile duct>8mm. Bile duct stones are predicted in patients with one or more of the four factors. An absence of bile duct stones is predicted in patients with none of the four factors) Shiozawa (005) serious 7 none (0.9,.00) 0.95 (0.9,.00) MODERATE Predictive model (Model predicts CBDS by measuring gamma-glutamyl transferase, common bile duct diameter, and amylase. Values are entered into an equation and scores 0 predict CBDS, all other scores predict an absence of gallstones) Barr (999) Predictors of common bile duct stones Alponat (997) Retrospective Retrospective serious 7 serious none (0.60, 0.98) 0.7 (0.9, 0.89) serious 8 none 9 Aspartate aminotransferase Adj OR=.9 (.5, 5.88) CBD diameter Adj OR=.9 (.85, 8.99), Cholangitis Adj OR= 5.0 (.55, 7.79) Methods of patient selection were unclear and selection could be present. Most studies did not state if the researchers/ clinicians were blinded to the results of the reference standard when interpreting index test result. Studies employed different exclusion criteria Wide variation in individual studies estimates of sensitivity and specificity. Most studies did not state if the researchers/clinicians were blinded to the results of the reference standard when interpreting index test result. Studies employed different exclusion criteria. Lower bound of one of the confidence intervals was at or below Studies used different exclusion criteria 6 Tseng (008) divides participants into separate groups who each undergo CT but with different scanning parameters. Unclear why this was done as results are not presented separately for the different groups 7 Unclear if researchers were blinded to results of the reference standard when interpreting the results of the index tests. 8 Wide confidence intervals for one or more predictive factor, where clinical action would differ if the upper or lower confidence interval were true. 9 Insufficient data are provided to enable calculation of confidence intervals. I. Risk factors for asymptomatic complications (Question ) GRADE profile studies Risk factors Attili (995) very serious Internal Clinical Guidelines, 0 assessment patients patients Effect very serious none 8 Biliary colic No significant predictors Complications The low number of events meant that analysis was

5 not possible Cholecystectomy Occurrence of biliary colic predicted cholecystectomy Death No associations between potential predictive factors and death were reported. Study population may not be representative of the actual population: participants were invited to participate in screening which may have led to self selection of people with different symptoms and risk factors to those in the general population. Men and women were recruited in different recruitment rounds that took place i n different years. Unclear why this approach was taken. Inappropriate statistical analysis: not all analyses are reported and those that are only report p values. I. Managing asymptomatic gallbladder stones (Question a) GRADE profile No evidence was found I.5 Managing symptomatic gallbladder stones (Question b) GRADE profile I.5. Laparoscopic cholecystectomy compared to laparoscopic cholecystectomy + intraoperative cholangiography Internal Clinical Guidelines, 0 studies Bile leak Soper (99) Bile duct injury Amott (005), Khan (0), Soper 99) serious Length of stay (Better indicated by lower values) Soper (99) Missed common bile duct stones Amott (005), Khan serious LC serious none 0/59 LC+IOC 0/56 serious none /06 /0 (0.65%) (0.66%) - - MODERATE RR 0.98 (0.7 to 5.59) 0 fewer per 000 (from 5 fewer to 0 more) serious 5,6 none - mean 0 higher (0 to 0 higher) serious none 0/06 /0 RR 0.56 (0.5 fewer per 000 (from (0.%) to.0) fewer to more) MODERATE IMPORTANT IMPORTANT

6 (0), Soper 99) Study wasn't powered to detect any event in either arm Unable to analyse zero event data Inappropriate method of randomisation (month of birth). Low event rates 5 Small sample size 6 No measures of dispersion are reported I.5. Laparoscopic cholecystectomy compared to Cholecystostomy No evidence was found I.5. Laparoscopic cholecystectomy compared to conservative management studies Disease progression serious Schmidt (0) Additional intervention required (assessed with: Cholecystectomy) Schmidt (0) serious Readmission (assessed with: Biliary pain) Schmidt (0) serious LC serious none /99 (%) Conservative management /0 (.7%) RR 0. (0. to 0.9) 87 fewer per 000 (from 9 fewer to fewer) serious none 5/0 (.%) required cholecystectomy in the conservative management group serious none 5/99 (5.%) 9/0 (8.6%) RR 0. (0.06 to.97) 5 fewer per 000 (from 75 fewer to 8 more) IMPORTANT Length of stay - not reported none IMPORTANT Mortality Schmidt (0) serious serious none 8/99 (8.%) /0 (.9%) RR.0 (0.5 to 9.9) 7 more per 000 (from 9 fewer to 7 more) IMPORTANT A single study reported in 6 separate publications (Vetrhus 00, 00, 00, 005; Schmidt 0a, 0b) Lack of consistency between publications from this study- same outcomes are reported differently, and data don't always add up, therefore there is some ambiguity as to whether the correct outcome and correct numbers are used in this analysis. Some patients had open surgery. Not downgraded as the majority of patients did receive laparoscopic surgery. Small sample size and few event rates Internal Clinical Guidelines, 0

7 I.5. Day case laparoscopic cholecystectomy compared to planned inpatient laparoscopic cholecystectomy studies Failed day case discharge Hollington (999), Johansson (006), Keulemans (998) Readmission Barthelsson (008), Hollington (999), Johansson (006), Keulemans (998) Length of stay Hollington (999), Johansson (006), Keulemans (998) no serious risk of serious serious serious serious none 8/9 (.%) serious none /5 (.%) none Day LC Overnight LC 8/9 (.%) of planned day cases required an inpatient admission - /6 (.9%) RR.7 (0.7 to 5.05) more per 000 (from fewer to 75 more) Hollington (999) /7 day case patients required prolonged hospitalisation of days or more Johansson (006) 8/5 day case patients were discharged within -6 hrs ( patients were admitted), /8 inpatients were discharged on the first day after surgery, 6/8 inpatients were discharged on the second day after surgery Keulemans (998) post surgical length of stay was Mean=7. SD= 0.8 hrs for the day case group and Mean = SD= for the inpatient group MODERATE IMPORTANT Mortality - not reported none IMPORTANT of life (measured with: Health Index ; Better indicated by higher values) Barthelsson (008), Johansson (006), Keulemans (998) serious Low event rates Study/Studies does provide details about randomisation procedures Study/Studies does provide details about randomisation procedures Assessment of outcome is different across all studies 5 Individual studies have small sample sizes serious 5 none SMD 0.9 higher (0. lower to.0 higher) IMPORTANT Internal Clinical Guidelines, 0

8 I.6 Managing Common bile duct stones (Question c) I.6. ERCP + Laparoscopic cholecystectomy compared to ERCP alone. studies ERCP+LC ERCP alone of life - not reported none Recurrence/disease progression Boerma (00), Lau (006) Additional intervention required (ERCP) Boerma (00), Lau (006) Additional intervention required (Cholecystectomy) Boerma (00), Lau (006) Mortality Boerma (00), Lau (006) Length of stay 0 No evidence available Few participants serious none 7/8 (5.%) serious none 0/8 serious none 0/8 serious none /8 (8%) 8/8 (.%) /8 (.9%) 8/8 (5.7%) 9/ (.5%) RR 0. (0.0 to 0.96) RR 0.05 (0.0 to 0.9) RR 0.0 (0 to 0.) RR 0.58 (0.9 to.5) 79 fewer per 000 (from fewer to 8 fewer) fewer per 000 (from 9 fewer to 7 fewer) 9 fewer per 000 (from 05 fewer to 57 fewer) MODERATE MODERATE MODERATE 57 fewer per 000 (from MODERATE IMPORTANT 96 fewer to 0 more) none IMPORTANT I.6. ERCP compared to conservative management studies Mortality Internal Clinical Guidelines, 0 ERCP Conservative mangement

9 8 Acosta (006), Fan (99), Folsch (997), Hui (00), Neoptolemos (998), Nitsche (995) Oria (007), Vracko (006) Disease progression 6 Acosta (006), Fan (99), Folsch (997), Nitsche (995), Hui (00), Vracko (006) Additional intervention required (ERCP) 6 Acosta (006), Fan (99), Folsch (997), Neoptolemos (998), Nitsche (995), Oria (007) very serious serious,, very serious serious very serious, Additional intervention required (Cholecystectomy) Acosta (006), Oria (007), Vracko (006), Zhou (00). Length of stay 0 No evidence available serious serious serious 5 none 5/55 (.9%) none 7/0 (8.%) none 0/ none 05/5 (68.6%) 5/50 (.7%) 7/07 (8.7%) 78/07 (9.%) 00/6 (6.%) RR.00 (0.5 to.86) RR 0.8 (0. to.) RR 0.05 (0.0 to 0.6) RR. (0.57 to.7) 0 fewer per 000 (from fewer to more) 9 fewer per 000 (from 0 fewer to 0 more) 8 fewer per 000 (from 6 fewer to 88 fewer) 9 more per 000 (from 67 fewer to 000 more) IMPORTAN IMPORTANT none IMPORTANT Unequal intervention and control groups: Hui (00) found significant differences in baseline liver function tests between intervention and control groups Some studies fail to assess baseline characteristics between intervention and control groups. Some studies fail to report randomisation procedures Differences between studies regarding patient characteristics: Majority focus on patients with pancreatitis, Hui (00) focuses on patients with cholangitis, Vracko (006) focuses on patients with cholecystitis. 5 Low event rates I.6. Biliary stent compared to cleared duct studies Mortality Chopra (996) Disease progression serious Internal Clinical Guidelines, 0 Stents serious none / (9.%) Duct clearance / (.7%) RR.00 (0.9 to 0.5) 7 more per 000 (from 8 fewer to 5 more)

10 Chopra (996) serious Additional intervention required (ERCP) Chopra (996) serious Additional intervention (Cholecystectomy) Chopra (996) serious serious none 0/ (.%) serious none /9 (.%) serious none 5/9 (.8%) 8/ (8.6%) / (76.7%) / (7%) RR.5 (0.55 to.86) RR 0. (0.7 to 0.7) RR.8 (0.7 to 7.9) 7 more per 000 (from 8 fewer to 6 more) 7 fewer per 000 (from 0 fewer to 560 fewer) 59 more per 000 (from 7 fewer to more) IMPORTANT IMPORTANT Length of stay - not reported none IMPORTANT No assessment of baseline characteristics. Unclear if there were important differences between groups that could confound the results Small sample size and few events I.6. Day case ERCP compared to planned inpatient ERCP No evidence was found I.6.5 ERCP clearance of bile duct with laparoscopic cholecystectomy compared to surgical clearance of bile duct with laparoscopic cholecystectomy I.6.5. Network comparisons assessment Network meta analysis results studies Length of stay 8 Bansal (00), Cuschieri (999), ElGeidie (0a), Elgeidie (0b), Hong (006), Noble Internal Clinical Guidelines, 0 serious, serious,5,6,7,8 Option Direct estimate In comparison to surgical BDE Indirect estimate Probably best Median rank (95% Ci) serious 9 none Surgical BDE (,) Pre-op ERCP 0. (-.6,.6) Intra-op ERCP 0.56 (-0.8,.50) 0.8 (-0.5,.5) 0.7 (-.0,.55) (,) 0.69 (,) Post-op ERCP (,)

11 (009), Rhodes (998), Rogers (00) Missed stones 5 serious, serious,5,7 Elgeidie (0a), Hong (006), Nathanson (005), Noble (009), Sgourakis (00) Failed procedure 9 Bansal (00), Cuschieri (999), ElGeidie (0a), ElGeidie (0b), Hong (006), Noble (009), Rhodes (998), Rogers (00), Sgourakis (00) Conversion to open surgery 9 Bansal (00), Cuschieri (999), ElGeidie (0a), ElGeidie (0b), Hong (006), Nathanson (005), Noble (009), Rhodes (998), Sgourakis (00). Internal Clinical Guidelines, 0 serious,, serious,5,6,7,8 serious,, serious,5,6,7,8 (.,.89) (-0.9, 5.) serious 9 none Surgical BDE (,) IMPORTANT Pre-op ERCP (,) (0., 9.90) (0.0, 76.) Intra-op ERCP 0.8 (0.05,.70) Post-op ERCP.86 (0.6,.) 0.0 (0.00,.) 0.89 (,) (,) (0.06, 68.0) serious 9 none Surgical BDE (,) IMPORTANT Pre-op ERCP (,) (.08, 9.7) (.5,.8) Intra-op ERCP 0.68 (0.,.9) Post-op ERCP.00 (0.6,.75) 0.76 (0.9,.) 0.99 (0.08,.7) 0.76 (,) 0.58 (,) serious 9 none Surgical BDE (,) IMPORTANT (,) Pre-op ERCP (0.8,.7) (0.,.) Intra-op ERCP 0.7 (0.,.8) Post-op ERCP 0.58 (0.07,.86) 0.65 (0.7,.) 0. (0.0, 6.) 0.9 (,) 0.60 (,) Not used Inappropriate randomisation: Hong (006) uses patient identifying numbers Unclear randomisation procedures used by Rhodes (998) Differences in inclusion criteria and patient comorbidities/symptoms- most studies indicate than no specific exclusion criteria were used, where as other studies impose selective exclusion criter ia. 5 ElGeidie (0a) excludes patients with acute cholangitis, gallstone pancreatitis,asa grades IV-V 6 Elgeidie (0b) excludes patients with cholangitis, pancreatitis, ASA IV-V 7 Noble (009) includes high risk patients (over 70 years of age, over 60 with a co morbidity or over 50 with a BMI >0)

12 8 Rogers (00) Excludes patients with ASA status >II, suppurative cholangitis, severe pancreatitis. 9 Wide credibility intervals for rankings within the network. Unclear randomisation procedures used by Sgourakis (00) I.6.5. Pairwise comparisons studies More than one ERCP required Bansal (00), Cuschieri (999), Nathanson (005), Rhodes (998) Mortality Cuschieri (999), Noble (009), Sgourakis (00) serious Inconsistency Indirectness Imprecision LC+ERCP LC+BDE serious none Pre op ERCP: 9/65 (5%) required > ERCP Intra op ERCP: not reported Post op ERCP: 8/85 (%) required ERCP serious none 5/5 (.%) / (5.%) RR 0. (0.5 to.) 9 fewer per 000 MODERATE (from fewer to more) I.7 Timing of intervention (Question 5) I.7. Early compared to delayed laparoscopic cholecystectomy for acute cholecystitis. Internal Clinical Guidelines, 0 studies Readmission due to symptoms Johansson (00), Lai (998), Lo (998), Macafee (009) Early LC serious serious none 0/08 Delayed LC 6/9 (8.8%) RR 0.05 (0.0 to 0.) 79 fewer per 000 (from 5 fewer to 87 fewer) Readmission due to surgical complications - not reported none Length of stay (Better indicated by lower values) 6 Johansson (00), Kolla (00), Lai (998), Lo (998), Macafee (009), Yadav (009) serious none MD.9 lower (.67 to.9 lower) MODERATE IMPORTANT

13 Gul (0) Mortality Johansson (00), Kolla (00), Lai (998), Lo (998) of life measured by mean VAS scores Gul (0) very serious 6,7 none 0 0 mean early=.77 days mean delayed= 0.0 days serious none 0/ 0/05 serious 7 none hour MD 0.57 higher (0. to 0.9 higher) hours MD.7 higher (. to.9 higher) hours MD 0. higher (0.0 lower to 0.76 higher) 8 hours MD 0.9 higher (0.08 lower to 0.6 higher) IMPORTANT MODERATE IMPORTANT MODERATE IMPORTANT Macafee (009) also includes patients with biliary colic. Unclear what proportion of the study sample this represents. This study represents approximately 8% of the population included in this outcome. Few events Length of stay was reported inconsistently by individual studies, therefore data reported have been converted into means and standard deviations Zero events were observed in both arms of the trial in all studies. Studies were underpowered to detect differences in mortality. 5 Not estimable due to zero events in both arms 6 Insufficient data are provide to estimate confidence intervals, 7 Small sample size I.7. Early compared to delayed laparoscopic cholecystectomy after ERCP for common bile duct stones studies Internal Clinical Guidelines, 0 ERCP followed by Early LC ERCP followed by Delayed LC (95% CI) Readmission due to symptoms - not reported 0 No evidence available none Readmission due to surgical complications 0 No evidence available none

14 Length of stay (Better indicated by lower values) Mortality serious none MD 0.7 higher (0.88 MODERATE IMPORTANT lower to. higher) very none 0/86 serious, 0/ IMPORTANT of life - not reported 0 No evidence available none IMPORTANT Small sample size Zero events- studies were not adequately powered to detect differences in mortality. Effect size cannot be estimated due to zero event data in both arms I.8 Patient information (Question 6) Internal Clinical Guidelines, 0 assessment studies Limitations Transferability Diet Blay, 005; Blay, 006; Young, 00 Wounds Blay, 005; Blay 006; Young, 00 Pain management Blay, 005; Young 00 Resuming activity Blay, 005; Blay 006 Memory Barthelsson, 00 General Young, 00 Mixed very serious issues no other Mixed very serious issues no other Mixed very serious issues no other Mixed very serious issues no other Mixed very serious issues no other Mixed very serious issues no other Results Patients and carers requested additional information on diet and fluids. Patients had questions on how their wounds should be cared for. Patients and carers requested additional information on wound care. Patients and carers requested additional information on pain management. 65% had not been told how long it would take to return to normal activities / requested information on activity 6% requested information on post operative activity Several respondents had no memory of the information given to them by the surgeon on discharge from hospital. 00% of day case patients and carers had sufficient discharge information, compared to % of overnight patients and

15 Information seeking Tamahanka, 009 Mixed very serious serious no other 55.6% of overnight carers who thought they had sufficient discharge information. % of patients with internet access used it to acquire additional information about their operations and 58% used internet search engines to acquire additional information Modified GRADE approach used, where qualitative evidence was identified from a range of study designs (e.g. RCT,, qualitative). Thus studies are grouped based on outcome type rather than study design as is used in the standard GRADE approach. Evidence from the various study designs was assessed using qualitative checklists as this was considered the most appropriate for the outcomes included in this review. Values/assumptions/theory underpinning the purpose of the study are not discussed. Lack of defensible/rigorous design/methodology. Absence of rich findings. Study includes people waiting for hernia repair as well as those with gallstone disease. Separate analyses are not reporte Internal Clinical Guidelines, 0

16 Error! No text of specified style in document. GRADE profiles Error! No text of specified style in document.