Online Supplementary Material

Transcription

1 Section 1. Adapted Newcastle-Ottawa Scale The adaptation consisted of allowing case-control studies to earn a star when the case definition is based on record linkage, to liken the evaluation of case-control studies to that of cohort studies. The tool is otherwise identical to the original posted by the authors. 1 NEWCASTLE-OTTAWA QUALIT ASSESSMENT SCALE CASE CONTROL STUDIES te: A study can be awarded a maximum of one star for each numbered item within the Selection and Exposure categories. A maximum of two stars can be given for Comparability. Selection 1) Is the case definition adequate? a) yes, with independent validation b) yes, eg record linkage or based on self reports c) no description 2) Representativeness of the cases a) consecutive or obviously representative series of cases b) potential for selection biases or not stated 3) Selection of Controls a) community controls b) hospital controls c) no description 4) Definition of Controls a) no history of disease (endpoint) b) no description of source Comparability 1) Comparability of cases and controls on the basis of the design or analysis a) study controls for (Select the most important factor.) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.) 1

2 Exposure 1) Ascertainment of exposure a) secure record (eg surgical records) b) structured interview where blind to case/control status c) interview not blinded to case/control status d) written self report or medical record only e) no description 2) Same method of ascertainment for cases and controls a) yes b) no 3) n-response rate a) same rate for both groups b) non respondents described c) rate different and no designation NEWCASTLE-OTTAWA QUALIT ASSESSMENT SCALE COHORT STUDIES te: A study can be awarded a maximum of one star for each numbered item within the Selection and Outcome categories. A maximum of two stars can be given for Comparability. Selection 1) Representativeness of the exposed cohort a) truly representative of the average (describe) in the community b) somewhat representative of the average in the community c) selected group of users eg nurses, volunteers d) no description of the derivation of the cohort 2) Selection of the non exposed cohort a) drawn from the same community as the exposed cohort b) drawn from a different source c) no description of the derivation of the non exposed cohort 2

3 3) Ascertainment of exposure a) secure record (eg surgical records) b) structured interview c) written self report d) no description 4) Demonstration that outcome of interest was not present at start of study a) yes b) no Comparability 1) Comparability of cohorts on the basis of the design or analysis a) study controls for (select the most important factor) b) study controls for any additional factor (This criteria could be modified to indicate specific control for a second important factor.) Outcome 1) Assessment of outcome a) independent blind assessment b) record linkage c) self report d) no description 2) Was follow-up long enough for outcomes to occur a) yes (select an adequate follow up period for outcome of interest) b) no 3) Adequacy of follow up of cohorts a) complete follow up - all subjects accounted for b) subjects lost to follow up unlikely to introduce bias - small number lost - > % (select an adequate %) follow up, or description provided of those lost) c) follow up rate < % (select an adequate %) and no description of those lost d) no statement 3

4 Section 2. RTI item bank instructions for our systematic review on the cardiovascular safety of glucose-lowering drugs in persons with type 2 diabetes mellitus (developed to complement the original instructions) Q1a Is the study design prospective, retrospective, or mixed? Prospective/retrospective data collection in relation to potential for recall bias: Q1a and Q1b Prospective Retrospective Mixed i.a. Prospective: data on exposure and other predictors were collected before outcome occurred (e.g., electronic medical records [EMR]) i.b. Retrospective: data on exposure and other predictors were collected after outcome occurred (e.g., birth defect surveillance systems) i.c. Mixed (this category may include studies with prospective collection of exposure and retrospective collection of other predictors, including potential confounders, or prospective data collection in one arm and retrospective data collection in another) Q1b Is the study design prospective, retrospective, or mixed? Prospective/retrospective collection of primary data in relation to ability to tailor primary data collection to current study: ii.a. Prospective: primary data collection started after current study was designed (e.g., surveybased studies) ii.b. Retrospective: primary data available already when study was designed (e.g., health care claims- and EMR-based studies) ii.c. Mixed Q2 Are critical inclusion/exclusion criteria clearly stated (does not require the reader to infer)? Type 2 diabetes mellitus: eligibility criteria should state that only type 2 diabetes mellitus patients were included in the study, or subjects with prescriptions for non-insulin blood glucose lowering drugs were included. Q2 Studies on specific subpopulations (e.g., individuals with heart failure or prior myocardial infarction): The authors should state clearly how the subpopulation was indentified. This includes the source of the information (codes in medical records or health care claims, drug use, etc.) Studies on incident outcomes: the authors should state clearly that the study subjects were free of the outcome at enrollment (e.g., heart failure, myocardial infarction, stroke, ventricular arrhythmias). In studies where the outcome is hospitalization for heart failure, study subjects are not required to be free of heart failure. 4

5 The description should include the duration of the disease-free interval to define absence of disease (for instance, individuals with no recorded codes for myocardial infarction in the year prior to enrollment were considered as not having a history of myocardial infarction). New-user design studies: the authors should state clearly how the new users were identified (or how the prevalent users were identified for exclusion). The description should include: a) new use of what (e.g., the compound, the drug class) b) the duration of the drug-free interval to qualify for new use (for example, a prescription for drug A after at least 1 year of no prescriptions for drug A was considered a new prescription for drug A) or when the drug-free interval started (e.g., newly treated within the study period ). Studies comparing active treatments: the authors should clearly state whether users of more than one comparison drug were included or excluded. If included, the authors should state how exposure was assigned, but that is covered in a later question. In studies comparing active treatments, the authors should clearly state whether users of other antidiabetic drugs (not included in the comparison insulin, for instance) were included or excluded. Q3 Are the inclusion/exclusion criteria measured using valid and reliable measures? Q3 Type 2 diabetes mellitus should be ascertained on the basis of physician reports, medical records, hospital discharge records, codes in electronic medical records or administrative data, non-insulin Cannot determine t applicable (NA). antidiabetic drug use combined with diagnostic codes, or combinations. Evaluate as partially if study subjects were identified only based on prescriptions for antidiabetic medications (some might have had other conditions such as polycystic ovarian syndrome). Studies on specific subpopulations: diagnoses of heart failure based on drug use only should not be considered reliable. Diagnoses of non-fatal myocardial infarction that did not require hospitalization should not be considered reliable. Studies on incident outcomes: disease-free interval for incident outcome: at least 1 year. New-user design: drug-free interval for new drug use: at least 6 months. Q4 Did the study apply inclusion/exclusion criteria uniformly to all comparison groups/arms of the study? Q4 Cannot determine Q5 Was the strategy for recruiting participants into the study the same across study groups of the study? For field studies. Q5 Cannot determine 5

6 Q6 Does the confidence intervals suggest lack of precision? Subjective appreciation. Q6 Q7 What is the level of detail in describing the intervention or exposure? Details required (note that this should apply to all exposure levels; i.e. exposed and unexposed, or exposed to each comparison drug): Q7 Medium Do the authors specify whether study subjects are receiving only the drug/s of interest (disregard insulin and non blood glucose lowering drugs)? Do the authors specify whether study subjects are receiving insulin? In studies that used as-treated exposure definitions, is it clear how exposure status is assigned to person time as patients switch or add medications? In studies that used intention-to-treat exposure definitions, do the authors specify how the exposure assignment was handled for subjects who switched drugs? (switchers might have been excluded) Q8 Are the important outcomes prespecified by the researchers? (deleted second part of the question) Mark no if the outcome or one outcome is not pre-specified. Q8 Q9 Is the selection of the comparison group appropriate? In case-control studies, was incidence-density sampling used? Q10 Any attempt to balance the selection between the groups (e.g., through stratification, matching, propensity scores) Q9 Cannot determine Q10 or cannot determine 6

7 Q11 Did researchers isolate the impact from a concurrent intervention or an unintended exposure that might bias results, e.g., through multivariate analysis, stratification or subgroup analysis? Exposures to consider (simultaneous to the assessment of exposure): insulin, oral antidiabetic medications, aspirin. Q11 Re-tailored between assessments or do not know Q12 Were the outcome assessors blinded to the intervention or exposure status of participants? only if in outcome ascertainment there was validation. Q12 Q13 Are exposures assessed using valid and reliable measures, implemented consistently across all study participants? if the information on use and duration of use comes from prescriptions issued, prescriptions filled, or prescriptions reimbursed. Q13, Q14 a, and Q14b Cannot determine if the duration of exposure is assumed rather than retrieved from data. Q14a Are outcomes assessed using valid and reliable measures, implemented consistently across all study participants? if outcome information is derived from hospital discharge records, medical records, or validated claim codes (either if validating within the study or applying previously validated algorithms or codes). Q14b Are outcomes assessed using valid and reliable measures, implemented consistently across all study participants? if, in cohort studies, the ascertainment of outcome was independent of exposure. if, in case-control studies, the selection of study participants was independent of exposure. Q15 Is the length of follow up the same for all groups? NA except when censoring criteria for exposed and unexposed differ. Q15 Re-tailored between assessments or cannot determine 7

8 Q16 Is the length of time following the exposure sufficient to support the evaluation of primary outcomes and harms? 1 year minimum (base assessment on median or mean if provided) Q16 Cannot determine Q17 Did attrition differ between groups by more than 20%? Q17 Cannot determine Q18 Does the analysis control for baseline differences between Q18 and 19 groups? Adequate control: multivariate adjustment, propensity-score matching (individual or frequency matching), matching on other variables, propensity-score included in the model as a covariate, stratification, Cannot determine NA. inverse probability weighting, restriction. Minimum variables for adjustment: age and sex (should be included in the propensity score) Q19 Are confounding and/or effect modifying variables assessed using valid and reliable measures across all study participants? Do not rate down because there is no analysis on effect modification. Do rate down if there is such analysis and the effect modifiers are not defined with valid and reliable measures. Q20 Were the important confounding and effect modifying variables taken into account in the design and/or analysis (e.g., through matching, stratification, interaction terms, MV analysis, or other statistical adjustment)? Consider: Q20 Cannot determine Common cardiovascular risk factors (e.g., smoking or body mass index) Time since diabetes diagnosis (in incident diabetic patients, this variable is not needed) Severity: microvascular complications, previously treated with insulin, or HbA1c (any of these is enough) Previous history of outcomes (if study is not on incident outcomes only) Cardiovascular risk factors: lipid profile or treatment, hypertension, blood pressure or antihypertensive treatment 8

9 Q21 In cases of loss to follow up (or differential loss to follow up), is the impact assessed (e.g., through sensitivity analysis or other adjustment method)? Include here as loss to follow-up the individuals, in database studies with complete case analysis, that are lost because of missing data, whether the authors mention it explicitly or not. Q21 Cannot determine Provide the specific amount and the evaluation performed by the researchers if any. The abstractor should also critically evaluate if he/she thinks that there is a compromise based on the % and differential losses with the data provided, but no based on a pre-specified threshold. If there is evidence of missing data and no assessment, answer no. If there is evidence of missing data and there is an assessment of it (even if only in the discussion), answer yes. If the evaluation it is not clear based on the information provided, we will discuss among investigators for consensus. Q22 Have any important intermediate variables been excluded from the adjustment? The only important intermediate variable that should not be adjusted for that we identified up to now is HbA1c, edema or diuretics for the outcome heart failure. Q22 Q23 Are the statistical methods used to assess the main harm or adverse event outcomes appropriate to the data? Exact methods should be used when any cell count < 5. The number of variables in the analysis should be adequate for the data. One rule of thumb is no more than one variable per 10 subjects included in the models. Q23 Cannot determine Do the authors discuss whether the assumptions of the statistical methods hold? The most important ones, and possibly easier to violate, are non informative censoring, proportional hazards, for Cox regression. Time-varying exposure. Q24 Are results believable taking study limitations into consideration? Q24 9

10 Q25 Is the source of funding identified? An explicit report on funding sources Added Q25 Q26 In cohort or nested case-control studies, is the exposure definition adequate to avoid immortal-time bias? te that the question asks about the potential for bias, not the actual presence of the bias, often untestable. Q26 Q27 Are there formulary restrictions? Relate to the information provided in the paper and consider when interpreting results. Q27 Authors mention there are formulary restrictions Authors mention there are no formulary restrictions mention of formulary restrictions Q28 Do study results seem to be affected by confounding by indication? Include channeling bias here, though some authors would prefer not to group them. Q28 Cannot determine Q29 Is the impact of unmeasured confounding important enough to affect the believability of results? Reference Q29 1. Wells G, Shea B, O Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses Available from: Accessed February 5,

11 Section 3. Domains and questions/items in the Newcastle-Ottawa Scale and in our adaptation of the RTI item bank Newcastle-Ottawa Scale Case-control studies Selection Case definition adequate Representativeness of cases Selection of controls Definition of controls Comparability Age and sex Additional factors Exposure Ascertainment of exposure Cases and controls: same ascertainment method Cases and controls: same non-response rate Cohort studies Selection Representativeness of the exposed cohort Selection of the non-exposed cohort Ascertainment of exposure Outcome of interest not present at study start Comparability Age and sex Additional factors Outcome Assessment of outcome Follow-up long enough Adequacy of follow-up RTI item bank Study population definition and selection Prospective/retrospective design: potential for recall bias Prospective/retrospective design: tailored data collection Critical inclusion/exclusion criteria: clearly stated? Critical inclusion/exclusion criteria: valid and reliable measures? Critical inclusion/exclusion criteria: applied uniformly? Strategy for recruitment: same across study groups Precision Exposure Level of detail in describing the exposure Outcome Important outcomes prespecified? Exposure groups Selection of the comparison group adequate? Allocation between the groups: balance Isolation from unintended exposures Blinding Outcome validation independent of exposure status Soundness of information Exposures: valid and reliable measures, consistently implemented? Outcomes: valid and reliable measures? Outcomes: measures consistently implemented? Follow-up 11

12 Newcastle-Ottawa Scale Case-control studies Cohort studies RTI item bank Length of follow-up: same for all groups? Length of follow-up: long enough? Attrition: different across exposure groups Analysis comparability Control for baseline differences Confounding: valid and reliable measures, consistently implemented? Confounding, effect modification: important variables were considered? Analysis outcome Loss to follow-up: assessment of impact? Intermediate variables not controlled for? Statistical methods appropriate? Interpretation Results: believable? Reporting Source of funding identified? Added questions Potential for immortal time bias Formulary restrictions present? Confounding by indication present? Unmeasured confounding present? 12