Biases in clinical research. Seungho Ryu, MD, PhD Kanguk Samsung Hospital, Sungkyunkwan University

Transcription

1 Biases in clinical research Seungho Ryu, MD, PhD Kanguk Samsung Hospital, Sungkyunkwan University

2 Learning objectives Describe the threats to causal inferences in clinical studies Understand the role of random variability in clinical studies Describe, understand, and learn how to control the 3 main types of bias: Confounding Information bias / measurement error Selection bias Discuss the concept of generalizability of study results DCR Chapters 4 and 9 2

3 Threats to causal inference Truth in the Universe infer Truth in the Study infer Findings in the study Research Question Random and systematic error Study Plan Random and systematic error Actual Study Target Population Intended Sample Actual subjects Design Implementation Phenomena of interest Intended variables Actual measurement s EXTERNAL VALIDITY INTERNAL VALIDITY 3

4 Bias Systematic difference between the true value and the measured value How close is the measured value to the true value? Synonym for bias: lack of validity Validity èon average, the measurement estimates the true measurement

5 Real example of random error! Body weight, bathroom scale True body weight 180 lbs Inconsistent scale, but set correctly at 0 lbs Moments apart: 1 st measurement: lbs 2 nd measurement: lbs 3 rd measurement: lbs 4 th measurement: lbs

6 Real example of bias! Body weight, bathroom scale True body weight 180 lbs Consistent scale, but fail to set it at 0 lbs: reads -5 lbs Moments apart: 1 st measurement: 175 lbs 2 nd measurement: 175 lbs 3 rd measurement: 175 lbs 4 th measurement: 175 lbs

7 Threats to causal inference Lack of precision Random variability - by chance We may observe an association that does not exist or may fail to observe an existing association Lack of internal validity Bias - Systematic errors Confounding Information bias / measurement error Selection bias 7

8 Threats to causal inference (continued) Incorrect assessment of the direction of causality: We believe that A è B But, in reality A ç B Lack of external validity (generalizability): True effect in the study population But, does not apply to other populations 8

9 Smoking and CVD mortality NHANES II Mortality Study Sample size: 9,205 Length of follow-up: 16 years Prevalence at baseline Current smokers: 32.2% Former smokers: 26.8% Never smokers: 41.0% Hazard ratio for all-cause mortality: Current vs. never smokers: 2.08 (95% CI ) Former vs. never smokers: 1.32 (95% CI ) 9

10 Smoking and CVD mortality NHANES II Mortality Study Hazard ratios for mortality in random samples of N = 500 Ex-smokers Curr. smk [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

11 Smoking and CVD mortality NHANES II Mortality Study Hazard ratios for mortality comparing current to never smokers N = 500 N = 1, N = 5,000 N = 9,205

12 Smoking and CVD mortality NHANES II Mortality Study Hazard ratios for mortality comparing former to never smokers N = 500 N = 1, N = 5,000 N = 9,205 12

13 Precision vs. validity Meta-analysis of long-term large randomized controlled trials of statins and coronary heart disease endpoints Cheung BMY, et al. Br J Clin Pharmacol 2004;57:

14 Streptokinase in AMI Meta-analysis Lau J, et al. N Engl J Med 1992;327:

15 Bias definition Deviation of results or inferences from the truth Any trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically different from the truth Last JM, ed. A dictionary of epidemiology, 4th ed. Oxford, Oxford University Press,

16 Bias classification Many different biases have been described Sackett DL. Bias in analytic research. J Chron Dis 1979;32:51-63 Delgado-Rodriguez M, Llorca J. J Epidemiol Community Health 2004;58: general types of biases: Confounding Misclassification / Information bias Selection bias 16

17 From causal effect to data

18 What is the counterfactual? Example: The risk experience an exposed individual would have had had he/she not been exposed, with all else being equal. The risk experience an unexposed individual would have had had he/she been exposed, with all else being equal. Not possible to follow the same person at the same time with and without exposure! The counterfactual is like a parallel hypothetical universe We use the counterfactual to describe our ideal reference group 18

19 Extension of the counterfactual model to a group of individuals Is risk of heart attack higher in obese individuals than it would have been if the individuals had not been obese? Actual state (at a given time and place) Same people Obese Presenter s Name individuals 1 to i Date R obese Counterfactual state (at the same time and place) Heart attack Not obese individuals 1 to i R not obese Heart attack 19

20 Counterfactual model: all else being equal? How to address the real world question: Is risk of heart attack higher in obese individuals than in non-obese individuals? NOT the same people Exposed group (at a given time and place) Obese R obese individuals 1 to i Presenter s Name Date Comparison group (at a given time and place) Not obese R not obese individuals i+1 to n Heart attack Heart attack These individuals should differ from the exposed individuals only on the exposure. 20

21 Advantages of randomization Produces comparable groups with respect to observed and to unobserved factors Control of confounding Control of selection bias of study participants at baseline Helps define exposure to intervention Presenter s Name Randomization is the time of onset of follow-up (time Date 0) Clearly defined groups in terms of intervention Provides a firm basis for statistical inferences Adds credibility to the findings

22 Observational effect of antihypertensive medication ARIC ARIC included 5,504 hypertensives Systolic blood pressure 140 mmhg Diastolic blood pressure 90 mmhg Use of antihypertensive medication Use of antihypertensive medication Presenter s Name Yes: 4,003 subjects (73%) Date No: 1,484 subjects (27%) Missing: 17

23 Observational effect of antihypertensive medication ARIC Proportion free of CHD Presenter s Name HR treated vs. untreated = 1.49 ( ) P < Treated Untreated 0.85 Date Survival time (y)

24 Comparison of treated and untreated hypertensives in ARIC Untreated Treated p (n = 1,484) (n = 4,003) Age (y) 55.3 (5.7) 55.6 (5.6) 0.09 Gender (% female) < Race (% white) Center (% W ) < BMI (kg/m 2 ) 28.6 (5.8) 29.9 (5.9) < Waist-hip ratio 0.94 (0.07) 0.95 (0.07) SBP (mmhg) (15.2) (20.0) < DBP (mmhg) Presenter s 87.1 Name(11.9) 78.4 (11.6) < Serum chol. (mmol/l) 5.64 (1.12) 5.68 (1.16) 0.17 Date HDL-chol. (mmol/l) 1.36 (0.46) 1.27 (0.42) < Current smokers (%) Current drinkers (%) < Diabetics (%) < Prevalent CHD (%) < Prevalent angina (%) <0.001 Prevalent stroke/tia (%) < Values are means (SD) or percentages

25 Confounders have to 1. Cause the disease (or be a surrogate measure of a cause) AND 2. Be associated with exposure (i.e., be distributed differently between exposed and unexposed), AND 3. Not affected by exposure (i.e., not be an intermediate variable in the causal pathway) Note: the 3 conditions are necessary for a variable to be a confounder 25

26 Concepts of confounding Response (R) Exposure (E)

27 Concepts of confounding Response (R) C is associated with E C causes R CONFOUNDING C = 1 C = 0 Exposure (E)

28 Concepts of confounding Response (R) C is associated with E C causes R CONFOUNDING C = 1 C = 0 Exposure (E)

29 Concepts of confounding MI No MI Coffee No coffee Odd ratio (OR)= Smokers Nonsmokers MI No MI MI No MI Coffee No Coffee OR in smokers= OR in nonsmokers=

30 Concepts of confounding Response (R) C does not cause R C is not associated with E C = 1 C = 0 Exposure (E)

31 Concepts of confounding Response (R) C is associated with E C does not cause R C = 1 C = 0 Exposure (E)

32

33 Confounding Smith GD, et al. BMJ 1997;315:

34 Asking about sex Smith GD, et al. BMJ 1997;315:

35 Comparability of exposure groups Smith GD, et al. BMJ 1997;315:

36 Sex and mortality Results Smith GD, et al. BMJ 1997;315:

37 Sex and mortality Recommendations! Smith GD, et al. BMJ 1997;315:

38 Causal associations Low sex frequency Death from myocardial infarction Low sex frequency Death from myocardial infarction Poor health

39 Marmor M, et al. Lancet 1982;1:

40 Hypotheses Marmor M, et al. Lancet 1982;1:

41 Methods Marmor M, et al. Lancet 1982;1:

42 Results and interpretation Marmor M, et al. Lancet 1982;1:

43 Amyl nitrite, HIV infection, and AIDS Sexual behavior HIV infection AIDS Use of amyl nitrite HIV infection causes AIDS HIV infection and use of amyl nitrite were associated in homosexual men 43

44 Mediator Mediators are: Affected by exposure On causal pathway between exposure and disease Cause of disease Mediators are intermediate variables, translating at least part of the effect of exposure on disease

45 Causal diagram Physical activity, HDL cholesterol, and MI Low physical activity Low HDL cholesterol Myocardial infarction Low physical activity is a cause low HDL cholesterol Low HDL cholesterol is a cause of myocardial infarction HOWEVER, low HDL cholesterol is an intermediate variable in the causal pathway between physical activity and myocardial infarction 45

46

47

48

49

50

51

52

53

54

55

56

57

58 Enrollment and follow-up in HERS Grady D, et al. JAMA 2002;288:

59 Grady D, et al. JAMA 2002;288:

60 60

61 van Vollenhoven, et al. Lupus 1999;8:

62 62

63 Jiang R, et al. JAMA 2002;288:

64 Uncontrolled confounding Unmeasured confounders Unknown confounders Known confounders that are too expensive or difficult to measure Residual confounding Confounder is measured imperfectly, and cannot be controlled completely 64

65 Results and interpretation Marmor M, et al. Lancet 1982;1:

66 In practice (I) Prior knowledge on the biological and other causal relationships is needed to properly identify which variables to adjust for Do NOT apply statistical criteria to decide if the conditions for confounding are present Testing for the association of confounder with exposure and of confounder with disease Stepwise selection procedures Consider if exposed and unexposed subjects are comparable with respect to their risk of disease (except for exposure) 66

67 In practice (II) Consider which determinants of disease may be responsible for the lack of comparability Elaborate causal diagram Identify causal factors that may be different between exposed and unexposed Obtain information on potential confounders Measuring confounders with error will result in residual confounding after adjustment Use statistical techniques to adjust for potential confounders 67

68 Methods to control for confounding In the design of the study Randomization Restriction Matching è primarily in case-control studies In the analysis Standardization Stratification Multivariate models Propensity scores Inverse probability weighting Sensitivity analysis 68

69 Thank you for your attention