Sources of uncertainty in epidemiological studies Anssi Auvinen University of Tampere STUK Radiation and Nuclear Safety Authority International Agency for Research on Cancer
Error and bias Concepts not used in their common everyday meaning in epidemiology and statistics Uncertainty, variability Do not imply something being done wrong or incorrectly Can distort the results to any direction Overestimate or underestimate the true effect Generate a false association or hide a real one
Observed result, O O = True Bias Confounding Random error
Random error Chance variation, noise Variability over time, between een subjects, etc. Any direction Overall effect 0 (cancels out) Depends on measures used Crude indicators increase Standardised protocols, calibration, trained research personnel decrease
Measurement Calculating a time-weighted average ~ simplification random error
Measurement Using residential measurements as a proxy for personal exposure Using residential measurements as a proxy for personal exposure ~ simplification random error
Random error If similar for groups compared (non-differential), tends to decrease differences (for dichotomous classification) Differential induces information bias Disease: For cancer relatively straight forward Microscopic confirmation, histological subtype Exposure: Demanding, challenging All sources covered? Long-term measurements? Relevant time period (prior to diagnosis)
Selection bias Distortion of results due to imbalanced inclusion of subjects into the study Non-representative sampling Selective recruitment Participating and non-participating subjects differ The lower the participation proportion, the higher the potential for selection bias More important, however, is the difference between included and non-included subjects In analytical studies, differences between groups to be compared If similar selection, effect cancels out
Selection bias Invited to participate ENTIRE (SOURCE) POPULATION STUDY POPULATION 80% participate 50% exposed 1/2 exposed 50% unexposed 80% participate 1/2 unexposed No selection bias: Study population reflects the source population
Selection bias Invited to participate ENTIRE (SOURCE) POPULATION STUDY POPULATION 50% exposed 80% participate 2/3 exposed 50% unexposed 40% participate 1/3 unexposed Selection bias distorts the results: Study population not equivalent to the source population
Selection bias People with disease Cases 4:4 3:1 People without disease Selection affects cases, but not controls Distorts results Controls 2:6 1:3
True status Exposed Unexposed Cases α β Controls γ δ True status in the source population Ideally, should be reflected in the study population p True odds ratio OR= α / β : γ / δ
Observed in a study Exposed Un- x i = sampling, exposed participation coefficient, how large proportion was included Observed OR=a/b:c/d No selection bias if x a /x c :x b /x d =1 Similar sampling probability for cases and controls given exposure Cases a = x a α b = x b β Controls c=x c γ d=x d δ NOTE: a is the observed value for α, b for β etc (the latter remain unobserved)
Selection bias Valid method of assessment does not reduce selection bias! Can be corrected for in the analysis, a s, but only if well evaluated Requires a separate validation study Selection by factors unrelated to exposure not consequential in casecontrol studies
Selection bias in studies of residential ELF EMF ELF exposure Leukemia risk Socioeconomic status Study participation
Selection bias in EMF epidemiology Residential ELF-EMF: Wire codes Very high current configuration associated with low income levels Very low current configuration associated with high income If participation is related to income, selection bias may exaggerate an effect by 3-24% Guerney et al. Epidemiology 1995
Selection bias in mobile phone studies Participation i i among controls 42-74% If invited, do those with (abundant) mobile phone use participate i t more (or less likely) l than those not using mobile phones? Eligible controls who agreed to participate Regular mobile phone use 69% Eligible controls who refused full interview Regular mobile phone use 56% Selection bias may underestimate OR by 10%
Information bias Differential information for the compared groups Cases and control in a case-control study Quality, yprecision, or amount of information Validity ~ correctness of information Sensitivity: Ability to detect those with feature Observed positive results/all true positives Specificity: Capacity to identify those free of it Observed negative results/all true negatives Not attributable to selection/inclusion of subjects
Information bias Recall bias: Differential reporting of past exposures for cases and controls Particularly more comprehensive exposure y p p history obtained from cases than controls
Information bias Depends on the Method of assessment Feature or event of interest Objective or subjective source of information Verification of reports Highest potential for bias if Low-impact everyday aspect of life p y y p Changes over time Perceived as related to the outcome
Information bias in mobile phone studies Quality of interview for mobile phone use Glioma cases 40% very good Controls 50% very ygood Validation: Reported vs. recorded For cases reported relative to recorded amount of use higher for use in the past (but not for controls) Vrijheid et al. JESEE 2009
Overestimation of call-time by time before interview 2.5 P trend <0.001 2 P trend 0.60 1.5 1 0.5 <1 1-2 2-3 23 3-4 >4 0 Cases Controls
Confounding To distinguish the effect of EMF, need to control for the effects of other factors Confounding: Effect of other determinants of risk (til (etiologic i factors) Negative confounding: Hides or reduces a true effect Positive confounding: Generates a spurious association or overestimates a true effect Can be corrected in the analysis If confounder measured reasonably well Easier to deal with than other biases
Confounding EMF exposure Disease risk Confounding Risk factor
Confounding Most non-infectious disease have multifactorial etiology Cardiovascular disease: Life-style factors, physiologic factors etc Lung cancer: Tobacco Not a major issue for leukemia, brain tumors Few well established risk factors Possibility of unknown confounders remains
Summary Several sources of uncertainty Random error (chance) Bias: Selection bias, information bias, confounding Can be minimised by design and conduct Subject selection and enrolment Assessment of exposure Need to be considered in the interpretation of the findings First examine non-causal explanations
Scientific results are not infallible, or free from uncertainty. Controversies are an essential aspect of science. The core of the scientific method is critical thinking and careful appraisal. S i tifi th d i t h i Scientific method is a technique for making productive use of doubt. John Dewey
This paper was produced for a meeting organized by Health & Consumers DG and represents the views of its author on the subject. These views have not been adopted or in any way approved by the Commission and should not be relied upon as a statement of the Commission's or Health & Consumers DG's views. The European Commission does not guarantee the accuracy of the data included in this paper, nor does it accept responsibility for any use made thereof.