Causal Inference from Complex Observa4onal Data. Samantha Kleinberg

Size: px

Start display at page:

Download "Causal Inference from Complex Observa4onal Data. Samantha Kleinberg"

Harriet Griffin
6 years ago
Views:

1 Causal Inference from Complex Observa4onal Data Samantha Kleinberg

2 Three key points We need causal knowledge Causes are hard to find It s not hopeless!

3 Most striking, society will need to shed some of its obsession for causality in exchange for simple correla4ons: not knowing why but only what. Mayer- Schonberger, V. and K. Cukier. (2013) Big Data: A Revolu3on That Will Transform How We Live, Work, and Think. Earnon Dolan/Houghton Mifflin Harcourt, (page 7).

4 Causal claims are everywhere

5 Why do we need causes? Predic4on Explana4on Interven4on

6 Predic4on Blackouts Season Smoking rate match sales lung cancer rate

7 Predic4on, con4nued Gene muta4on ê exercise tolerance Disease A

8 Explana4on (1) Why are two variables related? Diabetes CKD Blurred vision Weight loss Renal failure medica4on

9 Explana4on (2) General causes of illness vs. cause of a specific pa4ent s illness Why did an event happen? Why did a par4cular person develop lung cancer at age 42? What led to the U.S. recession in 2007? Is a stroke pa4ent s secondary brain injury due to seizures?

10 Automa4ng explana4on Methods for finding causes from data, but what about explaining events? Prac4cal problem, but challenging Informa4on incomplete Where do explana4ons come from? General and singular can differ Kleinberg (2013) Causality, Probability and Time

11 Interven4on Why do we need causes to take ac4on? Buying stocks Taking vitamins Decreasing sodium to prevent hypertension What happens if we intervene on a correlated factor?

12 Using causes to guide interven4on Blackouts Season Smoking rate match sales lung cancer rate

13 hdp://xkcd.com/552/

14 Correla4on without causa4on: nonsta4onary 4me series hdp://bama.ua.edu/~spren4c

15 AUTISM ORGANIC hdp://imgur.com/1wz6h

16 Correla4on without causa4on: hidden common causes

Neural correlates of interspecies perspec4ve taking in the post-

17 Correla4on without causa4on: mul4ple tes4ng Benned, C. M., Miller, M. B., & Wolford, G. L. (2009). Neural correlates of interspecies perspec4ve taking in the post- mortem atlan4c salmon: An argument for mul4ple comparisons correc4on. NeuroImage, 47(1), 125.

18 Mul4ple comparisons hdp://xkcd.com/882/

19 Causa4on without correla4on: Simpson s paradox Treatment Dead Alive A (72%) B (80%) Total

20 Causa4on without correla4on: Simpson s paradox Treatment Men Women Combined Dead Alive Dead Alive Dead Alive A (60%) 5 95 (95%) (72%) B (50%) (85%) (80%) Total Baker SG, Kramer BS (2001) Good for women, good for men, bad for people: Simpson's paradox and the importance of sex- specific analysis in observa4onal studies. Journal of women's health & gender- based medicine 10:

21 Hume Regularity Counterfactual Mill Mackie (INUS) Probabilis4c causality Lewis Granger causality Eells Suppes Reichenbach Logic- based Bayesian nets

22 Smoking x x Lung cancer Yellowed fingers y y Bayesian networks t t+1 [Pearl, 2000; Spirtes, Glymour and dynamic Bayesian networks Scheines, 2000] P(X t+1 W t ) P(X t+1 W t - Y t ) Granger causality [Granger, 1980]

23 Why is causal inference hard? No single defini4on No fail- proof method for finding it Observa4onal data

24 Philosophy: What is a cause?

25 Philosophy: What is a cause? Computer Science: How can we automate inference/ explana4on? How do we learn of causes?

26 Psychology: How do we gain and use causal knowledge? What s the rela4onship between moral and causal judgment? Philosophy: What is a cause? How do we learn of causes? Computer Science: How can we automate inference/ explana4on?

27 What s the rela4onship between moral and causal judgment? Psychology: How do we gain and use causal knowledge? Philosophy: What is a cause? How do we learn of causes? Computer Science: How can we automate inference/ explana4on? Epidemiology: What affects human health? Large- scale analysis of EHRs

28 What s the rela4onship between moral and causal judgment? Psychology: How do we gain and use causal knowledge? Philosophy: What is a cause? How do we learn of causes? Medicine/biology: Applica4ons to neuroscience, genomics Computer Science: How can we automate inference/ explana4on? BNs RCTs Epidemiology: What affects human health? Large- scale analysis of EHRs

Why do people behave as they do? Economics: Do policies achieve goals? Granger causality What s the rela4onship between moral and causal judgment? Psychology: How do we gain and use causal knowledge?

29 Why do people behave as they do? Economics: Do policies achieve goals? Granger causality What s the rela4onship between moral and causal judgment? Psychology: How do we gain and use causal knowledge? Philosophy: What is a cause? How do we learn of causes? Medicine/biology: Applica4ons to neuroscience, genomics Computer Science: How can we automate inference/ explana4on? BNs RCTs Epidemiology: What affects human health? Large- scale analysis of EHRs

30 Three main ques4ons What is a cause? Theories of what dis4nguishes them from correla4ons and how we can iden4fy them How can we find causes? Features of causes that allow us to learn about them When can we infer causes? Methods for inference from data Study design Applica4ons to challenging cases

31 Cause Smoking à LC Evidence Probability: are smokers more likely to get lung cancer? Mechanisms: how could smoking plausibly cause LC? Interven4on: If I made people smoke, would they get lung cancer? Methodology

32 CHF progression Stage A High Risk Stage B Asymptoma4c heart failure Stage C Symptoma4c heart failure Stage D Endstage heart failure Pa4ents have: - Hypertension - Coronary artery disease - diabetes - Prior heart adack - LV disfunc4on - Asymptoma4c valvular disease - Structural heart disease - Shortness of breath and fa4gue - Reduced exercise tolerance - Recurrent hospitaliza4on - Marked symptoms at rest despite interven4on Treatments are: - Treat high blood pressure - Encourage exercise Treatments for A plus, some cases - ACE inhibitors - Beta blockers Treatments for A plus: - Salt restric4on - ACE inhibitor - Beta blocker Treatments for A, B, C and: - Heart transplant - Hospice Adapted from ACC/AHA Guidelines for the Evaluation and Management of Chronic Heart Failure in the Adult Hunt et al. 2001

33 Electronic health record (EHR) data Poten4al Find drug side effects Markers for disease Challenges Cannot do all possible tests Gaps in treatment, change of providers Ambiguity in 4ming Observa4on of symptoms, not disease itself

34 Logic- based causal inference Complex, temporal rela4onships v ; 15,apple g Poten4al causes raise probability of and are earlier than effects P (e c) >P(e) Kleinberg & Mishra (UAI 2009)

35 Which causes are significant? Main idea: looking for beder explana4ons for the effect Assess average difference cause makes to probability of effect " avg (c, e) = P x2x c P (e c ^ x) P (e c ^ x) X\c Determine which ε are sta4s4cally significant

36 Inferring 4ming Instead of accep4ng/rejec4ng hypotheses, refine them from data Can start by tes4ng rela4onships between all variables and CHF in 1-2 weeks, and ul4mately infer "high AST leads to CHF in 4-10 days Kleinberg, S. (2013) Causality, Probability, and Time.

37 Intui4on behind procedure 1) Too wide c 2) Shifted r x s 3) Too narrow actual r' s' " avg (c, e) = P x2x c P (e c ^ x) P (e c ^ x) X\c P (e c ^ x) = #(c ^ x ^ e) #(c ^ x)

Name FDR FNR Avg. # itera4ons W7 0.046 0.003 6.31 W13 0.045 0.002 6.65 W27 0.151 0.006 5.76 PWR2 0.226 0.002 4.66 PWR4 0.462 0.011 3.

38 Name FDR FNR Avg. # itera4ons W W W PWR PWR Note: Total number of FP in W7 is 3, while in PWR4 it is 12, and W27 8. Timing inference start end Name mean med. std. mean med. std. W W W PWR PWR

39 Geisinger CHF data 3,838 cases, 28,843 controls Average of 7 years of data, 86 encounters (2.2 years pre diagnosis) Variables Comorbidities and problem list Vitals (BP, height, weight, pulse, temperature) Medica4ons Lab values

40 Results dx_hypothyroidism dx_diabetes dx_overweight 10 9 first_an4htn_combo 11 8 dx_urinary_symptoms Months before CHF diagnosis

41 Replica4on Geisinger (~32K pa4ents) in PA Stable, rural popula4on 95% white Columbia (~13K pa4ents in AIM clinic) in NY Significant in/out- migra4on, urban 57% white, 28% black, 15% asian S. Kleinberg and N. Elhadad (2013) Lessons Learned in Replica4ng Data- Driven Experiments in Mul4ple Medical Systems and Pa4ent Popula4ons. AMIA Annual Symposium.

42 Data comparison Geisinger popula4on 3,838 cases, 28,843 controls Structured comorbidi4es and problem list Vitals (BP, height, weight, pulse, temperature) Medica4ons Lab values CUMC popula4on 1,853 cases 11,765 controls Medica4on orders ICD9 codes Lab values

43 CHF - CUMC potassium_channel_block PCO2A- cri4cal- low dx_angina dx_copd_asthma ICAI loop_diure4c salicylates heparin an4coag biguanide beta_block thyroid_hormone cardiac_glycosides insulin calcium_block Months before CHF diagnosis

45 Result comparison Some common findings Diabetes (insulin) Thyroid dysfunc4on (hypothyroidism, thyroid hormone prescrip4on) Hypertension (an4hypertensive combina4on medica4ons, calcium channel/beta blockers Geisinger onset 4mings likely more accurate, popula4on larger

46 Opera>onal criteria 1 outpa4ent visit or 1 medica4on order 2+ medica4on orders only 2+ outpa4ent visits only 1 outpa4ent visit and 1 medica4on order only Problem list only Meet 2 of 3 criteria Meet all 3 criteria none 1 minor, 0 major Number of Framingham criteria 2+ minor, 0 major 0 minor, 1 major 1 minor, 1 major 1 major, 2+ minor 2+ major Total Meets opera4onal criteria and Framingham criteria, N=2294 (35%) Meets opera4onal criteria but not Framingham criteria, N=2900 (45%) Meets Framingham criteria but not opera4onal criteria, N=424 (7%) Does not meet opera4onal criteria or Framingham criteria, N=879 (14%) Total Wu et al. (2010). Prediction Modeling Using EHR Data: Challenges, Strategies, and a Comparison of Machine Learning Approaches. Med Care 48(6):S

48 hdp://discoverysedge.mayo.edu/ar4ﬁcial- pancreas/

49 Uncertain data Error in measurements Missing data Delay Inconsistent 4mescales

50 Discre4za4on 1" 0.9" 0.8" 0.7" 0.6" 0.5" 0.4" 0.3" 0.2" 0.1" 0" 1" 11" 21" 31" 41" 51" 61" 71" 81" 91" 101" 111" 121" 131" 141" 151" 161" 171" 181" 191" 201" 1" 0.9" 0.8" 0.7" 0.6" 0.5" 0.4" 0.3" 0.2" 0.1" 0" 1" 11" 21" 31" 41" 51" 61" 71" 81" 91" 101" 111" 121" 131" 141" 151" 161" 171" 181" 191" 201" hypo" eu" hyper"

51 Adding uncertainty P (e c, x) = P P t ecx t cx P (e c, x) = P P (e, c, x) Pt P (c, x) t D. Hutchison and S. Kleinberg (2013) Causality and Experimenta4on in the Sciences.

52 Causes of changes in glucose Cohort: 17 subjects with T1DM Sensor data (collected for >72 hours) Glucose values Insulin dosage Ac4vity Sleep stage Heart rate Temperature With N. Heintzman (UCSD)

53 Results very vigorous exercise leads to hyperglycemia (fdr <.01) in 5-30 minutes Found using both HR (anaerobic ac4vity zone) and METs Not found with strict discre4za4on Supported by literature (Marliss and Vranic, 2002; Riddell and Perkins, 2006)

54 Rare events Financial markets Twider: 8 TB/day ICU: 5 sec measurements

55 Why not use current methods? Rare event mining No informa4on about impact of events False alarms Causal inference Probabilis4c: can t handle infrequent events Granger: assesses whole 4me series Complexity: efficiency is key with big data

56 Approach Infer normal model Use huge volume of data Find how much is not explained by model Determine how explanatory rare event is Comparing to average dis4nguishes between unmeasured and infrequent events Kleinberg (2013) IJCAI

57 Experiments Simulated financial 4me series data 5 structures [next slide]: 2 with 10 causal rela4onships, 3 with 20 1 or 3 rare causes in each 3 different probabili4es for rare events (0.005, , ) 25 variables 4,000 or 10,000 4me points 480 (5 structures, 3 probabili4es, 4 runs each, 2 types of rare causes, 2 observa4on lengths, 2 4me periods)

58 Structures

59 Results 4,000 4mepoints 10,000 4mepoints

60 Mo data mo problems Big good Uncertainty Selec4on bias Signal:noise Interpreta4on Time Ground truth

61 Open problems (known unknowns) Hidden variables Rela4onships that change over 4me Hypothesis genera4on Es4ma4ng uncertainty Tes4ng assump4ons Automa4ng explana4on

Causal Inference from Complex Observational Data. Samantha Kleinberg Stevens Institute of Technology

Causal Inference from Complex Observational Data Samantha Kleinberg Stevens Institute of Technology samantha.kleinberg@stevens.edu Three key points We need causal knowledge Causes are hard to find and