Causal Thinking in the Twilight Zone

Size: px
Start display at page:

Download "Causal Thinking in the Twilight Zone"

Transcription

1 Observational Studies 1 (2015) Submitted 7/15; Published 8/15 TECHNICAL REPORT R-456 August 2015 Causal Thinking in the Twilight Zone Judea Pearl Computer Science Department University of California, Los Angeles Los Angeles, CA 90095, USA judea@cs.ucla.edu To students of causality, the writings of William Cochran provide an excellent and intriguing vantage point for studying how statistics, lacking the necessary mathematical tools, managed nevertheless to cope with increasing demands for policy evaluation from observational studies. Cochran met this challenge in the years , when statistics was preparing for a profound, albeit tortuous transition from a science of data, to a science of data generating processes. The former, governed by Fisher s dictum (Fisher, 1922) the object of statistical methods is the reduction of data was served well by the traditional language of probability theory. The latter, on the other hand, seeking causal effects and policy recommendations, required an extension of probability theory to facilitate mathematical representations of generating processes. No such representation was allowed into respectable statistical circles in the s, when Cochran started looking into the social effects of public housing in Baltimore. While data showed improvement in health and well-being of families that moved from slums to public housing, it soon became obvious that the estimated improvement was strongly biased; Cochran reasoned that in order to become eligible for public housing the parent of a family may have to possess both initiative and some determination in dealing with the bureaucracy, thus making their families more likely to obtain better healthcare than non-eligible families. 1 This led him to suggest adjustment for covariates for the explicit purpose of reducing this causal effect bias. While there were others before Cochran who applied adjustment for various purposes, Cochran is credited for introducing this technique to statistics (Salsburg, 2002) primarily because he popularized the method and taxonomized it by purpose of usage. Unlike most of his contemporaries, who considered cause-effect relationships ill-defined outside the confines of Fisherian experiments, Cochran had no qualm admitting that he sought such relationships in observational studies. He in fact went as far as defining the objective of an observational study: to elucidate causal-and-effect relationships in situations where controlled experiments are infeasible (Cochran, 1965). Indeed, in the paper before us, the word cause is used fairly freely, and other causal terms such as effect, influence, and explanation are almost as frequent as regression or variance. Still, Cochran was well aware that he was dealing with unchartered extra-statistical territory and cautioned us: Claim of proof of cause and effect must carry with it an explanation of the mechanism by which this effect is produced. 1. Narrated in Cochran (1983, p. 24) c 2015 Judea Pearl.

2 Causal Thinking in the Twilight Zone Today, when an analyst declares that a claim depends on the mechanism by which an effect is produced we expect the analyst to specify what features of the mechanism would make the claim valid. For example, when Rosenbaum and Rubin (1983) claimed that propensity score methods may lead to unbiased estimates of causal effects, they conditioned the claim on a counterfactual assumption named strong ignorability. Such identifying assumptions, though cognitively formidable, provided a formal instrument for proving that some adjustments can yield unbiased estimates. Similarly, when a structural analyst makes the claim that an indirect effect is estimable from observational studies, the claim must follow assumptions about the structure of the underlying graph which, again, assures us of zero-bias estimates (see Pearl, 2014b). Things were quite different in Cochran s era; an appeal to a mechanism, like an appeal to subject matter information stood literally for a confession of helplessness, since mechanisms and causal relationships had no representation in statistics. Structural equation models (SEM), the language used by economists to represent mechanisms, were deeply mistrusted by statisticians, who could not bring themselves to distinguish structural from regression models (Guttman, 1977; Freedman, 1987; Cliff, 1983; Wermuth, 1992; Holland, 1995). 2 Counterfactuals, on the other hand, were still in the embryonic state that Neyman left them in symbols with no model, no formal connection to realizable variables, and no inferential machinery with which to support or refute claims. 3 Fisher s celebrated advice: make your theories elaborate was no help in this transitional era of pre-formal causation; there is no way to elaborate on a theory that cannot be represented in some language. It is not surprising, therefore, that Cochran s conclusions are quite gloomy: It is well known that evidence of a relationship between x and y is no proof that x causes y. The scientific philosophers to whom we might turn for expert guidance on this tricky issue are a disappointment. Almost unanimously and with evident delight they throw the idea of cause and effect overboard. As the statistical study of relationships has become more sophisticated, the statistician might admit, however, that his point of view is not very different, even if he wishes to retain the terms cause and effect. It is likewise not surprising that in the present article, Cochran does not offer readers any advice on which covariates are likely to reduce bias and which would amplify bias. Any such advice, as we know today, requires a picture of reality, which Cochran understood to be both needed and lacking at his time. 4 On the positive side, though, he did have the vision to anticipate the emergence of a new type of research paradigm within statistics, a paradigm centered on mechanisms: A claim of proof of cause and effect must carry with it an explanation of the mechanism by which the effect is produced. Except in cases where the 2. This mistrust persists to some degree even in our century, see Berk (2004) or Sobel (2008). 3. These had to wait for Rubin (1974), Robins (1986), and the structural semantics of Balke and Pearl (1994). 4. To the best of my knowledge, the only adjustment-related advice in the entire statistics literature prior to 1980 was Cox s warning that the concomitant observations be quite unaffected by the treatments (Cox, 1958, p. 48) ; it was the first defiance of an unwritten taboo against the use of data-generating models. 201

3 Pearl mechanism is obvious and undisputed, this may require a completely different type of research from the observational study that is being summarized. I believe the type of research we see flourishing today, based on a symbiosis between the graphical and counterfactual languages (Morgan and Winship, 2014; Vanderweele, 2015; Bareinboim and Pearl, 2015) would perfectly meet Cochran s vision of a completely different type of research. This research differs fundamentally from the type of research conducted in Cochran s generation. First, it commences with a commitment to understanding what reality must be like for a statistical routine to succeed and, second, it represents reality in terms of data-generating models (read: mechanisms ), rather than probability distributions. Encoded as nonparametric structural equations, these models have led to a fruitful symbiosis between graphs and counterfactuals and have unified the potential outcome framework of Neyman, Rubin, and Robins with the econometric tradition of Haavelmo, Marschak, and Heckman. In this symbiosis, counterfactuals (potential outcomes) emerge as natural byproducts of structural equations and serve to formally articulate research questions of interest. Graphical models, on the other hand, are used to encode scientific assumptions in a qualitative (i.e., nonparametric) and transparent language and to identify the logical ramifications of these assumptions, in particular their testable implications. 5 A summary of results emerging from this symbiotic methodology is given in Pearl (2014a) and includes complete solutions 6 to several long-standing problem areas, ranging from policy evaluation (Tian and Shpitser, 2010) and selection bias (Bareinboim, Tian and Pearl, 2014) to external validity (Bareinboim and Pearl, 2015; Pearl and Bareinboim, 2014) and missing data (Mohan, Pearl and Tian, 2014). This development has not met with universal acceptance. Cox and Wermuth (2015), for example, are still reluctant to endorse the tools that this symbiosis has spawned, questioning in essence whether interventions can ever be mathematized. 7 Others regard the symbiosis as unscientific (Rubin, 2008) or less than helpful (Imbens and Rubin, 2015, p. 22) insisting for example that investigators should handle ignorability judgments by unaided intuition. I strongly believe, however, and I say it with a deep sense of responsibility, that future explorations of observational studies will rise above these inertial barriers and take full advantage of the tools that the graphical-counterfactual symbiosis now offers. References Bareinboim, E. and Pearl, J. (2015). Causal inference from big data: Theoretical foundations and the data-fusion problem. Tech. Rep. R-450, Department of Computer Science, 5. Note that the potential outcome framework alone does not meet these qualifications. Scientific assumptions must be converted to conditional ignorability statements (Rosenbaum and Rubin 1983; Imbens and Rubin, 2015) which, being cognitively formidable, escape the scrutiny of plausibility judgment and impede the search for their testable implications. 6. By complete solution I mean a method of producing consistent estimates of (causal) parameters of interests, applicable to any hypothesized model, and accompanied by a proof that no other method can do better except by strengthening the model assumptions. 7. Unwittingly, the very calculus that they reject happens to resolve the problem that they pose ( indirect confounding ) in just four steps (Pearl, 2015a; Pearl, 2015b) 202

4 Causal Thinking in the Twilight Zone University of California, Los Angeles, CA. Forthcoming, Proceedings of the National Academy of Sciences. Bareinboim, E., Tian, J. and Pearl, J. (2014). Recovering from selection bias in causal and statistical inference. In Proceedings of the Twenty-eighth AAAI Conference on Artificial Intelligence (C. E. Brodley and P. Stone, eds.). AAAI Press, Palo Alto, CA. Best Paper Award, Berk, R. (2004). Regression Analysis: A Constructive Critique. Sage, Thousand Oaks, CA. Cliff, N. (1983). Some cautions concerning the application of causal modeling methods. Multivariate Behavioral Research, 18, Cochran, W. (1965). The planning of observational studies of human population. Journal of the Royal Statistical Society (Series A), Cochran, W. G. (1983). Planning and Analysis of Observational Studies. Wiley, New York. Cox, D. (1958). The Planning of Experiments. John Wiley and Sons, NY. Cox, D. and Wermuth, N. (2015). Design and interpretation of studies: Relevant concepts from the past and some extensions. Observational Studies, 1, Fisher, R.A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London, Series A, 222, Freedman, D. (1987). As others see us: A case study in path analysis (with discussion). Journal of Educational Statistics, 12, Guttman, L. (1977). What is not what in statistics. The Statistician Holland, P. (1995). Some reflections on Freedmans critiques. Foundations of Science, 1, URL Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, New York. Mohan, K., Pearl, J. and Tian, J. (2013). Graphical models for inference with missing data. In Advances in Neural Information Processing Systems 26 (C. Burges, L. Bottou, M.Welling, Z. Ghahramani and K.Weinberger, eds.). Curran Associates, Inc., Morgan, S. L. and Winship, C. (2014). Counterfactuals and Causal Inference: Methods and Principles for Social Research (Analytical Methods for Social Research). 2nd ed. Cambridge University Press, New York. Pearl, J. (2014a). The deductive approach to causal inference. Journal of Causal Inference, 2, Pearl, J. (2014b). Interpretation and identification of causal mediation. Psychological Methods, Pearl, J. (2015a). Indirect Confounding and Causal Calculus (On three papers by Cox and Wermuth). Blog entry: Pearl, J. (2015b). Indirect Confounding and Causal Calculus (On three papers by Cox and Wermuth). Tech. Rep. R-457, Department of Computer Science, University of California, Los Angeles, CA. Pearl, J. and Bareinboim, E. (2014). External validity: From do-calculus to transportability across populations. Statistical Science, 29, Robins, J. (1986). A new approach to causal inference in mortality studies with a sustained exposure period applications to control of the healthy workers survivor effect. Mathematical Modeling, 7,

5 Pearl Rosenbaum, P. and Rubin, D. (1983). The central role of propensity score in observational studies for causal effects. Biometrika, 70, Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, Rubin, D. (2008). Authors reply (to Ian Shriers Letter to the Editor). Statistics in Medicine, 27, Salsburg, D. (2002). The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Henry Holt and Company, LLC, New York. Sobel, M. (2008). Identification of causal parameters in randomized studies with mediating variables. Journal of Educational and Behavioral Statistics, 33, Tian, J. and Shpitser, I. (2010). On identifying causal effects. In Heuristics, Probability and Causality: A Tribute to Judea Pearl (R. Dechter, H. Geffner and J. Halpern, eds.). College Publications, UK, VanderWeele, T. (2015). Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press, New York. Wermuth, N. (1992). On block-recursive regression equations. Brazilian Journal of Probability and Statistics (with discussion), 6,

Causal Thinking in the Twilight Zone

Causal Thinking in the Twilight Zone Forthcoming: Observational Studies Special Issue on William Cochran's contributions. TECHNICAL REPORT R-456 July 2015 Causal Thinking in the Twilight Zone Judea Pearl University of California, Los Angeles

More information

Book review of Herbert I. Weisberg: Bias and Causation, Models and Judgment for Valid Comparisons Reviewed by Judea Pearl

Book review of Herbert I. Weisberg: Bias and Causation, Models and Judgment for Valid Comparisons Reviewed by Judea Pearl Book review of Herbert I. Weisberg: Bias and Causation, Models and Judgment for Valid Comparisons Reviewed by Judea Pearl Judea Pearl University of California, Los Angeles Computer Science Department Los

More information

Propensity Score Analysis Shenyang Guo, Ph.D.

Propensity Score Analysis Shenyang Guo, Ph.D. Propensity Score Analysis Shenyang Guo, Ph.D. Upcoming Seminar: April 7-8, 2017, Philadelphia, Pennsylvania Propensity Score Analysis 1. Overview 1.1 Observational studies and challenges 1.2 Why and when

More information

The Logic of Causal Inference

The Logic of Causal Inference The Logic of Causal Inference Judea Pearl University of California, Los Angeles Computer Science Department Los Angeles, CA, 90095-1596, USA judea@cs.ucla.edu September 13, 2010 1 Introduction The role

More information

Generalizing Experimental Findings

Generalizing Experimental Findings Forthcoming, Journal of Causal Inference, Causal, Casual, and Curious Section. TECHNICAL REPORT R-452 July 2015 Generalizing Experimental Findings Judea Pearl University of California, Los Angeles Computer

More information

Supplement 2. Use of Directed Acyclic Graphs (DAGs)

Supplement 2. Use of Directed Acyclic Graphs (DAGs) Supplement 2. Use of Directed Acyclic Graphs (DAGs) Abstract This supplement describes how counterfactual theory is used to define causal effects and the conditions in which observed data can be used to

More information

Generalizing Experimental Findings

Generalizing Experimental Findings Submitted to Journal of Causal Inference, Causal, Casual, and Curious Section. TECHNICAL REPORT R-452 June 2015 Generalizing Experimental Findings Judea Pearl University of California, Los Angeles Computer

More information

Generalization and Theory-Building in Software Engineering Research

Generalization and Theory-Building in Software Engineering Research Generalization and Theory-Building in Software Engineering Research Magne Jørgensen, Dag Sjøberg Simula Research Laboratory {magne.jorgensen, dagsj}@simula.no Abstract The main purpose of this paper is

More information

ICPSR Causal Inference in the Social Sciences. Course Syllabus

ICPSR Causal Inference in the Social Sciences. Course Syllabus ICPSR 2012 Causal Inference in the Social Sciences Course Syllabus Instructors: Dominik Hangartner London School of Economics Marco Steenbergen University of Zurich Teaching Fellow: Ben Wilson London School

More information

Structural Approach to Bias in Meta-analyses

Structural Approach to Bias in Meta-analyses Original Article Received 26 July 2011, Revised 22 November 2011, Accepted 12 December 2011 Published online 2 February 2012 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/jrsm.52 Structural

More information

Careful with Causal Inference

Careful with Causal Inference Careful with Causal Inference Richard Emsley Professor of Medical Statistics and Trials Methodology Department of Biostatistics & Health Informatics Institute of Psychiatry, Psychology & Neuroscience Correlation

More information

1 Transportability and selection bias

1 Transportability and selection bias TECHNICAL REPORT R-452 July 2015 J. Causal Infer. 2015; 3(2): 259 266 Causal, Casual and Curious Judea Pearl* Generalizing Experimental Findings DOI 10.1515/jci-2015-0025 Abstract: This note examines one

More information

Introduction to Observational Studies. Jane Pinelis

Introduction to Observational Studies. Jane Pinelis Introduction to Observational Studies Jane Pinelis 22 March 2018 Outline Motivating example Observational studies vs. randomized experiments Observational studies: basics Some adjustment strategies Matching

More information

Causal Methods for Observational Data Amanda Stevenson, University of Texas at Austin Population Research Center, Austin, TX

Causal Methods for Observational Data Amanda Stevenson, University of Texas at Austin Population Research Center, Austin, TX Causal Methods for Observational Data Amanda Stevenson, University of Texas at Austin Population Research Center, Austin, TX ABSTRACT Comparative effectiveness research often uses non-experimental observational

More information

August 29, Introduction and Overview

August 29, Introduction and Overview August 29, 2018 Introduction and Overview Why are we here? Haavelmo(1944): to become master of the happenings of real life. Theoretical models are necessary tools in our attempts to understand and explain

More information

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis?

How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? How Does Analysis of Competing Hypotheses (ACH) Improve Intelligence Analysis? Richards J. Heuer, Jr. Version 1.2, October 16, 2005 This document is from a collection of works by Richards J. Heuer, Jr.

More information

PharmaSUG Paper HA-04 Two Roads Diverged in a Narrow Dataset...When Coarsened Exact Matching is More Appropriate than Propensity Score Matching

PharmaSUG Paper HA-04 Two Roads Diverged in a Narrow Dataset...When Coarsened Exact Matching is More Appropriate than Propensity Score Matching PharmaSUG 207 - Paper HA-04 Two Roads Diverged in a Narrow Dataset...When Coarsened Exact Matching is More Appropriate than Propensity Score Matching Aran Canes, Cigna Corporation ABSTRACT Coarsened Exact

More information

Lecture 9 Internal Validity

Lecture 9 Internal Validity Lecture 9 Internal Validity Objectives Internal Validity Threats to Internal Validity Causality Bayesian Networks Internal validity The extent to which the hypothesized relationship between 2 or more variables

More information

Judea Pearl. Cognitive Systems Laboratory. Computer Science Department. University of California, Los Angeles, CA

Judea Pearl. Cognitive Systems Laboratory. Computer Science Department. University of California, Los Angeles, CA THREE STATISTICAL PUZZLES Judea Pearl Cognitive Systems Laboratory Computer Science Department University of California, Los Angeles, CA 90024 judea@cs.ucla.edu Preface The puzzles in this note are meant

More information

Overview of Perspectives on Causal Inference: Campbell and Rubin. Stephen G. West Arizona State University Freie Universität Berlin, Germany

Overview of Perspectives on Causal Inference: Campbell and Rubin. Stephen G. West Arizona State University Freie Universität Berlin, Germany Overview of Perspectives on Causal Inference: Campbell and Rubin Stephen G. West Arizona State University Freie Universität Berlin, Germany 1 Randomized Experiment (RE) Sir Ronald Fisher E(X Treatment

More information

Causal Inference Course Syllabus

Causal Inference Course Syllabus ICPSR Summer Program Causal Inference Course Syllabus Instructors: Dominik Hangartner Marco R. Steenbergen Summer 2011 Contact Information: Marco Steenbergen, Department of Social Sciences, University

More information

Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference

Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference Bradford Hill Criteria for Causal Inference Based on a presentation at the 2015 ANZEA Conference Julian King Director, Julian King & Associates a member of the Kinnect Group www.julianking.co.nz www.kinnect.co.nz

More information

Jake Bowers Wednesdays, 2-4pm 6648 Haven Hall ( ) CPS Phone is

Jake Bowers Wednesdays, 2-4pm 6648 Haven Hall ( ) CPS Phone is Political Science 688 Applied Bayesian and Robust Statistical Methods in Political Research Winter 2005 http://www.umich.edu/ jwbowers/ps688.html Class in 7603 Haven Hall 10-12 Friday Instructor: Office

More information

Questions and Answers

Questions and Answers In R. Dechter, H. Geffner, and J.Y. Halpern (Eds.), Heuristics, Probability and Causality: A Tribute to Judea Pearl, College Publication, 2010. 31 Questions and Answers Nils J. Nilsson Few people have

More information

Wason's Cards: What is Wrong?

Wason's Cards: What is Wrong? Wason's Cards: What is Wrong? Pei Wang Computer and Information Sciences, Temple University This paper proposes a new interpretation

More information

Impact and adjustment of selection bias. in the assessment of measurement equivalence

Impact and adjustment of selection bias. in the assessment of measurement equivalence Impact and adjustment of selection bias in the assessment of measurement equivalence Thomas Klausch, Joop Hox,& Barry Schouten Working Paper, Utrecht, December 2012 Corresponding author: Thomas Klausch,

More information

Summary. 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department

Summary. 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department 20 May 2014 EMA/CHMP/SAWP/298348/2014 Procedure No.: EMEA/H/SAB/037/1/Q/2013/SME Product Development Scientific Support Department evaluating patients with Autosomal Dominant Polycystic Kidney Disease

More information

PARTIAL IDENTIFICATION OF PROBABILITY DISTRIBUTIONS. Charles F. Manski. Springer-Verlag, 2003

PARTIAL IDENTIFICATION OF PROBABILITY DISTRIBUTIONS. Charles F. Manski. Springer-Verlag, 2003 PARTIAL IDENTIFICATION OF PROBABILITY DISTRIBUTIONS Charles F. Manski Springer-Verlag, 2003 Contents Preface vii Introduction: Partial Identification and Credible Inference 1 1 Missing Outcomes 6 1.1.

More information

Lab 2: The Scientific Method. Summary

Lab 2: The Scientific Method. Summary Lab 2: The Scientific Method Summary Today we will venture outside to the University pond to develop your ability to apply the scientific method to the study of animal behavior. It s not the African savannah,

More information

STATISTICAL INFERENCE 1 Richard A. Johnson Professor Emeritus Department of Statistics University of Wisconsin

STATISTICAL INFERENCE 1 Richard A. Johnson Professor Emeritus Department of Statistics University of Wisconsin STATISTICAL INFERENCE 1 Richard A. Johnson Professor Emeritus Department of Statistics University of Wisconsin Key words : Bayesian approach, classical approach, confidence interval, estimation, randomization,

More information

Practical propensity score matching: a reply to Smith and Todd

Practical propensity score matching: a reply to Smith and Todd Journal of Econometrics 125 (2005) 355 364 www.elsevier.com/locate/econbase Practical propensity score matching: a reply to Smith and Todd Rajeev Dehejia a,b, * a Department of Economics and SIPA, Columbia

More information

Propensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy Research

Propensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy Research 2012 CCPRC Meeting Methodology Presession Workshop October 23, 2012, 2:00-5:00 p.m. Propensity Score Methods for Estimating Causality in the Absence of Random Assignment: Applications for Child Care Policy

More information

(CORRELATIONAL DESIGN AND COMPARATIVE DESIGN)

(CORRELATIONAL DESIGN AND COMPARATIVE DESIGN) UNIT 4 OTHER DESIGNS (CORRELATIONAL DESIGN AND COMPARATIVE DESIGN) Quasi Experimental Design Structure 4.0 Introduction 4.1 Objectives 4.2 Definition of Correlational Research Design 4.3 Types of Correlational

More information

Evaluation of Medication-Mediated Effects in Pharmacoepidemiology. By EJ Tchetgen Tchetgen and K Phiri

Evaluation of Medication-Mediated Effects in Pharmacoepidemiology. By EJ Tchetgen Tchetgen and K Phiri Evaluation of Medication-Mediated Effects in Pharmacoepidemiology By EJ Tchetgen Tchetgen and K Phiri eappendix 1 Additional discussion of Identification of NDE(a,a*) and NIE(a,a*) It is well known that

More information

Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps

Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps Chapter 3 Tools for Practical Theorizing: Theoretical Maps and Ecosystem Maps Chapter Outline I. Introduction A. Understanding theoretical languages requires universal translators 1. Theoretical maps identify

More information

Research Approach & Design. Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section

Research Approach & Design. Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section Research Approach & Design Awatif Alam MBBS, Msc (Toronto),ABCM Professor Community Medicine Vice Provost Girls Section Content: Introduction Definition of research design Process of designing & conducting

More information

Causal Inference in Statistics and the Quantitative Sciences

Causal Inference in Statistics and the Quantitative Sciences Causal Inference in Statistics and the Quantitative Sciences Erica E. M. Moodie (McGill University) and David A. Stephens (McGill University) May 3 8, 2009 1 A Short Overview of the Field Causal inference

More information

Kepler tried to record the paths of planets in the sky, Harvey to measure the flow of blood in the circulatory system, and chemists tried to produce

Kepler tried to record the paths of planets in the sky, Harvey to measure the flow of blood in the circulatory system, and chemists tried to produce Stats 95 Kepler tried to record the paths of planets in the sky, Harvey to measure the flow of blood in the circulatory system, and chemists tried to produce pure gold knowing it was an element, though

More information

Quantitative Methods. Lonnie Berger. Research Training Policy Practice

Quantitative Methods. Lonnie Berger. Research Training Policy Practice Quantitative Methods Lonnie Berger Research Training Policy Practice Defining Quantitative and Qualitative Research Quantitative methods: systematic empirical investigation of observable phenomena via

More information

Underlying Theory & Basic Issues

Underlying Theory & Basic Issues Underlying Theory & Basic Issues Dewayne E Perry ENS 623 Perry@ece.utexas.edu 1 All Too True 2 Validity In software engineering, we worry about various issues: E-Type systems: Usefulness is it doing what

More information

Pros. University of Chicago and NORC at the University of Chicago, USA, and IZA, Germany

Pros. University of Chicago and NORC at the University of Chicago, USA, and IZA, Germany Dan A. Black University of Chicago and NORC at the University of Chicago, USA, and IZA, Germany Matching as a regression estimator Matching avoids making assumptions about the functional form of the regression

More information

INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1

INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1 INTERVIEWS II: THEORIES AND TECHNIQUES 5. CLINICAL APPROACH TO INTERVIEWING PART 1 5.1 Clinical Interviews: Background Information The clinical interview is a technique pioneered by Jean Piaget, in 1975,

More information

Epidemiologic Causation: Jerome Cornfield s Argument for a Causal Connection between Smoking and Lung Cancer

Epidemiologic Causation: Jerome Cornfield s Argument for a Causal Connection between Smoking and Lung Cancer EpidemiologicCausation:JeromeCornfield sargument foracausalconnectionbetweensmokingandlung Cancer RogerStanev ** rstanev@interchange.ubc.ca ABSTRACT Acentralissueconfrontingbothphilosophersandpractitionersinformulatingananalysisofcausationis

More information

METHODOLOGY FOR DISSERTATION

METHODOLOGY FOR DISSERTATION METHODOLOGY FOR DISSERTATION In order to expose the methods of scientific work, it is necessary to briefly clarify the terms of methodology, methods and scientific methods. The methodology comes from the

More information

II. The Behavioral Approach to Understanding Cognition

II. The Behavioral Approach to Understanding Cognition II. The Behavioral Approach to Understanding Cognition The 3-term contingency is the interpretive workhorse of behavior analysis. But there are Formidable objections to adequacy of 3-term contingency to

More information

Comments on Limits of Econometrics by David Freedman. Arnold Zellner. University of Chicago

Comments on Limits of Econometrics by David Freedman. Arnold Zellner. University of Chicago Comments on Limits of Econometrics by David Freedman Arnold Zellner University of Chicago David Freedman s impressive paper reveals well his deep understanding of not only statistical techniques and their

More information

Comments on David Rosenthal s Consciousness, Content, and Metacognitive Judgments

Comments on David Rosenthal s Consciousness, Content, and Metacognitive Judgments Consciousness and Cognition 9, 215 219 (2000) doi:10.1006/ccog.2000.0438, available online at http://www.idealibrary.com on Comments on David Rosenthal s Consciousness, Content, and Metacognitive Judgments

More information

Research Design. Beyond Randomized Control Trials. Jody Worley, Ph.D. College of Arts & Sciences Human Relations

Research Design. Beyond Randomized Control Trials. Jody Worley, Ph.D. College of Arts & Sciences Human Relations Research Design Beyond Randomized Control Trials Jody Worley, Ph.D. College of Arts & Sciences Human Relations Introduction to the series Day 1: Nonrandomized Designs Day 2: Sampling Strategies Day 3:

More information

Cross-Lagged Panel Analysis

Cross-Lagged Panel Analysis Cross-Lagged Panel Analysis Michael W. Kearney Cross-lagged panel analysis is an analytical strategy used to describe reciprocal relationships, or directional influences, between variables over time. Cross-lagged

More information

Should Causality Be Defined in Terms of Experiments? University of Connecticut University of Colorado

Should Causality Be Defined in Terms of Experiments? University of Connecticut University of Colorado November 2004 Should Causality Be Defined in Terms of Experiments? David A. Kenny Charles M. Judd University of Connecticut University of Colorado The research was supported in part by grant from the National

More information

Response to the ASA s statement on p-values: context, process, and purpose

Response to the ASA s statement on p-values: context, process, and purpose Response to the ASA s statement on p-values: context, process, purpose Edward L. Ionides Alexer Giessing Yaacov Ritov Scott E. Page Departments of Complex Systems, Political Science Economics, University

More information

CSC2130: Empirical Research Methods for Software Engineering

CSC2130: Empirical Research Methods for Software Engineering CSC2130: Empirical Research Methods for Software Engineering Steve Easterbrook sme@cs.toronto.edu www.cs.toronto.edu/~sme/csc2130/ 2004-5 Steve Easterbrook. This presentation is available free for non-commercial

More information

Durkheim. Durkheim s fundamental task in Rules of the Sociological Method is to lay out

Durkheim. Durkheim s fundamental task in Rules of the Sociological Method is to lay out Michelle Lynn Tey Meadow Jane Jones Deirdre O Sullivan Durkheim Durkheim s fundamental task in Rules of the Sociological Method is to lay out the basic disciplinary structure of sociology. He begins by

More information

Logistic regression: Why we often can do what we think we can do 1.

Logistic regression: Why we often can do what we think we can do 1. Logistic regression: Why we often can do what we think we can do 1. Augst 8 th 2015 Maarten L. Buis, University of Konstanz, Department of History and Sociology maarten.buis@uni.konstanz.de All propositions

More information

Evolutionary Approach to Investigations of Cognitive Systems

Evolutionary Approach to Investigations of Cognitive Systems Evolutionary Approach to Investigations of Cognitive Systems Vladimir RED KO a,1 b and Anton KOVAL a Scientific Research Institute for System Analysis, Russian Academy of Science, Russia b National Nuclear

More information

Modelling Causation in Evidence-Based Legal Reasoning ICAIL Doctoral Consortium

Modelling Causation in Evidence-Based Legal Reasoning ICAIL Doctoral Consortium Modelling Causation in Evidence-Based Legal Reasoning ICAIL Doctoral Consortium Ruta Liepina Law Department, European University Institute 1 Research Questions Research questions in this project revolve

More information

Understanding the Causal Logic of Confounds

Understanding the Causal Logic of Confounds Understanding the Causal Logic of Confounds Björn Meder (bmeder@uni-goettingen.de) York Hagmayer (york.hagmayer@bio.uni-goettingen.de) Michael R. Waldmann (michael.waldmann@bio.uni-goettingen.de) Department

More information

Prediction, Causation, and Interpretation in Social Science. Duncan Watts Microsoft Research

Prediction, Causation, and Interpretation in Social Science. Duncan Watts Microsoft Research Prediction, Causation, and Interpretation in Social Science Duncan Watts Microsoft Research Explanation in Social Science: Causation or Interpretation? When social scientists talk about explanation they

More information

Design of Experiments & Introduction to Research

Design of Experiments & Introduction to Research Design of Experiments & Introduction to Research 1 Design of Experiments Introduction to Research Definition and Purpose Scientific Method Research Project Paradigm Structure of a Research Project Types

More information

Effects of propensity score overlap on the estimates of treatment effects. Yating Zheng & Laura Stapleton

Effects of propensity score overlap on the estimates of treatment effects. Yating Zheng & Laura Stapleton Effects of propensity score overlap on the estimates of treatment effects Yating Zheng & Laura Stapleton Introduction Recent years have seen remarkable development in estimating average treatment effects

More information

VALIDITY OF QUANTITATIVE RESEARCH

VALIDITY OF QUANTITATIVE RESEARCH Validity 1 VALIDITY OF QUANTITATIVE RESEARCH Recall the basic aim of science is to explain natural phenomena. Such explanations are called theories (Kerlinger, 1986, p. 8). Theories have varying degrees

More information

Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis

Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis EFSA/EBTC Colloquium, 25 October 2017 Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis Julian Higgins University of Bristol 1 Introduction to concepts Standard

More information

Chapter 21 Multilevel Propensity Score Methods for Estimating Causal Effects: A Latent Class Modeling Strategy

Chapter 21 Multilevel Propensity Score Methods for Estimating Causal Effects: A Latent Class Modeling Strategy Chapter 21 Multilevel Propensity Score Methods for Estimating Causal Effects: A Latent Class Modeling Strategy Jee-Seon Kim and Peter M. Steiner Abstract Despite their appeal, randomized experiments cannot

More information

Principles of Sociology

Principles of Sociology Principles of Sociology DEPARTMENT OF ECONOMICS ATHENS UNIVERSITY OF ECONOMICS AND BUSINESS [Academic year 2017/18, FALL SEMESTER] Lecturer: Dimitris Lallas Principles of Sociology 4th Session Sociological

More information

Temporalization In Causal Modeling. Jonathan Livengood & Karen Zwier TaCitS 06/07/2017

Temporalization In Causal Modeling. Jonathan Livengood & Karen Zwier TaCitS 06/07/2017 Temporalization In Causal Modeling Jonathan Livengood & Karen Zwier TaCitS 06/07/2017 Temporalization In Causal Modeling Jonathan Livengood & Karen Zwier TaCitS 06/07/2017 Introduction Causal influence,

More information

Propensity Score Analysis and Strategies for Its Application to Services Training Evaluation

Propensity Score Analysis and Strategies for Its Application to Services Training Evaluation Propensity Score Analysis and Strategies for Its Application to Services Training Evaluation Shenyang Guo, Ph.D. School of Social Work University of North Carolina at Chapel Hill June 14, 2011 For the

More information

Funnelling Used to describe a process of narrowing down of focus within a literature review. So, the writer begins with a broad discussion providing b

Funnelling Used to describe a process of narrowing down of focus within a literature review. So, the writer begins with a broad discussion providing b Accidental sampling A lesser-used term for convenience sampling. Action research An approach that challenges the traditional conception of the researcher as separate from the real world. It is associated

More information

Assignment 4: True or Quasi-Experiment

Assignment 4: True or Quasi-Experiment Assignment 4: True or Quasi-Experiment Objectives: After completing this assignment, you will be able to Evaluate when you must use an experiment to answer a research question Develop statistical hypotheses

More information

Identifying Mechanisms behind Policy Interventions via Causal Mediation Analysis

Identifying Mechanisms behind Policy Interventions via Causal Mediation Analysis Identifying Mechanisms behind Policy Interventions via Causal Mediation Analysis December 20, 2013 Abstract Causal analysis in program evaluation has largely focused on the assessment of policy effectiveness.

More information

School of Nursing, University of British Columbia Vancouver, British Columbia, Canada

School of Nursing, University of British Columbia Vancouver, British Columbia, Canada Data analysis in qualitative research School of Nursing, University of British Columbia Vancouver, British Columbia, Canada Unquestionably, data analysis is the most complex and mysterious of all of the

More information

Principal Stratification A goal or a tool?

Principal Stratification A goal or a tool? The International Journal of Biostatistics Vol. 7: Issue 1, Article 20, Pages 1 13, ISSN (Online) 1557-4679, DOI: 10.2202/1557-4679.1322, March 2011. TECHNICAL REPORT R-382 March 2011 Principal Stratification

More information

The use of quasi-experiments in the social sciences: a content analysis

The use of quasi-experiments in the social sciences: a content analysis Qual Quant (2011) 45:21 42 DOI 10.1007/s11135-009-9281-4 The use of quasi-experiments in the social sciences: a content analysis Marie-Claire E. Aussems Anne Boomsma Tom A. B. Snijders Published online:

More information

EXPERIMENTAL RESEARCH DESIGNS

EXPERIMENTAL RESEARCH DESIGNS ARTHUR PSYC 204 (EXPERIMENTAL PSYCHOLOGY) 14A LECTURE NOTES [02/28/14] EXPERIMENTAL RESEARCH DESIGNS PAGE 1 Topic #5 EXPERIMENTAL RESEARCH DESIGNS As a strict technical definition, an experiment is a study

More information

Identifying Peer Influence Effects in Observational Social Network Data: An Evaluation of Propensity Score Methods

Identifying Peer Influence Effects in Observational Social Network Data: An Evaluation of Propensity Score Methods Identifying Peer Influence Effects in Observational Social Network Data: An Evaluation of Propensity Score Methods Dean Eckles Department of Communication Stanford University dean@deaneckles.com Abstract

More information

Evaluating health management programmes over time: application of propensity score-based weighting to longitudinal datajep_

Evaluating health management programmes over time: application of propensity score-based weighting to longitudinal datajep_ Journal of Evaluation in Clinical Practice ISSN 1356-1294 Evaluating health management programmes over time: application of propensity score-based weighting to longitudinal datajep_1361 180..185 Ariel

More information

Competency Rubric Bank for the Sciences (CRBS)

Competency Rubric Bank for the Sciences (CRBS) Competency Rubric Bank for the Sciences (CRBS) Content Knowledge 1 Content Knowledge: Accuracy of scientific understanding Higher Order Cognitive Skills (HOCS) 3 Analysis: Clarity of Research Question

More information

The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016

The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 This course does not cover how to perform statistical tests on SPSS or any other computer program. There are several courses

More information

Bill Wilson. Categorizing Cognition: Toward Conceptual Coherence in the Foundations of Psychology

Bill Wilson. Categorizing Cognition: Toward Conceptual Coherence in the Foundations of Psychology Categorizing Cognition: Toward Conceptual Coherence in the Foundations of Psychology Halford, G.S., Wilson, W.H., Andrews, G., & Phillips, S. (2014). Cambridge, MA: MIT Press http://mitpress.mit.edu/books/categorizing-cognition

More information

P(HYP)=h. P(CON HYP)=p REL. P(CON HYP)=q REP

P(HYP)=h. P(CON HYP)=p REL. P(CON HYP)=q REP BAYESIAN NETWORKS IN PHILOSOPHY STEPHAN HARTMANN, LUC BOVENS 1. Introduction There is a long philosophical tradition of addressing questions in philosophy of science and epistemology by means of the tools

More information

Simpson s Paradox and the implications for medical trials

Simpson s Paradox and the implications for medical trials Simpson s Paradox and the implications for medical trials Norman Fenton, Martin Neil and Anthony Constantinou 31 August 2015 Abstract This paper describes Simpson s paradox, and explains its serious implications

More information

Doing After Seeing. Seeing vs. Doing in Causal Bayes Nets

Doing After Seeing. Seeing vs. Doing in Causal Bayes Nets Doing After Seeing Björn Meder (bmeder@uni-goettingen.de) York Hagmayer (york.hagmayer@bio.uni-goettingen.de) Michael R. Waldmann (michael.waldmann@bio.uni-goettingen.de) Department of Psychology, University

More information

Measuring Impact. Conceptual Issues in Program and Policy Evaluation. Daniel L. Millimet. Southern Methodist University.

Measuring Impact. Conceptual Issues in Program and Policy Evaluation. Daniel L. Millimet. Southern Methodist University. Measuring mpact Conceptual ssues in Program and Policy Evaluation Daniel L. Millimet Southern Methodist University 23 May 2013 DL Millimet (SMU) Measuring mpact May 2013 1 / 25 ntroduction Primary concern

More information

Representation Theorems for Multiple Belief Changes

Representation Theorems for Multiple Belief Changes Representation Theorems for Multiple Belief Changes Dongmo Zhang 1,2 Shifu Chen 1 Wujia Zhu 1,2 Zhaoqian Chen 1 1 State Key Lab. for Novel Software Technology Department of Computer Science and Technology

More information

Recent advances in non-experimental comparison group designs

Recent advances in non-experimental comparison group designs Recent advances in non-experimental comparison group designs Elizabeth Stuart Johns Hopkins Bloomberg School of Public Health Department of Mental Health Department of Biostatistics Department of Health

More information

A Brief Introduction to Bayesian Statistics

A Brief Introduction to Bayesian Statistics A Brief Introduction to Statistics David Kaplan Department of Educational Psychology Methods for Social Policy Research and, Washington, DC 2017 1 / 37 The Reverend Thomas Bayes, 1701 1761 2 / 37 Pierre-Simon

More information

Title:Continuity of GP care is associated with lower use of complementary and alternative medical providers A population-based cross-sectional survey

Title:Continuity of GP care is associated with lower use of complementary and alternative medical providers A population-based cross-sectional survey Author's response to reviews Title:Continuity of GP care is associated with lower use of complementary and alternative medical providers A population-based cross-sectional survey Authors: Anne Helen Hansen

More information

Samples, Sample Size And Sample Error. Research Methodology. How Big Is Big? Estimating Sample Size. Variables. Variables 2/25/2018

Samples, Sample Size And Sample Error. Research Methodology. How Big Is Big? Estimating Sample Size. Variables. Variables 2/25/2018 Research Methodology Samples, Sample Size And Sample Error Sampling error = difference between sample and population characteristics Reducing sampling error is the goal of any sampling technique As sample

More information

Methodological Thinking on Disciplinary Research Fields of Theories of Sports Training from the Philosophical Perspective Bin LONG

Methodological Thinking on Disciplinary Research Fields of Theories of Sports Training from the Philosophical Perspective Bin LONG 2017 4th International Conference on Advanced Education and Management (ICAEM 2017) ISBN: 978-1-60595-519-3 Methodological Thinking on Disciplinary Research Fields of Theories of Sports Training from the

More information

BIOSTATISTICAL METHODS AND RESEARCH DESIGNS. Xihong Lin Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA

BIOSTATISTICAL METHODS AND RESEARCH DESIGNS. Xihong Lin Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA BIOSTATISTICAL METHODS AND RESEARCH DESIGNS Xihong Lin Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA Keywords: Case-control study, Cohort study, Cross-Sectional Study, Generalized

More information

Causality and Statistical Learning 1. Andrew Gelman Feb 2010

Causality and Statistical Learning 1. Andrew Gelman Feb 2010 Causality and Statistical Learning 1 Andrew Gelman 2 24 Feb 2010 In social science we are sometimes in the position of studying descriptive questions (for example: In what places do working-class whites

More information

Do not copy, post, or distribute

Do not copy, post, or distribute C H A P T E R 2 Counterfactual Framework and Assumptions T his chapter examines conceptual frameworks that guide the estimation of treatment effects as well as important assumptions that are embedded in

More information

Propensity Score Methods for Causal Inference with the PSMATCH Procedure

Propensity Score Methods for Causal Inference with the PSMATCH Procedure Paper SAS332-2017 Propensity Score Methods for Causal Inference with the PSMATCH Procedure Yang Yuan, Yiu-Fai Yung, and Maura Stokes, SAS Institute Inc. Abstract In a randomized study, subjects are randomly

More information

Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior

Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior 1 Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior Gregory Francis Department of Psychological Sciences Purdue University gfrancis@purdue.edu

More information

Approaches to Improving Causal Inference from Mediation Analysis

Approaches to Improving Causal Inference from Mediation Analysis Approaches to Improving Causal Inference from Mediation Analysis David P. MacKinnon, Arizona State University Pennsylvania State University February 27, 2013 Background Traditional Mediation Methods Modern

More information

Modelling Double-Moderated-Mediation & Confounder Effects Using Bayesian Statistics

Modelling Double-Moderated-Mediation & Confounder Effects Using Bayesian Statistics Modelling Double-Moderated-Mediation & Confounder Effects Using Bayesian Statistics George Chryssochoidis*, Lars Tummers**, Rens van de Schoot***, * Norwich Business School, University of East Anglia **

More information

Support system for breast cancer treatment

Support system for breast cancer treatment Support system for breast cancer treatment SNEZANA ADZEMOVIC Civil Hospital of Cacak, Cara Lazara bb, 32000 Cacak, SERBIA Abstract:-The aim of this paper is to seek out optimal relation between diagnostic

More information

Quantitative Research. By Dr. Achmad Nizar Hidayanto Information Management Lab Faculty of Computer Science Universitas Indonesia

Quantitative Research. By Dr. Achmad Nizar Hidayanto Information Management Lab Faculty of Computer Science Universitas Indonesia Quantitative Research By Dr. Achmad Nizar Hidayanto Information Management Lab Faculty of Computer Science Universitas Indonesia Depok, 2 Agustus 2017 Quantitative Research: Definition (Source: Wikipedia)

More information

[1] provides a philosophical introduction to the subject. Simon [21] discusses numerous topics in economics; see [2] for a broad economic survey.

[1] provides a philosophical introduction to the subject. Simon [21] discusses numerous topics in economics; see [2] for a broad economic survey. Draft of an article to appear in The MIT Encyclopedia of the Cognitive Sciences (Rob Wilson and Frank Kiel, editors), Cambridge, Massachusetts: MIT Press, 1997. Copyright c 1997 Jon Doyle. All rights reserved

More information

Meta-Analysis. Zifei Liu. Biological and Agricultural Engineering

Meta-Analysis. Zifei Liu. Biological and Agricultural Engineering Meta-Analysis Zifei Liu What is a meta-analysis; why perform a metaanalysis? How a meta-analysis work some basic concepts and principles Steps of Meta-analysis Cautions on meta-analysis 2 What is Meta-analysis

More information

How to interpret results of metaanalysis

How to interpret results of metaanalysis How to interpret results of metaanalysis Tony Hak, Henk van Rhee, & Robert Suurmond Version 1.0, March 2016 Version 1.3, Updated June 2018 Meta-analysis is a systematic method for synthesizing quantitative

More information