SAMPLING AND SAMPLE SIZE

Size: px
Start display at page:

Download "SAMPLING AND SAMPLE SIZE"

Transcription

1 SAMPLING AND SAMPLE SIZE Andrew Zeitlin Georgetown University and IGC Rwanda With slides from Ben Olken and the World Bank s Development Impact Evaluation Initiative

2 2 Review We want to learn how a program can affect a group Set up: assume we have performed our lottery and we have identified our treatment and the control groups What we would like to measure is difference: Effect of the Program = Mean in treatment - Mean in control Example: average farmers' income who adopted fertilizer because of new incentive program vs the farmers in the control group who didn t receive any incentives

3 3 Bias vs. Noise What if we let farmers choose whether or not to use fertilizer? Our study would be biased! If only the most wealthy, educated farmers choose to use fertilizer, then we will not be able to see the effect of fertilizer between treatment and control. We would see the effect of being wealthy, educated and using fertilizer. This is why we randomize! To remove other factors that might create bias.

4 4 Bias vs. Noise What if we only pick ten farmers in treatment and control, and we randomly get the four richest farmers in our control group, and only poor farmers in the control group? The fact that the randomization did not balance farmer wealth means our groups are not really similar? The control group may make greater gains (due to more resources) despite the fertilizer. Bottom line: randomization removes bias, but it does not remove random noise in the data. This is why we worry about sampling!

5 What do we mean by noise? 5

6 What do we mean by noise? 6

7 Why random is necessary but not enough Random does not mean balanced! It just means it is not unbalanced for any reason. Which of the following coin flips was random? T, H, T, T, H, H, T, H, H, T T, T, H, H, H, H, H, T, H, H This one we made up This was a random coin flip 5 heads in a row!

8 A random sample is accurate, but may not be precise What is the average age of the people in this room? If I pick the youngest looking person in the room and ask their age, I am biasing the type of response I am likely to get. If I pick someone at random and ask their age, that is not biased, but still doesn t tell me much since it is just one person. If I everyone except for one person at random I am likely to get close to the right average age: this is a good random sample. If I ask everyone, it is no longer a sample of the room it is the universe

9 Which of these is more accurate? I. II. 88% A. I. B. II. C. Don t know 12% 0% A. B. C.

10 Accuracy versus Precision Precision (Sample Size) es(mates truth Accuracy (Randomization)

11 11 Real World Constraints Random sampling can be noisy! In a world with no budget constraint we could collect data on ALL the individuals (universe) in the treatment and in the control groups. In practice, we do not observe the entire population, just a sample. Example: we do not have data for all farmers of the country/ region, but just for a random sample of them in treatment and control groups Bottom line: Estimated Effect = True Effect + Noise

12 THE basic questions in statistics How confident can you be in your results? à How big does your sample need to be?

13 13 Hypothesis Testing In criminal law, most institutions follow the rule: innocent until proven guilty The presumption is that the accused is innocent and the burden is on the prosecutor to show guilt The jury or judge starts with the null hypothesis that the accused person is innocent The prosecutor has a hypothesis that the accused person is guilty

14 Hypothesis Testing In program evaluation, instead of presumption of innocence, the rule is: presumption of insignificance The Null hypothesis (H 0 ) is that there was no (zero) impact of the program The burden of proof is on the evaluator to show a significant effect of the program

15 Hypothesis Testing: Conclusions If it is very unlikely (less than a 5% probability) that the difference is solely due to chance: We reject our null hypothesis We may now say: our program has a statistically significant impact

16 16 Two Types of Mistakes (1) First type of error: conclude that the program has an effect, when in fact at best it has no effect Significance level of a test: Probability that you will falsely conclude that the program has an effect, when in fact it does not If you find an effect with a level of 5%, you can be 95% confident in the validity of your conclusion that the program had an effect Common levels are: 5%, 10%, 1%

17 17 Two Types of Mistakes (2) Second Type of Error: You conclude that the program has no effect when indeed it had an effect, but it was not measured with enough precision (or noise got in the way) Power of a test: Probability to find a significant effect if there truly is an effect Higher power is better since I am more likely to have an effect to report

18 Practical steps There are two, related ways one might apply this logic: 1. Start from the sample size that you can afford. Figure out what would be the smallest true effect that you could detect with reasonable confidence and power. Ø This is known as the minimum detectable effect for a given design. 2. Start from a plausible effect size, and figure out how big a sample you need in order to be able to detect this with reasonable confidence and power. Ø We will focus on this second approach.

19 19 Practical Steps Ø Set a pre-specified confidence level (5%) i.e. just set the initial point of the line in the graph Ø Decide a level of power. Common values used are 80% or 90%. Intuitively, the larger the sample, the larger the power. Power is a planning tool: one minus the power is the probability to be disappointed. Ø Set a range of pre-specified effect sizes (what you think the program will do) Ø What is the smallest effect that should prompt a policy response?

20 Picking an Effect Size to choose sample We can guess an effect size using economics past data on the outcome of interest or even past evaluations What is the smallest effect that should justify the program to be adopted? Cost of this program v the benefits it brings Cost of this program v the alternative use of the money

21 Underpowered Common danger: picking effect sizes that are too optimistic the sample size may be set too low to detect an actual effect! Example: Evaluators believe a program will increase high school graduation by 15 percentage points. They survey enough schools to see increases of 12 percentage points or more. The program increased graduation rates by 10 percentage points, but they missed that entirely due to lack of power! They report the program had no statistically significant effect, even though it actually had one!

22 22 How difficult is this to do? Proposition I: There exists at least one statistician in the world who has already put into a magic formula the optimal sample size required to address this problem Proposition II: The rule has also been implemented for almost all computer software Not difficult to do, and only requires simple calculations to understand the logic (really simple!)

23 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

24 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

25 Larger effect= More Power to Detect A device detects all animals over six feet (1.8 meters) tall. Power to detect adult men: Under 10% Power to detect adult women: Under 1% Power to detect adult mice: 0% Power to detect adult giraffes: 100% The taller the animal (effect size) we care about, the more power we have (and the less we need)

26 Effect Size: 1*SE Hypothesized effect size determines distance between 0.5 means Standard Deviation 0.35 H H β control 0.2 treatment

27 Effect Size = 1*SE H H β control treatment 0.2 significance

28 Power: 26% If the true impact was 1*SE 0.5 H H β control treatment power The Null Hypothesis would be rejected only 26% of the time

29 Effect Size: 3*SE *SE control treatment Bigger hypothesized effect sizeà distributions farther apart

30 Effect size 3*SE: Power= 91% H H β control treatment power Bigger Effect size means more power

31 What effect size should you use when designing your experiment? A. Smallest effect size that is still cost effective B. Largest effect size you expect your program to produce C. Both D. Neither 50% 50% 0% 0% A. B. C. D.

32 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

33 By increasing sample size you increase control treatment power 50% 33% A. Accuracy B. Precision C. Both D. Neither E. Don t know 17% 0% 0% A. B. C. D. E.

34 Larger sample size= More power to detect We want to know the average age in the city If we randomly pick one person in the city, we might pick a 100 year old. If we randomly pick 2000 people, even if we pick the 100 year old as one of them, he will be balanced out by the other random selections. This intuition extends to effect sizes.

35 Power: Effect size = 1SD, Sample size = N control treatment significance

36 Power: Sample size = 4N control treatment significance

37 Power: 64% control treatment power

38 Power: Sample size = 9N control treatment significance

39 Power: 91% control treatment power

40 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

41 More variance= Less power to detect Imagine the following intervention: Giving away ten bags of rice In this example, this program has a large effect on ALL poor people, and no effect on ALL rich people. Low Variance: If our population is all poor, we only need to sample one person to see the true effect of giving away rice High Variance: If our population is half poor, and half rich (high variance) and we randomly sample twenty people, what happens if only 5 are poor?

42 What are typical ways to reduce the underlying (population) variance A. Include covariates B. Increase the sample C. Do a baseline survey D. All of the above E. A and B F. A and C 80% 20% 0% 0% 0% 0% A. B. C. D. E. F.

43 Variance There is sometimes very little we can do to reduce the noise The underlying variance is what it is We can try to absorb variance: using a baseline controlling for other variables In practice, controlling for other variables (besides the baseline outcome) buys you very little

44 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

45 More balanced treatment assignment => More power What s better? 99 people who get the treatment and one control 50 treatment and 50 control This logic continues. What s better? 60 people who get the treatment and 40 control 50 treatment and 50 control

46 Sample split: 50% C, 50% T H H β control treatment 0.2 significance

47 Power: 91% control treatment power

48 If it s not split? What happens to the relative fatness if the split is not Say 25-75?

49 Sample split: 25% C, 75% T H H β control treatment 0.2 significance

50 Power: 83% control treatment power

51 How unbalanced is too unbalanced? Bloom (2006): Because precision erodes slowly until the degree of imbalance becomes extreme (roughly P 0.2 or P 0.8), there is considerable latitude for using an unbalanced allocation. This helps if Politics dictate a small control group Costs dictate a small treatment group

52 Power: main ingredients 1. Effect Size 2. Sample Size 3. Variance 4. Proportion of sample in T vs. C 5. Clustering

53 Clustered design: definition In sampling: When clusters of individuals (e.g. schools, communities, etc) are randomly selected from the population, before selecting individuals for observation In randomized evaluation: When clusters of individuals are randomly assigned to different treatment groups

54 Clustered design: intuition You want to know how close the upcoming national elections will be Method 1: Randomly select 50 people from entire Indian population Method 2: Randomly select 5 families, and ask ten members of each family their opinion

55 Low intra-cluster correlation (ICC) aka ρ (rho)

56 HIGH intra-cluster correlation (ρ)

57 All uneducated people live in one village. People with only primary education live in another. College grads live in a third, etc. ICC (ρ) on education will be.. A. High B. Low C. No effect on rho D. Don t know

58 If ICC (ρ) is high, what is a more efficient way of increasing power? A. Include more clusters in the sample B. Include more people in clusters C. Both D. Don t know

59 Further topics: Imperfect compliance In some cases, policymakers/researchers can assign individuals to a given treatment arm, but this doesn t mean they will take it up. What does this mean for power? Consider an extreme cases in which nobody in the treatment group takes up. In that case, no matter how big the sample size, you can t detect the treatment s impact because you never see it. Alternatively, what happens if everybody ends up getting the treatment in both treatment and control groups? The required sample size is inversely proportional to (c d) 2 where c is the fraction of treated who comply, and d is fraction of control who defy

60 60 Wrap-up on Power Power calculations look scary but they are just a formalization of common sense At times we do not have the right information to conduct it very properly However, it is important to spend effort on them: Avoid launching studies that will have no power at all: waste of time and money, potentially harmful Devote the appropriate resources to the studies that you decide to conduct (and not too much)

61 Appendix: The nuts and bolts (1) For an experimental design with perfect compliance and individual-level assignment (no clustering), Minimum detectable effect, for sample size N MDE=( t 1 κ + t α/2 ) σ 2 /P(1 P)N Minimum sample size, for hypothesized effect size β N= ( t 1 κ + t α/2 ) 2 σ 2 / β 2 P(1 P)

62 Appendix: The nuts and bolts (2) When compliance becomes imperfect, with c the fraction of those assigned to treatment who take up and d the fraction of control who do likewise. Minimum detectable effect, for sample size N MDE= ( t 1 κ + t α/2 )/(c d) σ 2 /P(1 P)N Minimum sample size, for hypothesized effect size β N= ( t 1 κ + t α/2 ) 2 σ 2 / (c d) 2 β 2 P(1 P)

63 Appendix: The nuts and bolts (3) For an experimental design with perfect compliance and group-based assignment, Minimum detectable effect, for J groups with n members MDE=( t 1 κ + t α/2 ) σ 2 /JP(1 P) (ρ+ 1 ρ/n ) Minimum number of groups, for hypothesized effect size β J= ( t 1 κ + t α/2 ) 2 σ 2 / β 2 P(1 P) (ρ+ 1 ρ/n )

Abdul Latif Jameel Poverty Action Lab Executive Training: Evaluating Social Programs Spring 2009

Abdul Latif Jameel Poverty Action Lab Executive Training: Evaluating Social Programs Spring 2009 MIT OpenCourseWare http://ocw.mit.edu Abdul Latif Jameel Poverty Action Lab Executive Training: Evaluating Social Programs Spring 2009 For information about citing these materials or our Terms of Use,

More information

Sampling for Impact Evaluation. Maria Jones 24 June 2015 ieconnect Impact Evaluation Workshop Rio de Janeiro, Brazil June 22-25, 2015

Sampling for Impact Evaluation. Maria Jones 24 June 2015 ieconnect Impact Evaluation Workshop Rio de Janeiro, Brazil June 22-25, 2015 Sampling for Impact Evaluation Maria Jones 24 June 2015 ieconnect Impact Evaluation Workshop Rio de Janeiro, Brazil June 22-25, 2015 How many hours do you expect to sleep tonight? A. 2 or less B. 3 C.

More information

Planning Sample Size for Randomized Evaluations.

Planning Sample Size for Randomized Evaluations. Planning Sample Size for Randomized Evaluations www.povertyactionlab.org Planning Sample Size for Randomized Evaluations General question: How large does the sample need to be to credibly detect a given

More information

Statistical Power Sampling Design and sample Size Determination

Statistical Power Sampling Design and sample Size Determination Statistical Power Sampling Design and sample Size Determination Deo-Gracias HOUNDOLO Impact Evaluation Specialist dhoundolo@3ieimpact.org Outline 1. Sampling basics 2. What do evaluators do? 3. Statistical

More information

Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha

Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha Glossary From Running Randomized Evaluations: A Practical Guide, by Rachel Glennerster and Kudzai Takavarasha attrition: When data are missing because we are unable to measure the outcomes of some of the

More information

EXERCISE: HOW TO DO POWER CALCULATIONS IN OPTIMAL DESIGN SOFTWARE

EXERCISE: HOW TO DO POWER CALCULATIONS IN OPTIMAL DESIGN SOFTWARE ...... EXERCISE: HOW TO DO POWER CALCULATIONS IN OPTIMAL DESIGN SOFTWARE TABLE OF CONTENTS 73TKey Vocabulary37T... 1 73TIntroduction37T... 73TUsing the Optimal Design Software37T... 73TEstimating Sample

More information

ECONOMIC EVALUATION IN DEVELOPMENT

ECONOMIC EVALUATION IN DEVELOPMENT ECONOMIC EVALUATION IN DEVELOPMENT Autumn 2015 Michael King 1 A plan to end poverty.. I have identified the specific investments that are needed [to end poverty]; found ways to plan and implement them;

More information

Sheila Barron Statistics Outreach Center 2/8/2011

Sheila Barron Statistics Outreach Center 2/8/2011 Sheila Barron Statistics Outreach Center 2/8/2011 What is Power? When conducting a research study using a statistical hypothesis test, power is the probability of getting statistical significance when

More information

Power & Sample Size. Dr. Andrea Benedetti

Power & Sample Size. Dr. Andrea Benedetti Power & Sample Size Dr. Andrea Benedetti Plan Review of hypothesis testing Power and sample size Basic concepts Formulae for common study designs Using the software When should you think about power &

More information

Objectives. Quantifying the quality of hypothesis tests. Type I and II errors. Power of a test. Cautions about significance tests

Objectives. Quantifying the quality of hypothesis tests. Type I and II errors. Power of a test. Cautions about significance tests Objectives Quantifying the quality of hypothesis tests Type I and II errors Power of a test Cautions about significance tests Designing Experiments based on power Evaluating a testing procedure The testing

More information

15.301/310, Managerial Psychology Prof. Dan Ariely Recitation 8: T test and ANOVA

15.301/310, Managerial Psychology Prof. Dan Ariely Recitation 8: T test and ANOVA 15.301/310, Managerial Psychology Prof. Dan Ariely Recitation 8: T test and ANOVA Statistics does all kinds of stuff to describe data Talk about baseball, other useful stuff We can calculate the probability.

More information

Where does "analysis" enter the experimental process?

Where does analysis enter the experimental process? Lecture Topic : ntroduction to the Principles of Experimental Design Experiment: An exercise designed to determine the effects of one or more variables (treatments) on one or more characteristics (response

More information

Sampling and Power Calculations East Asia Regional Impact Evaluation Workshop Seoul, South Korea

Sampling and Power Calculations East Asia Regional Impact Evaluation Workshop Seoul, South Korea Sampling and Power Calculations East Asia Regional Impact Evaluation Workshop Seoul, South Korea Jacobus Cilliers, World Bank Overview What is sampling and why is it necessary? Why does sample size matter?

More information

Chapter 7: Descriptive Statistics

Chapter 7: Descriptive Statistics Chapter Overview Chapter 7 provides an introduction to basic strategies for describing groups statistically. Statistical concepts around normal distributions are discussed. The statistical procedures of

More information

APPENDIX N. Summary Statistics: The "Big 5" Statistical Tools for School Counselors

APPENDIX N. Summary Statistics: The Big 5 Statistical Tools for School Counselors APPENDIX N Summary Statistics: The "Big 5" Statistical Tools for School Counselors This appendix describes five basic statistical tools school counselors may use in conducting results based evaluation.

More information

Lesson 11.1: The Alpha Value

Lesson 11.1: The Alpha Value Hypothesis Testing Lesson 11.1: The Alpha Value The alpha value is the degree of risk we are willing to take when making a decision. The alpha value, often abbreviated using the Greek letter α, is sometimes

More information

Applied Statistical Analysis EDUC 6050 Week 4

Applied Statistical Analysis EDUC 6050 Week 4 Applied Statistical Analysis EDUC 6050 Week 4 Finding clarity using data Today 1. Hypothesis Testing with Z Scores (continued) 2. Chapters 6 and 7 in Book 2 Review! = $ & '! = $ & ' * ) 1. Which formula

More information

Chapter 8 Estimating with Confidence

Chapter 8 Estimating with Confidence Chapter 8 Estimating with Confidence Introduction Our goal in many statistical settings is to use a sample statistic to estimate a population parameter. In Chapter 4, we learned if we randomly select the

More information

Planning sample size for impact evaluations

Planning sample size for impact evaluations Planning sample size for impact evaluations David Evans, Banco Mundial Basado en slides de Esther Duflo (J-PAL) y Jed Friedman (Banco Mundial) Size of the sample for impact evaluations Pergunta geral De

More information

Psychology Research Process

Psychology Research Process Psychology Research Process Logical Processes Induction Observation/Association/Using Correlation Trying to assess, through observation of a large group/sample, what is associated with what? Examples:

More information

Sample Size, Power and Sampling Methods

Sample Size, Power and Sampling Methods Sample Size, Power and Sampling Methods Mary Ann McBurnie, PhD Senior Investigator, Kaiser Permanente Center for Health Research Steering Committee Chair, Community Health Applied Research Network (CHARN)

More information

Confidence in Sampling: Why Every Lawyer Needs to Know the Number 384. By John G. McCabe, M.A. and Justin C. Mary

Confidence in Sampling: Why Every Lawyer Needs to Know the Number 384. By John G. McCabe, M.A. and Justin C. Mary Confidence in Sampling: Why Every Lawyer Needs to Know the Number 384 By John G. McCabe, M.A. and Justin C. Mary Both John (john.mccabe.555@gmail.com) and Justin (justin.mary@cgu.edu.) are in Ph.D. programs

More information

Statistical Significance and Power. November 17 Clair

Statistical Significance and Power. November 17 Clair Statistical Significance and Power November 17 Clair Big Picture What are we trying to estimate? Causal effect of some treatment E(Y i T i =1) E(Y i T i =0) In words, we re comparing the average outcome

More information

Randomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school

Randomization as a Tool for Development Economists. Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as a Tool for Development Economists Esther Duflo Sendhil Mullainathan BREAD-BIRS Summer school Randomization as one solution Suppose you could do a Randomized evaluation of the microcredit

More information

Tutorial. Understanding the Task. People don t often read editorials critically, believing the writer may know more about the subject than they do.

Tutorial. Understanding the Task. People don t often read editorials critically, believing the writer may know more about the subject than they do. Tutorial D I S TI N G U I S H I N G F AC T S FR O M E X P E R T O P I N I O N S E D I TO R I A L R E AD I N G M E D I C AL C AR E TH A T S N O T E VE N FI T FO R A H O R S E M AR G A R E T WE N T E Understanding

More information

AP STATISTICS 2008 SCORING GUIDELINES (Form B)

AP STATISTICS 2008 SCORING GUIDELINES (Form B) AP STATISTICS 2008 SCORING GUIDELINES (Form B) Question 4 Intent of Question The primary goals of this question were to assess a student s ability to (1) design an experiment to compare two treatments

More information

Power of a Clinical Study

Power of a Clinical Study Power of a Clinical Study M.Yusuf Celik 1, Editor-in-Chief 1 Prof.Dr. Biruni University, Medical Faculty, Dept of Biostatistics, Topkapi, Istanbul. Abstract The probability of not committing a Type II

More information

Fixed Effect Combining

Fixed Effect Combining Meta-Analysis Workshop (part 2) Michael LaValley December 12 th 2014 Villanova University Fixed Effect Combining Each study i provides an effect size estimate d i of the population value For the inverse

More information

Welcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias.

Welcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias. Welcome to this series focused on sources of bias in epidemiologic studies. In this first module, I will provide a general overview of bias. In the second module, we will focus on selection bias and in

More information

Still important ideas

Still important ideas Readings: OpenStax - Chapters 1 11 + 13 & Appendix D & E (online) Plous - Chapters 2, 3, and 4 Chapter 2: Cognitive Dissonance, Chapter 3: Memory and Hindsight Bias, Chapter 4: Context Dependence Still

More information

Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior

Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior 1 Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior Gregory Francis Department of Psychological Sciences Purdue University gfrancis@purdue.edu

More information

PSYCHOLOGY 300B (A01) One-sample t test. n = d = ρ 1 ρ 0 δ = d (n 1) d

PSYCHOLOGY 300B (A01) One-sample t test. n = d = ρ 1 ρ 0 δ = d (n 1) d PSYCHOLOGY 300B (A01) Assignment 3 January 4, 019 σ M = σ N z = M µ σ M d = M 1 M s p d = µ 1 µ 0 σ M = µ +σ M (z) Independent-samples t test One-sample t test n = δ δ = d n d d = µ 1 µ σ δ = d n n = δ

More information

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo Please note the page numbers listed for the Lind book may vary by a page or two depending on which version of the textbook you have. Readings: Lind 1 11 (with emphasis on chapters 10, 11) Please note chapter

More information

Outline. 1.What is sampling? 2.How large of a sample size do we need? 3.How can we increase statistical power?

Outline. 1.What is sampling? 2.How large of a sample size do we need? 3.How can we increase statistical power? Sample size Outline 1.What is sampling? 2.How large of a sample size do we need? 3.How can we increase statistical power? Sampling Why do we sample? Usually we cannot gather data from entire target population.

More information

t-test for r Copyright 2000 Tom Malloy. All rights reserved

t-test for r Copyright 2000 Tom Malloy. All rights reserved t-test for r Copyright 2000 Tom Malloy. All rights reserved This is the text of the in-class lecture which accompanied the Authorware visual graphics on this topic. You may print this text out and use

More information

Business Statistics Probability

Business Statistics Probability Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment

More information

Population. population. parameter. Census versus Sample. Statistic. sample. statistic. Parameter. Population. Example: Census.

Population. population. parameter. Census versus Sample. Statistic. sample. statistic. Parameter. Population. Example: Census. Population Population the complete collection of ALL individuals (scores, people, measurements, etc.) to be studied the population is usually too big to be studied directly, then statistics is used Parameter

More information

Readings: Textbook readings: OpenStax - Chapters 1 11 Online readings: Appendix D, E & F Plous Chapters 10, 11, 12 and 14

Readings: Textbook readings: OpenStax - Chapters 1 11 Online readings: Appendix D, E & F Plous Chapters 10, 11, 12 and 14 Readings: Textbook readings: OpenStax - Chapters 1 11 Online readings: Appendix D, E & F Plous Chapters 10, 11, 12 and 14 Still important ideas Contrast the measurement of observable actions (and/or characteristics)

More information

The Logic of Causal Order Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised February 15, 2015

The Logic of Causal Order Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised February 15, 2015 The Logic of Causal Order Richard Williams, University of Notre Dame, https://www3.nd.edu/~rwilliam/ Last revised February 15, 2015 [NOTE: Toolbook files will be used when presenting this material] First,

More information

Previous Example. New. Tradition

Previous Example. New. Tradition Experimental Design Previous Example New Tradition Goal? New Tradition =? Challenges Internal validity How to guarantee what you have observed is true? External validity How to guarantee what you have

More information

Belief behavior Smoking is bad for you I smoke

Belief behavior Smoking is bad for you I smoke LP 12C Cognitive Dissonance 1 Cognitive Dissonance Cognitive dissonance: An uncomfortable mental state due to a contradiction between two attitudes or between an attitude and behavior (page 521). Belief

More information

Statistical inference provides methods for drawing conclusions about a population from sample data.

Statistical inference provides methods for drawing conclusions about a population from sample data. Chapter 14 Tests of Significance Statistical inference provides methods for drawing conclusions about a population from sample data. Two of the most common types of statistical inference: 1) Confidence

More information

Chapter 23. Inference About Means. Copyright 2010 Pearson Education, Inc.

Chapter 23. Inference About Means. Copyright 2010 Pearson Education, Inc. Chapter 23 Inference About Means Copyright 2010 Pearson Education, Inc. Getting Started Now that we know how to create confidence intervals and test hypotheses about proportions, it d be nice to be able

More information

5. is the process of moving from the specific to the general. a. Deduction

5. is the process of moving from the specific to the general. a. Deduction Applied Social Psychology Understanding and Addressing Social and Practical Problems 3rd Edition Gruman Test Bank Full Download: https://testbanklive.com/download/applied-social-psychology-understanding-and-addressing-social-and-practical-p

More information

Conduct an Experiment to Investigate a Situation

Conduct an Experiment to Investigate a Situation Level 3 AS91583 4 Credits Internal Conduct an Experiment to Investigate a Situation Written by J Wills MathsNZ jwills@mathsnz.com Achievement Achievement with Merit Achievement with Excellence Conduct

More information

Ch. 1 Collecting and Displaying Data

Ch. 1 Collecting and Displaying Data Ch. 1 Collecting and Displaying Data In the first two sections of this chapter you will learn about sampling techniques and the different levels of measurement for a variable. It is important that you

More information

Statistics for Psychology

Statistics for Psychology Statistics for Psychology SIXTH EDITION CHAPTER 3 Some Key Ingredients for Inferential Statistics Some Key Ingredients for Inferential Statistics Psychologists conduct research to test a theoretical principle

More information

Chapter 19. Confidence Intervals for Proportions. Copyright 2010 Pearson Education, Inc.

Chapter 19. Confidence Intervals for Proportions. Copyright 2010 Pearson Education, Inc. Chapter 19 Confidence Intervals for Proportions Copyright 2010 Pearson Education, Inc. Standard Error Both of the sampling distributions we ve looked at are Normal. For proportions For means SD pˆ pq n

More information

REVIEW FOR THE PREVIOUS LECTURE

REVIEW FOR THE PREVIOUS LECTURE Slide 2-1 Calculator: The same calculator policies as for the ACT hold for STT 315: http://www.actstudent.org/faq/answers/calculator.html. It is highly recommended that you have a TI-84, as this is the

More information

Biostatistics 3. Developed by Pfizer. March 2018

Biostatistics 3. Developed by Pfizer. March 2018 BROUGHT TO YOU BY Biostatistics 3 Developed by Pfizer March 2018 This learning module is intended for UK healthcare professionals only. Job bag: PP-GEP-GBR-0986 Date of preparation March 2018. Agenda I.

More information

Psychology Research Process

Psychology Research Process Psychology Research Process Logical Processes Induction Observation/Association/Using Correlation Trying to assess, through observation of a large group/sample, what is associated with what? Examples:

More information

Chapter 12: Introduction to Analysis of Variance

Chapter 12: Introduction to Analysis of Variance Chapter 12: Introduction to Analysis of Variance of Variance Chapter 12 presents the general logic and basic formulas for the hypothesis testing procedure known as analysis of variance (ANOVA). The purpose

More information

I. Introduction and Data Collection B. Sampling. 1. Bias. In this section Bias Random Sampling Sampling Error

I. Introduction and Data Collection B. Sampling. 1. Bias. In this section Bias Random Sampling Sampling Error I. Introduction and Data Collection B. Sampling In this section Bias Random Sampling Sampling Error 1. Bias Bias a prejudice in one direction (this occurs when the sample is selected in such a way that

More information

Risk Aversion in Games of Chance

Risk Aversion in Games of Chance Risk Aversion in Games of Chance Imagine the following scenario: Someone asks you to play a game and you are given $5,000 to begin. A ball is drawn from a bin containing 39 balls each numbered 1-39 and

More information

INTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ

INTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ INTRODUCTION TO STATISTICS SORANA D. BOLBOACĂ OBJECTIVES Definitions Stages of Scientific Knowledge Quantification and Accuracy Types of Medical Data Population and sample Sampling methods DEFINITIONS

More information

Chapter 12. The One- Sample

Chapter 12. The One- Sample Chapter 12 The One- Sample z-test Objective We are going to learn to make decisions about a population parameter based on sample information. Lesson 12.1. Testing a Two- Tailed Hypothesis Example 1: Let's

More information

OCW Epidemiology and Biostatistics, 2010 David Tybor, MS, MPH and Kenneth Chui, PhD Tufts University School of Medicine October 27, 2010

OCW Epidemiology and Biostatistics, 2010 David Tybor, MS, MPH and Kenneth Chui, PhD Tufts University School of Medicine October 27, 2010 OCW Epidemiology and Biostatistics, 2010 David Tybor, MS, MPH and Kenneth Chui, PhD Tufts University School of Medicine October 27, 2010 SAMPLING AND CONFIDENCE INTERVALS Learning objectives for this session:

More information

Political Science 15, Winter 2014 Final Review

Political Science 15, Winter 2014 Final Review Political Science 15, Winter 2014 Final Review The major topics covered in class are listed below. You should also take a look at the readings listed on the class website. Studying Politics Scientifically

More information

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo Please note the page numbers listed for the Lind book may vary by a page or two depending on which version of the textbook you have. Readings: Lind 1 11 (with emphasis on chapters 5, 6, 7, 8, 9 10 & 11)

More information

Lecture 2: Learning and Equilibrium Extensive-Form Games

Lecture 2: Learning and Equilibrium Extensive-Form Games Lecture 2: Learning and Equilibrium Extensive-Form Games III. Nash Equilibrium in Extensive Form Games IV. Self-Confirming Equilibrium and Passive Learning V. Learning Off-path Play D. Fudenberg Marshall

More information

First Problem Set: Answers, Discussion and Background

First Problem Set: Answers, Discussion and Background First Problem Set: Answers, Discussion and Background Part I. Intuition Concerning Probability Do these problems individually Answer the following questions based upon your intuitive understanding about

More information

MATH-134. Experimental Design

MATH-134. Experimental Design Experimental Design Controlled Experiment: Researchers assign treatment and control groups and examine any resulting changes in the response variable. (cause-and-effect conclusion) Observational Study:

More information

Reliability, validity, and all that jazz

Reliability, validity, and all that jazz Reliability, validity, and all that jazz Dylan Wiliam King s College London Published in Education 3-13, 29 (3) pp. 17-21 (2001) Introduction No measuring instrument is perfect. If we use a thermometer

More information

Infinity-Valued Logic. A really powerful way to evaluate, grade, monitor and decide.

Infinity-Valued Logic. A really powerful way to evaluate, grade, monitor and decide. A really powerful way to evaluate, grade, monitor and decide. Let s start with single-value logic: Next comes dual-value logic: Then three-value logic: Then four-value logic: Beyond that we can move to

More information

Population. Sample. AP Statistics Notes for Chapter 1 Section 1.0 Making Sense of Data. Statistics: Data Analysis:

Population. Sample. AP Statistics Notes for Chapter 1 Section 1.0 Making Sense of Data. Statistics: Data Analysis: Section 1.0 Making Sense of Data Statistics: Data Analysis: Individuals objects described by a set of data Variable any characteristic of an individual Categorical Variable places an individual into one

More information

NEED A SAMPLE SIZE? How to work with your friendly biostatistician!!!

NEED A SAMPLE SIZE? How to work with your friendly biostatistician!!! NEED A SAMPLE SIZE? How to work with your friendly biostatistician!!! BERD Pizza & Pilots November 18, 2013 Emily Van Meter, PhD Assistant Professor Division of Cancer Biostatistics Overview Why do we

More information

10 Intraclass Correlations under the Mixed Factorial Design

10 Intraclass Correlations under the Mixed Factorial Design CHAPTER 1 Intraclass Correlations under the Mixed Factorial Design OBJECTIVE This chapter aims at presenting methods for analyzing intraclass correlation coefficients for reliability studies based on a

More information

The t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance?

The t-test: Answers the question: is the difference between the two conditions in my experiment real or due to chance? The t-test: Answers the question: is the difference between the two conditions in my experiment "real" or due to chance? Two versions: (a) Dependent-means t-test: ( Matched-pairs" or "one-sample" t-test).

More information

Randomized Evaluations

Randomized Evaluations Randomized Evaluations Introduction, Methodology, & Basic Econometrics using Mexico s Progresa program as a case study (with thanks to Clair Null, author of 2008 Notes) Sept. 15, 2009 Not All Correlations

More information

Chapter 19. Confidence Intervals for Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc.

Chapter 19. Confidence Intervals for Proportions. Copyright 2010, 2007, 2004 Pearson Education, Inc. Chapter 19 Confidence Intervals for Proportions Copyright 2010, 2007, 2004 Pearson Education, Inc. Standard Error Both of the sampling distributions we ve looked at are Normal. For proportions For means

More information

Statistical Tests Using Experimental Data

Statistical Tests Using Experimental Data Statistical Tests Using Experimental Data Alec Brandon July 15, 2015 Alternative title So you ve worked your tail off and have some experimental data. Now what? Why are we even talking about statistics?

More information

Still important ideas

Still important ideas Readings: OpenStax - Chapters 1 13 & Appendix D & E (online) Plous Chapters 17 & 18 - Chapter 17: Social Influences - Chapter 18: Group Judgments and Decisions Still important ideas Contrast the measurement

More information

CHAPTER ONE CORRELATION

CHAPTER ONE CORRELATION CHAPTER ONE CORRELATION 1.0 Introduction The first chapter focuses on the nature of statistical data of correlation. The aim of the series of exercises is to ensure the students are able to use SPSS to

More information

CHAPTER THIRTEEN. Data Analysis and Interpretation: Part II.Tests of Statistical Significance and the Analysis Story CHAPTER OUTLINE

CHAPTER THIRTEEN. Data Analysis and Interpretation: Part II.Tests of Statistical Significance and the Analysis Story CHAPTER OUTLINE CHAPTER THIRTEEN Data Analysis and Interpretation: Part II.Tests of Statistical Significance and the Analysis Story CHAPTER OUTLINE OVERVIEW NULL HYPOTHESIS SIGNIFICANCE TESTING (NHST) EXPERIMENTAL SENSITIVITY

More information

A Case Study: Two-sample categorical data

A Case Study: Two-sample categorical data A Case Study: Two-sample categorical data Patrick Breheny January 31 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/43 Introduction Model specification Continuous vs. mixture priors Choice

More information

Attitude toward Fundraising - Positive Attitude toward fundraising refers to how fundraising is valued and integrated within an organization

Attitude toward Fundraising - Positive Attitude toward fundraising refers to how fundraising is valued and integrated within an organization Attitude toward Fundraising - Positive Attitude toward fundraising refers to how fundraising is valued and integrated within an organization We believe fundraising is an opportunity to talk personally

More information

Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016

Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 Good afternoon everyone. My name is Stan Orchowsky, I'm the research director for the Justice Research and Statistics Association. It's my pleasure to welcome you this afternoon to the next in our Training

More information

Research Questions, Variables, and Hypotheses: Part 2. Review. Hypotheses RCS /7/04. What are research questions? What are variables?

Research Questions, Variables, and Hypotheses: Part 2. Review. Hypotheses RCS /7/04. What are research questions? What are variables? Research Questions, Variables, and Hypotheses: Part 2 RCS 6740 6/7/04 1 Review What are research questions? What are variables? Definition Function Measurement Scale 2 Hypotheses OK, now that we know how

More information

Chapter 8 Estimating with Confidence. Lesson 2: Estimating a Population Proportion

Chapter 8 Estimating with Confidence. Lesson 2: Estimating a Population Proportion Chapter 8 Estimating with Confidence Lesson 2: Estimating a Population Proportion Conditions for Estimating p These are the conditions you are expected to check before calculating a confidence interval

More information

Methods for Determining Random Sample Size

Methods for Determining Random Sample Size Methods for Determining Random Sample Size This document discusses how to determine your random sample size based on the overall purpose of your research project. Methods for determining the random sample

More information

STAT 200. Guided Exercise 4

STAT 200. Guided Exercise 4 STAT 200 Guided Exercise 4 1. Let s Revisit this Problem. Fill in the table again. Diagnostic tests are not infallible. We often express a fale positive and a false negative with any test. There are further

More information

Individual Packet. Instructions

Individual Packet. Instructions Individual Packet Instructions Step : Introductions and Instructions ( minutes). Start by having each person introduce themselves including their name and what they found most interesting about the introductory

More information

INTERNAL VALIDITY, BIAS AND CONFOUNDING

INTERNAL VALIDITY, BIAS AND CONFOUNDING OCW Epidemiology and Biostatistics, 2010 J. Forrester, PhD Tufts University School of Medicine October 6, 2010 INTERNAL VALIDITY, BIAS AND CONFOUNDING Learning objectives for this session: 1) Understand

More information

Classification. Methods Course: Gene Expression Data Analysis -Day Five. Rainer Spang

Classification. Methods Course: Gene Expression Data Analysis -Day Five. Rainer Spang Classification Methods Course: Gene Expression Data Analysis -Day Five Rainer Spang Ms. Smith DNA Chip of Ms. Smith Expression profile of Ms. Smith Ms. Smith 30.000 properties of Ms. Smith The expression

More information

Do not copy, post, or distribute

Do not copy, post, or distribute Hypothesis Testing LEARNING OBJECTIVES CHAPTER 7 After reading and studying this chapter, you should be able to do the following: Define the terms Type I error and Type II error, and explain their significance

More information

Appendix B Statistical Methods

Appendix B Statistical Methods Appendix B Statistical Methods Figure B. Graphing data. (a) The raw data are tallied into a frequency distribution. (b) The same data are portrayed in a bar graph called a histogram. (c) A frequency polygon

More information

Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank)

Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Evaluating Social Programs Course: Evaluation Glossary (Sources: 3ie and The World Bank) Attribution The extent to which the observed change in outcome is the result of the intervention, having allowed

More information

WRITTEN PRELIMINARY Ph.D. EXAMINATION. Department of Applied Economics. January 17, Consumer Behavior and Household Economics.

WRITTEN PRELIMINARY Ph.D. EXAMINATION. Department of Applied Economics. January 17, Consumer Behavior and Household Economics. WRITTEN PRELIMINARY Ph.D. EXAMINATION Department of Applied Economics January 17, 2012 Consumer Behavior and Household Economics Instructions Identify yourself by your code letter, not your name, on each

More information

e.com/watch?v=hz1f yhvojr4 e.com/watch?v=kmy xd6qeass

e.com/watch?v=hz1f yhvojr4   e.com/watch?v=kmy xd6qeass What you need to know before talking to your statistician about sample size Sharon D. Yeatts, Ph.D. Associate Professor of Biostatistics Data Coordination Unit Department of Public Health Sciences Medical

More information

Inferential Statistics

Inferential Statistics Inferential Statistics and t - tests ScWk 242 Session 9 Slides Inferential Statistics Ø Inferential statistics are used to test hypotheses about the relationship between the independent and the dependent

More information

Reflection Questions for Math 58B

Reflection Questions for Math 58B Reflection Questions for Math 58B Johanna Hardin Spring 2017 Chapter 1, Section 1 binomial probabilities 1. What is a p-value? 2. What is the difference between a one- and two-sided hypothesis? 3. What

More information

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo Business Statistics The following was provided by Dr. Suzanne Delaney, and is a comprehensive review of Business Statistics. The workshop instructor will provide relevant examples during the Skills Assessment

More information

Chapter 8 Estimating with Confidence. Lesson 2: Estimating a Population Proportion

Chapter 8 Estimating with Confidence. Lesson 2: Estimating a Population Proportion Chapter 8 Estimating with Confidence Lesson 2: Estimating a Population Proportion What proportion of the beads are yellow? In your groups, you will find a 95% confidence interval for the true proportion

More information

Online Introduction to Statistics

Online Introduction to Statistics APPENDIX Online Introduction to Statistics CHOOSING THE CORRECT ANALYSIS To analyze statistical data correctly, you must choose the correct statistical test. The test you should use when you have interval

More information

Handout 16: Opinion Polls, Sampling, and Margin of Error

Handout 16: Opinion Polls, Sampling, and Margin of Error Opinion polls involve conducting a survey to gauge public opinion on a particular issue (or issues). In this handout, we will discuss some ideas that should be considered both when conducting a poll and

More information

Final Exam Practice Test

Final Exam Practice Test Final Exam Practice Test The t distribution and z-score distributions are located in the back of your text book (the appendices) You will be provided with a new copy of each during your final exam True

More information

Lec 02: Estimation & Hypothesis Testing in Animal Ecology

Lec 02: Estimation & Hypothesis Testing in Animal Ecology Lec 02: Estimation & Hypothesis Testing in Animal Ecology Parameter Estimation from Samples Samples We typically observe systems incompletely, i.e., we sample according to a designed protocol. We then

More information

Chi Square Goodness of Fit

Chi Square Goodness of Fit index Page 1 of 24 Chi Square Goodness of Fit This is the text of the in-class lecture which accompanied the Authorware visual grap on this topic. You may print this text out and use it as a textbook.

More information

Never P alone: The value of estimates and confidence intervals

Never P alone: The value of estimates and confidence intervals Never P alone: The value of estimates and confidence Tom Lang Tom Lang Communications and Training International, Kirkland, WA, USA Correspondence to: Tom Lang 10003 NE 115th Lane Kirkland, WA 98933 USA

More information