Unit 3: Collecting Data. Observational Study Experimental Study Sampling Bias Types of Sampling

Transcription

1 Unit 3: Collecting Data Observational Study Experimental Study Sampling Bias Types of Sampling Feb 7 10:12 AM The step of data collection is critical to obtain reliable information for your study. 2 Types of Studies 1. Def 1: observational study where an investigator observes characteristics of a sample selected from one or more existing populations Goal to draw conclusions about the corresponding population or about the difference between 2 or more populations. 2. Def 2: experimental study investigator observes how a response variable behaves when one or more explanatory variables, or factors, are manipulated Goal to determine the effect of the manipulated explanatory variables on the response variable. *Both study types can be used to compare groups, but in an experiment the researcher controls who is in which group. Feb 7 10:24 AM 1

2 *In an observational study it is impossible to draw clear cause and effect conclusions because we cannot rule out the result is due to some other variable. Def 3: confounding variable a variable that is related to both group membership and the response variable of interest in a research study. ex In the article "Panel Can't Determine the Value of Daily Vitamins" (San Luis Obispo Tribune, July 1, 2003) observational study where concern was that people who take vitamins might be healthier just because they tend to take better care of themselves in general. Feb 7 10:29 AM *Think carefully about the objectives of a statistical study before planning how to collect the data* Feb 7 10:30 AM 2

3 Sampling When sampling your goal is to collect a sample that can be representative of your data. *there is no way to tell by looking at a sample if it is representative of the population so our assurance comes form the method used to select the sample Def 4: Sample Bias the tendency for samples to differ from the corresponding population in some systematic way 3 types 1. Selection Bias when the way the sample was selected excludes some part of the interested population ex a researcher constructs telephone study of all the residents of a particular city, but the method of selection excludes the homeless or those without telephones Selection bias can also occur if only volunteers or self selected individuals are used in a study ex the people who choose to participate in the study may have views that differ from those who choose not to participate. Observational 2. Measurement or response bias method of observation tends to produce values that systematically differ from the true value in some way. Types of measurement bias improperly calibrated scale questions on a survey are worded in a way that influences response ex from a Gallup Survey sponsored by the American Paper Institute "It is estimated that disposable diapers account for less than 2 percent of the trash in today's landfills. In contrast, beverage containers, third class mail and yard waste are estimated to account for about 21 percent of trash in landfills. Given this, in your opinion, would it be fair to tax or ban disposable diapers?" appearance or behavior of the person asking the question the group or organization conducting the study tendency of people not to be completely honest when asked about illegal behavior or unpopular beliefs Feb 7 10:47 AM 3

4 3. Nonresponse bias when responses are not obtained from all individuals selected for inclusion in the sample. may not be avoidable in most survey's, so bias rate is lowest when response rate is high. to reduce nonresponse bias it is critical to make a serious effort to follow up with individuals who do not respond to any initial request for info. surveys commonly collected by mail, phone, or personal interview It is important to note that bias is introduced by the way in which a sample is selected or by the way in which data is collected. Increaseing the size of the sample does nothing to reduce bias if the method of selection is flawed or if nonresponse rate remains high. Feb 7 11:22 AM Random Sampling Def 5 simple random sample sample selected from the population that ensures every different possible sample of the desired size has the same chance of being selected. ex sample of 10 employees chosen from all those who work at a large design firm, sample must account for the different subsets of employees, take into account part time employees, they must all have an equal chance of selection. Types or Random Sampling lottery put all names in a 'hat', mix up, and selected decided sample size sampling frame put all names in a list and correspond them with a number and use a random number generator Def 6: sampling without replacement once an individual from a population is selected they cannot be selected again Def 7: sampling with replacement after an individual from the population is selected for inclusion and info recorded the individual is placed back in the population and can be selected again. Feb 7 11:31 AM 4

5 *the goal of random sampling is to produce a sample that is likely to be representative of the population, it however does not guarantee that the sample will be representative *Common misconception that if the sample is relatively small compared to the population size, the sample cannot accurately reflect the population. FALSE Feb 7 11:45 AM Other Sampling Methods simple random sampling provides researchers with a sampling method that is objective and free of bias by some alternate methods may be less costly, easier to implement, or even more accurate. 1. Stratified Random Sampling used if the population can be divided into a set of non overlapping subgroups allows us to make more accurate inferences about a population separate simple random samples are selected from each sub group ex student body population at Fairchild Wheeler is made up of 30% suburban students and 70% Bridgeport students. To select 10 students who best represent the student population you would randomly select 3 suburban students and 7 Bridgeport students. Def 8: strata subgroups of population, each individual group called a stratum Feb 7 11:53 AM 5

6 2. Cluster Sampling involves dividing the population of interest into nonoverlapping subgroups, clusters, clusters are selected at random and then all individuals in cluster are included in the sample. ideal situation for cluster sampling is when the clusters represent the characteristics of the population, if this happens then you can have a small number of clusters, if this is not the case it is better to have a bigger sample size. ex There are 600 seniors at a high school, each senior homeroom has about 25 students in it. If administration wants to have a sample of 75 students they could randomly select 3 homerooms for the sample. Feb 7 11:59 AM 3. Systematic Sampling used when it is possible to view the population as consisting of a list or some other sequential arrangement. a value of k is specified, then one of the first k individuals is selectedat random, after which every kth person in the sequence is included in the sample (called 1 in the k systematic sample) k chosen to achieve a desired sample size. ex sample of faculty members at a university can be selected from the faculty phone directory. One of the first k = 20 faculty members listed could be selected at random then every 20th faculty member after that on the list would be included in the sample. 900 faculty members, 1 in 20 systematic sample would result in a sample size of 45 Feb 7 12:06 PM 6

7 4. Convenience Sampling: DON'T GO THERE! using an easily available or convenient group to form a sample ex voluntary response sampling like responding to an advertisement Feb 7 12:11 PM Observational studies attempt to measure personal opinion or attitudes using responses frequently to a survey. Def 8: survey a voluntary encounter between strangers in which an interviewer seeks information from a respondent Types in person over the phone written questionnaire Feb 9 3:08 PM 7

8 Simple Experiments Remember: experiment is a study in which one or more or more explanatory variables are manipulated in order to observe the effect on a response variable Def 9: explanatory variables variables that have values that can be controlled by the experimenter (Factors) Def 10: response variable variable that is not controlled and is measured as part of the experiment. Def 11: experimental condition any particular combination of variables for the explanatory variables. (Treatments) ex The Study is Determining the effect of room temperature on performance on first year calculus exams explanatory room temp response exam performance experimental conditions different temps room is set at (65 and 75 degrees We use the explanatory and response variables to describe the purpose of the experiment in the following terms: Experimental A well designed experiment requires more than just manipulating the explanatory variables; the design must also eliminate other possible explanations of the experimental results will not be conclusive. Def 12: extraneous variable a variable that is not one of the explanatory variables in the study but is thought to affect the response variable. Exam example time of day different instructor different textbook ability of students Feb 9 2:05 PM 8

9 Key Concepts in Experimental Design 1. direct control instructor sets standards for the variables she can control ex textbook, time of day 2. blocking blocking creates groups that are similar with respect to blocking variables; then all treatments are tried in each block ex instructor blocking variable, If there are 10 section under experiment and 5 instructors each instructor would be given a class at 65 and 75 degrees. *two variables are confounding if their efforts on the response variable cannot be distinguished from on another (usually between an extraneous variable and explanatory variable) ex one teacher teaching 65 degrees and second teacher teaching 75 degrees. 3. random assignment used to handle extraneous variables, used to create experimental groups as much alike as possible. ex students assigned to calculus using a random mechanism 4. replication ensures that there is an adequate number of observations for each experimental condition Feb 9 2:07 PM Def 13: completely randomized designs experimental designs in which units are assigned at random to treatments or in which treatments are assigned at random to trials Def 14: randomized block design design where blocking is used *volunteers can be used in experimental designs because the randomization of assignments of the treatments should result in comparable groups so treatment effects can still be assessed. Feb 9 2:31 PM 9

10 Additional Considerations for Experimental Design 1. control group used to determine whether some treatment has an effect, it is an experimental group that does not receive the treatment, 2. placebo something that is identical to the treatment received by the treatment group except that it contains no active ingredient. The group getting the placebo does not know that they are. Used when the power of suggestion might have an effect. 3. single blind experiment in which the subjects do not know what treatments they have received 4. double blind experiment in which neither the subjects nor the individuals who measure the response know which treatment was received. Feb 9 2:53 PM Feb 9 3:06 PM 10