Aim: Intro Chp. 4 Designing Studies

Transcription

1 RECALL: Aim: Intro Chp. 4 Designing Studies The distinction between population and sample is basic to statistics. To make sense of any sample result, you must know what population the sample represents Definition: The population in a statistical study is the entire group of individuals about which we want information. A sample is the part of the population from which we actually collect information. We use information from a sample to draw conclusions about the entire population. Do Now: Page 226/ #s 1, 2, & 4 #2 Population: All the artifacts discovered at the dig. Sample: Those artifacts that are chosen for inspection #4 Population: 45,000 people who made credit card purchases. Sample: 137 people who returned the survey form. AutoSave 1

2 Activity: See no evil, hear no evil? Follow the directions on Page 206 Turn in your results. AutoSave 2

3 Choosing a sample from a large, varied population is not that easy. Step 1: Define the population we want to describe. Step 2: Say exactly what we want to measure. A sample survey is a study that uses an organized plan to choose a sample that represents some specific population. Step 3: Decide how to choose a sample from the population. Need to be aware of how to Sample Badly How can we choose a sample that we can trust to represent the population? There are a number of different methods to select samples. AutoSave 3

4 Definition: 1. Choosing individuals who are easiest to reach results in a convenience sample. Convenience samples often produce unrepresentative data why? The design of a statistical study shows bias if it systematically favors certain outcomes. Convenience samples are almost guaranteed to show bias. 2. Therefore, voluntary response samples, in which people decide whether to join the sample in response to an open invitation. A voluntary response sample consists of people who choose themselves by responding to a general appeal. Voluntary response samples show bias because people with strong opinions (often in the same direction) are most likely to respond. AutoSave 4

5 How to Sample Well: Random Sampling The statistician s remedy is to allow impersonal chance to choose the sample. A sample chosen by chance rules out both favoritism by the sampler and self selection by respondents. Random sampling, the use of chance to select a sample, is the central principle of statistical sampling. 3. Simple random sample (SRS) of size n consists of n individuals from the population chosen in such a way that every set of n individuals has a equal chance to be the sample actually selected. In practice, people use random numbers generated by a computer or calculator to choose samples. If you don t have technology handy, you can use a table of random digits. AutoSave 5

6 Definition: How to Choose an SRS A table of random digits is a long string of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 with these properties: Each entry in the table is equally likely to be any of the 10 digits 0 9. The entries are independent of each other. That is, knowledge of one part of the table gives no information about any other part. Step 1: Label. Give each member of the population a numerical label of the same length. Step 2: Table. Read consecutive groups of digits of the appropriate length from Table D. Your sample contains the individuals whose labels you find. Example: How to Choose an SRS, pg. 213 Spring Break Problem: Use Table D at line 130 to choose an SRS of 4 hotels. 01 Aloha Kai 08 Captiva 15 Palm Tree 22 Sea Shell 02 Anchor Down 09 Casa del Mar 16 Radisson 23 Silver Beach 03 Banana Bay 10 Coconuts 17 Ramada 24 Sunset Beach 04 Banyan Tree 11 Diplomat 18 Sandpiper 25 Tradewinds 05 Beach Castle 12 Holiday Inn 19 Sea Castle 26 Tropical Breeze 06 Best Western 13 Lime Tree 20 Sea Club 27 Tropical Shores 07 Cabana 14 Outrigger 21 Sea Grape 28 Veranda AutoSave 6

7 AutoSave 7

8 Intro Chapter 4 Spring Break! Choosing an SRS with Table D The school newspaper is planning an article on family friendly places to stay over spring break at a nearby beach town. The editors intend to call 4 randomly chosen hotels to ask about their amenities for families with children. They have an alphabetized list of all 28 hotels in the town. PROBLEM: Use Table D at line 130 to choose an SRS of 4 hotels for the editors to call. SOLUTION: We ll use the two step process for selecting an SRS using Table D. Step 1: Label. Two digits are needed to label the 28 resorts. We have added labels 01 to 28 to the alphabetized list of resorts below. AutoSave 8

9 AutoSave 9

10 Graphing calculator: Check that your calculator s random number generator is working properly. TI 83/84: Press, then select PRB and 5:randInt(. Complete the commandrandint(1, 28) and press AutoSave 10

11 Other Sampling Methods One common alternative to an SRS involves sampling important groups (called strata) within the population separately. These subsamples are combined to form one stratified random sample. 4. Stratified Random Sample: To select a stratified random sample, first classify the population into groups of similar individuals, called strata. Then choose a separate SRS in each stratum and combine these SRSs to form the full sample. Although a stratified random sample can sometimes give more precise information about a population than an SRS, both sampling methods are hard to use when populations are large and spread out over a wide area. In that situation, we d prefer a method that selects groups of individuals that are near one another 5. Cluster Sample: To take a cluster sample, first divide the population into smaller groups. Ideally, these clusters should mirror the characteristics of the population. Then choose an SRS of the clusters. All individuals in the chosen clusters are included in the sample AutoSave 11

12 AutoSave 12

13 Aim: Experimental Design Do Now: a) What are the types of sampling bias that can occur? b)give an example AutoSave 13

14 Types of Statistical Bias Voluntary response bias occurs when subjects voluntarily choose to be in the sample, and people usually volunteer only if they have strong opinions. Undercoverage occurs when some groups of people are ignored when the sample is being chosen. A survey sent by e mail ignores people without computers. Non response bias occurs when some subjects chosen for the sample do not answer. (only people with a lot of time on their hands respond.) Response bias occurs when subjects give incorrect answers, either because they have forgotten details or they lie about embarrassing or illegal activities. AutoSave 14

15 Aim: Inference for Sampling Do Now: Explain why it would be better to use homeroom classes as clusters than to use math classes as clusters? What is the statistical meaning of random? SRS does not favor any part of the population. Although a stratified random sample can sometimes give more precise information about a population than an SRS, both sampling methods are hard to use when populations are large and spread out over a wide area. In that situation, we d prefer a method that selects groups of individuals that are near one another. (cluster sampling) Clusters should mirror the characteristics of the population. Random in statistics means due to chance. AutoSave 15

16 Inference for Sampling: The purpose of a sample is to give us information about a larger population. The process of drawing conclusions about a population on the basis of sample data is called inference. Why should we rely on random sampling? 1. To eliminate bias in selecting samples from the list of available individuals. 2. The laws of probability allow trustworthy inference about the population a) Results from random samples come with a margin of error that sets bounds on the size of the likely error. b) Larger random samples give better information about the population than smaller samples. Errors with samples that can occur can be classified as sample errors or non sampling errors. Sample Errors NON sampling Errors Convenience sampling: doesn't represent the population fairly. (Bias) Voluntary Response: Only those with strong opinions would be included. Participation is voluntary.(bias) Sample frame (the list made in choosing the sample) under coverage: occurs when some groups are left out of the process of choosing the sample. Non response: refusal to participate or can't be contacted. (note: this is often after the sample has been selected) Response Bias: When individuals provide false or invalid responses. Wording of questions can influence answers given to a sample survey. CYU: page 224 AutoSave 16

17 GATHERING DATA OBSERVATIONAL STUDIES EXPERIMENTS SIMULATIONS AutoSave 17

18 Types of study designs 1. Observational Studies: are aimed to gather information about a population without disturbing the population in process. Observational studies watch the behavior of subjects without imposing any treatments. 2. Experiments don't just observe, they actively impose some treatment on individuals to measure their responses. The individuals studied in an experiment are often called subjects. A treatment is any specific experimental condition applied to the subjects. 3. Simulations imitate chance behavior based on a model that accurately reflects the situation. AutoSave 18

19 The distinction between observational study and experiment is one of the most important in statistics. When our goal is to understand cause and effect, experiments are the only source of fully convincing data. AutoSave 19

20 Observational Study versus Experiment Observational studies of the effect of one variable on another often fail because of confounding between the explanatory variable and one or more lurking variables. Definition: A lurking variable is a variable that is not among the explanatory or response variables in a study but that may influence the response variable. Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other. Well designed experiments take steps to avoid confounding. AutoSave 20

21 Is it possible to get convincing evidence of a cause and effect relationship? Yes, through a well planned design experiment. Experiments study relationships between two or more variables. It must identify at least one explanatory variable (independent) and at least one response variable (dependent). x y Note: Dashed line shows an association Arrow shows a cause and effect link. Causation x is explanatory, y is response, z is a lurking variable. AutoSave 21

22 A good design experiment ensures that it will show the effect of the explanatory on the response variable. Sometimes other variables can make this difficult to determine. Lurking Variables x y z The observed association between the variables x and y is explained by a lurking variable z. Both x and y change in response to changes in z. This common response creates an association even if there is no direct causal link between the variables x and y. Common Response AutoSave 22

23 Another variable that can influence the response is a Confounding Variable x? z? y Both the explanatory variable and the lurking variable may influence the response variable. X and Z are associated, so we can't distinguish the influence of x from the influence of z. Hard to say if x influences y at all. Example: Confounding in the Nova Southeastern University study from Chapter case. (pg. 257) AutoSave 23

24 Classwork: CYU page 233 AutoSave 24

25 For each of the examples below, determine what the lurking variable could be. Lurking Variable (a.k.a. extraneous variable) 1. Studies show that there is a strong positive association between the number of firefighters who respond to a fire and the amount of damage done to the structure. a) Is it possible to draw any conclusions based on this information? b) What could be a lurking variable? 2. In 1990 s researchers measured the number of television set per person x and the life expectancy y for the world s nations. It showed a high positive correlation. a) What conclusion could we draw from the given information? b) What could be a lurking variable? 3. A group of college students think herbal tea can improve people s moods. They make weekly visits to a local nursing home, where they visit with the residents and serve them tea. The staff reports that the residents are more cheerful and healthy. a) What assumptions can be made if any? b) What could be the lurking variable? AutoSave 25

26 Experiments All experiments consist of 3 parts: 1) One or more independent variables, also known as factors. Factors will contain 2 or more levels. A combination of factor levels is known as treatments. 2) One or more dependent variables 3) Experimental units : the collection of individuals to which the treatments are applied. When the units are human, they are often called subjects. AutoSave 26

27 Example In this hypothetical experiment, the researcher is studying the possible effects of Vitamin C and Vitamin E on health. There are two factors dosage of Vitamin C and dosage of Vitamin E. The Vitamin C factor has three levels 0 mg per day, 250 mg per day, and 500 mg per day. The Vitamin E factor has 2 levels 0 mg per day and 400 mg per day. The experiment has six treatments. Treatment 1 is 0 mg of E and 0 mg of C, Treatment 2 is 0 mg of E and 250 mg of C, and so on. Vitamin E 0 mg 400 mg Vitamin C 0 mg 250 mg 500 mg 0 mg AutoSave 27

28 Language of Experiments A Louse y situation: A study published in the New England Journal of Medicine (March 11, 2010) compared two medicines to treat head lice: an oral medication called ivermectin and a topical lotion containing malathion. Researchers studied 812 people in 376 households in seven areas around the world. Of the 185 households randomly assigned to ivermectin, 171 were free from head lice after two weeks compared with only 151 of the 191 households randomly assigned to malathion. Identify the: a) experimental units: b) explanatory variable: c) response variable: d) treatments: AutoSave 28

29 Growing tomatoes: Does adding fertilizer affect the productivity of tomato plants? How about the amount of water given to the plants? To answer these questions, a gardener plants 24 similar tomato plants in identical pots in his greenhouse. He will add fertilizer to the soil in half of the pots. Also, he will water 8 of the plants with 0.5 gallon of water per day, 8 of the plants with 1 gallon of water per day, and the remaining 8 plants with 1.5 gallons of water per day. At the end of three months, he will record the total weight of tomatoes produced on each plant. Identify the: a) experimental units: b) explanatory variable: c) response variable: d) treatments: AutoSave 29

30 Many students regularly consume caffeine to help them stay alert. Thus, it seems plausible that taking caffeine might increase an individual s pulse rate. Is this true? Here is an initial plan for the experiment: a) measure initial pulse rate b) give each student some caffeine c) wait for a specified time d) measure final pulse rate e) compare final and initial rates What are some problems with this plan? Even if every student s pulse rate went up, we couldn t attribute the increase to caffeine. Perhaps the excitement of being in an experiment or the sugar in the cola made their pulse rates increase. In other words, there are many variables that are potentially confounded with caffeine. What lurking variables should we be aware of? AutoSave 30

31 How to Experiment Well: The Randomized Comparative Experiment Experiments without random assignment are never a good idea A good experiment needs comparison to prevent some lurking variables from becoming confounded with the explanatory variable. Ex. Have some students drink cola with caffeine while others drink cola without caffeine AutoSave 31

32 Explain how to use all three principles of experimental design in the caffeine experiment. Control: There should be a control group that receives noncaffeinated cola. Also, the subjects in each group should receive exactly the same amount of cola served at the same temperature. Also, each type of cola should look and taste exactly the same and have the same amount of sugar. Subjects should drink the cola at the same rate and wait the same amount of time before measuring their pulse rates. If all of these lurking variables are controlled, they will not be confounded with caffeine or be an additional source of variability in pulse rates. Randomization: Subjects should be randomly assigned to one of the two treatments. This should roughly balance out the effects of the lurking variables we cannot control, such as body size, caffeine tolerance, and the amount of food recently eaten. Replication: We want to use as many subjects as possible to help make the treatment groups as equivalent as possible. This will give us a better chance to see the effects of caffeine, if there are any. AutoSave 32

33 Explain how we can conduct the caffeine experiment in a doubleblind manner. This means that neither the subjects nor the individuals measuring the results know what treatment was administered. In this case, we would need to ensure the subjects and those who came in contact with them were not told what type of cola they were drinking. What is the advantage of this? AutoSave 33

34 SUMMARY: With randomization, replication, and control, each treatment group should be nearly identical, and the effects of other variables should be about the same in each group. Now, if changes in the explanatory variable are associated with changes in the response variable, we can attribute the changes to the explanatory variable or the chance variation in the random assignment. AutoSave 34

35 Experimental Design What is it? AutoSave 35

36 3 Types of Design Completely Randomized Randomized Block Matched Pairs Experimental design refers to a plan for assigning subject and treatments to groups AutoSave 36

37 The Principles of Experimental Design 1. Control: In a well designed experiment, we try to control all sources of variation other than the factors (explanatory) we are testing by making conditions as similar as possible to all treatment groups. 2. Randomize: use impersonal chance to assign subjects to treatments. 3. Replicate: use enough subjects in each group to reduce chance variation in the results. AutoSave 37

38 Randomized Comparative Experimental Design is one of the most important ideas in statistics. To prevent lurking and confounding variables we should design experiments to compare two or more treatments. a.k.a. Completely Randomized experimental design Randomized Comparative Experimental Design Group 1 Treatment 1 Random Assignment Compare Group 2 Treatment 2 AutoSave 38

39 Block Design is a group of experimental subjects that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a block design, the random assignment of subjects to treatments is carried out separately within each block. Block Randomized Comparative Experimental Design Block 1 Random Assignment Group 1 Treatment 1 Group 2 Treatment 2 Compare Subjects Divided by common characteristics Block 2 Random Assignment Group 1 Treatment 1 Compare Group 2 Treatment 2 AutoSave 39

40 Matched Pairs Design is a special case of block design in which each block consists of only 2 subjects. Match up the subjects so that they are as similar as possible, and then randomly choose one from each pair to be in the treatment group and the other to be in the control group. Matched Pairs Randomized Comparative Experimental Design Block 1 Random Assignment Group 1 Treatment 1 Group 2 Treatment 2 Compare Block 2 Random Assignment Group 1 Treatment 1 Group 2 Treatment 2 Compare Subjects Divided by common characteristics Block 3 Random Assignment Group 1 Treatment 1 Compare Group 2 Treatment 2 Block Random Assignment Group 1 Treatment 1 Compare Group 2 Treatment 2 AutoSave 40

41 AutoSave 41

42 Standing Sitting AutoSave 42

43 Better Texting? A cell phone company is considering two different keyboard designs (A and B) for its new line of cell phones. Researchers would like to conduct an experiment using subjects who are frequent texters and subjects who are not frequent texters. The subjects will be asked to text several different messages in 5 minutes. The response variable will be the number of correctly typed words. (a) Explain why a randomized block design might be preferable to a completely randomized design for this experiment. Because the subjects include people of varying texting abilities, a completely randomized design would lead to lots of variability in the response variable. However, in a randomized block design, the experienced texters would be compared to each other and the novice texters would be compared to each other. This will make it easier to see a difference in the response variable if there is one. (b) Outline a randomized block experiment using 100 frequent texters and 200 novice testers. Here is an outline of this experiment. To randomly assign the experienced subjects to the two groups, put 50 slips of paper labeled A in a hat and 50 slips of paper labeled B in a hat. Mix up the papers and have each subject select one to determine which design he or she will use. Follow a similar process for the 200 novice texters. After the experiment compare the mean number of words correct for the two different designs within each block. AutoSave 43

44 An article in a local newspaper reported that dogs kept as pets tend to be overweight. Veterinarians say that diet and exercise will help these chubby dogs get in shape. The veterinarians propose two different diets (Diet A and Diet B) and two different exercise programs (Plan 1 and Plan 2). Diet A: owners control the proportions of dog food and dog treats Diet B: a mixture of fresh vegetables with the dog food and substitute regular dog treats with baby carrots. Plan 1: three 30 minute walks a week Plan 2: 20 minute walks daily Sixty dog owners volunteer to take part in an experiment to help their chubby dogs lose weight. 1. Identify the following: a) The subjects: b) The factor(s) and the number of level(s) for each: c) The number of treatments: d) Whether of not the experiment is blind (or double blind): e) The response variable: 2. Design an experiment to determine whether the diet and exercise program are effective in helping dogs to lose weight. AutoSave 44

45 Confounding Variables Confounding variables are more common in experiments. It occurs due to poor design in an experiment. It arises when the response we see in an experiment is at least partially attributable to uncontrolled variables. For example: You want to know if Drug X has an effect on causing sleep. The experimenter must take care to design the experiment so that he can be very sure that the subjects in the study fell asleep because of the influence of his Drug X, and that the sleepiness was not caused by other factors. Those other factors would be confounding variables. Confounding refers to a problem that can arise in an experiment, when there is another variable that may effect the response and is in some way tied together with the factor under investigation, leaving us unable to tell which of the two variables (or perhaps some interaction) caused the observed response. For example, we plant tomatoes in a garden that's half shaded. We test a fertilizer by putting it on the plants in the sun and apply none to the shaded plants. Months later the fertilized plants bear more and better tomatoes. Why? Well, maybe it's the fertilizer, maybe it's the sun, maybe we need both. We're unable to conclude that the fertilizer works because any effect of fertilizer is confounded with any effect of the extra sunshine. A confounding variable is one whose effects on the response variable cannot be distinguished from one or more of the explanatory variables in the study. AutoSave 45

46 Note: Dashed line shows an association Arrow shows a cause and effect link. x is explanatory, y is response, z is a lurking variable. x y Causation x y z The observed association between the variables x and y is explained by a lurking variable z. Both x and y change in response to changes in z. This common response creates an association even if there is no direct causal link between the variables x and y. Common Response x? y? z Both the explanatory variable and the lurking variable may influence the response variable. X and Z are associated, so we can't distinguish the influence of x from the influence of z. Hard to say if x influences y at all. Confounding A confounding variable is one whose effects on the response variable cannot be distinguished from one or more of the explanatory variables in the study. AutoSave 46

47 Characteristics of a Well Designed Experiment A well designed experiment includes design features that allow researchers to eliminate extraneous variables as an explanation for the observed relationship between the independent variable(s) and the dependent variable. Some of these features are listed below. > Control. Control refers to steps taken to reduce the effects of extraneous variables (i.e., variables other than the independent variable and the dependent variable). These extraneous variables are called lurking variables. > Control involves making the experiment as similar as possible for experimental units in each treatment condition. Three control strategies are control groups, placebos, and blinding. «Control group. A control group is a baseline group that receives no treatment or a neutral treatment. To assess treatment effects, the experimenter compares results in the treatment group to results in the control group. «Placebo. Often, participants in an experiment respond differently after they receive a treatment, even if the treatment is neutral. A neutral treatment that has no "real" effect on the dependent variable is called a placebo, and a participant's positive response to a placebo is called the placebo effect. «To control for the placebo effect, researchers often administer a neutral treatment (i.e., a placebo) to the control group. The classic example is using a sugar pill in drug research. The drug is considered effective only if participants who receive the drug have better outcomes than participants who receive the sugar pill. «Blinding. Of course, if participants in the control group know that they are receiving a placebo, the placebo effect will be reduced or eliminated; and the placebo will not serve its intended control purpose. «Blinding is the practice of not telling participants whether they are receiving a placebo. In this way, participants in the control and treatment groups experience the placebo effect equally. Often, knowledge of which groups receive placebos is also kept from people who administer or evaluate the experiment. This practice is called double blinding. It prevents the experimenter from "spilling the beans" to participants through subtle cues; and it assures that the analyst's evaluation is not tainted by awareness of actual treatment conditions. > Randomization. Randomization refers to the practice of using chance methods (random number tables, flipping a coin, etc.) to assign experimental units to treatments. In this way, the potential effects of lurking variables are distributed at chance levels (hopefully roughly evenly) across treatment conditions. > Replication. Replication refers to the practice of assigning each treatment to many experimental units. In general, the more experimental units in each treatment condition, the lower the variability of the dependent measures. Confounding Confounding occurs when the experimental controls do not allow the experimenter to reasonably eliminate plausible alternative explanations for an observed relationship between independent and dependent variables. Consider this example. A drug manufacturer tests a new cold medicine with 200 participants 100 men and 100 women. The men receive the drug, and the women do not. At the end of the test period, the men report fewer colds. This experiment implements no controls! As a result, many variables are confounded, and it is impossible to say whether the drug was effective. For example, gender is confounded with drug use. Perhaps, men are less vulnerable to the particular cold virus circulating during the experiment, and the new medicine had no effect at all. Or perhaps the men experienced a placebo effect. This experiment could be strengthened with a few controls. Women and men could be randomly assigned to treatments. One treatment group could receive a placebo, with blinding. Then, if the treatment group (i.e., the group getting the medicine) had sufficiently fewer colds than the control group, it would be reasonable to conclude that the medicine was effective in preventing colds. AutoSave 47

48 Understanding association and relationships with lurking & confounding variables. Note: Dashed line shows an association Arrow shows a cause and effect link. x y x is explanatory, y is response, z is a lurking variable. Causation x y z The observed association between the variables x and y is explained by a lurking variable z. Both x and y change in response to changes in z. This common response creates an association even if there is no direct causal link between the variables x and y. Common Response x? y? z Both the explanatory variable and the lurking variable may influence the response variable. X and Z are associated, so we can't distinguish the influence of x from the influence of z. Hard to say if x influences y at all. Confounding AutoSave 48

49 Principles of Experimental Design 1. Control for lurking variables that might affect the response: 2 or more treatments.the remedy for confounding is to perform a comparative experiment in which some units receive one treatment and similar units receive another. 2. Must have a random assignment. (method that uses chance methods.) 3. Replication assigning each treatment to many experimental units (subjects). In general, the more included, the lower the variability of the dependent measures. AutoSave 49

50 Definition: The Randomized Comparative Experiment In a completely randomized design, the treatments are assigned to all the experimental units completely by chance. Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment. AutoSave 50

51 Inference for Experiments In an experiment, researchers usually hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation. We can use the laws of probability, which describe chance behavior, to learn whether the treatment effects are larger than we would expect to see if only chance were operating. If they are, we call them statistically significant. An observed effect so large that it would rarely occur by chance is called statistically significant. A statistically significant association in data from a well designed experiment does imply causation. AutoSave 51

52 AutoSave 52