AP Statistics Unit 4.2 Day 3 Notes: Experimental Design OBSERVATION -observe outcomes without imposing any treatment EXPERIMENT -actively impose some treatment in order to observe the response I ve developed a new rabbit foot, Hippity Hop. Hippity Hop makes fur soft & shiny! Increases energy! Has 100% of daily vitamins & essential oils! Can I just make these claims? NO What must I do to make these claims? DO AN EXPERIMENT Who (what) should I test this on? RABBITS What do I test? TYPE OF FOOD EXPERIMENTAL UNITS -the single individual (person, animal, plant, etc.) to which the different treatments are assigned FACTOR - is the explanatory variable LEVEL - a specific value for the factor RESPONSE VARIABLE - what you measure TREATMENT - a specific experimental condition applied to the units Name: Date: Per: I plan to test my new rabbit food. What are my experimental units? What is my factor? What is the response variable? Expt1: I ll use my pet rabbit, Lucky! Results: Since Lucky s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food! NOTHING TO COMPARE CONTROL GROUP - a group that is used to compare the factor against; can be a placebo or the old /current item PLACEBO - a dummy treatment that can have no physical effect Expt2: RABBITS TYPE OF FOOD HOW WELL THEY GROW Now I ll use Lucky & my friend s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old rabbit food. Results: Wow! Lucky is bigger & shinier so Hippity Hop is better! HOW DID THEY COMPARE BEFORE TREATMENT
Expt3: The first five rabbits that I catch will get Hippity Hop food and the remaining five will get the old food. Results: The Hippity Hop rabbits have scored higher so it s the better food! Expt4: Number the rabbits from 1-10. Place the numbers in a hat. The first five numbers pulled from the hat will be the rabbits that get Hippity Hop food. The remaining rabbits get the old food. Results: I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food. I AM THE ONE EVALUATING AND I KNOW WHICH GOT MY SPECIAL FOOD - BLINDING method used so that units do not know which treatment they are getting DOUBLE BLINDING - neither the units nor the evaluator know which treatment a subject received Expt4 results: 1 ST 5 CAUGHT MAYBE THE FATTEST Hippity Hop Rabbit Food makes fur soft and shiny, & increases energy for all types of rabbits! Principles of experimental design CONTROL 1) of effects of extraneous variables on the response by comparing treatment groups to a control group (placebo or old ) REPLICATION 2) of the experiment on many subjects to quantify the natural variation in the experiment RANDOMIZATION 3) - the use of chance to assign subjects to treatments The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!!!!! Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. What are the experimental units? PLOTS OF LAND What is the factor (explanatory variable)? TYPE OF FERTILIZER How many levels? 3 TYPES OF FERTILIZER What is the response variable? YIELD OF CROP How many treatments? 3 Can I make this claim? NO, ONLY TESTED ON SOME TYPES OF RABBBITS.
Example 2: A consumer group wants to test cake pans to see which works the best (bakes evenly). It will test aluminum, glass, and plastic pans in both gas and electric ovens. Experiment units? CAKE BATTER Types of Experimental Designs 1) Complete randomized design all experimental units are allocated at random among all treatments Factors? TYPE OF PAN AND TYPE OF OVEN Levels? 3 TYPES OF PAN AND 2 TYPES OF OVEN Response variable? HOW EVENLY THE CAKE BAKES Number of treatments? 6 Example: In this design, we would randomly select 60 students and randomly split them into two groups with 30 each. One group does not take the practice exam, while the other does. We have the two groups then take the actual exam and we compare results. Example 3: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared. 2) Randomized block design Units are blocked into groups (homogeneous) and then randomly assigned to treatments Units should be blocked on a variable that affects the response!!!!!! Why is the same type of seed used on all 15 plots? THIS IS FOR CONTROL DIFFERENT SEEDS MIGHT GROW DIGFERENTLY. What are other potential extraneous variables? SOIL TYPE, AMOUNT OF RAIN Does this experiment have a placebo? Explain NO, NOT NEEDED. Example: Student maturity is a huge factor in college success. Another idea might be to split our sample by academic year - those in their first year versus those in their second. Essentially, we're stratifying the sample, and then doing a completely randomized design on each of the "strata".
3) Matched Pair design match up experimental units according to similar characteristics & randomly assign on to one treatment & the other automatically gets the 2nd treatment have each unit do both treatments in random order the assignment of treatments is dependent HAVE EACH PERSON DO ALL 4 PROGRAMS BUT RANDOMIZE THE ORDER THAT THEY DO THEM IN. Example: There are a couple ways to do matchedpairs - we could find people who are very similar somehow, and have one do the practice exam and the other not. Unfortunately, there are so many factors affecting performance on the exam, this pretty impractical. Another way to do a matched-pair design is to have the same individual before and after the treatment. In this case, we could do just that - give the exam, have students study the practice exam, and then give the exam again. The problem with this design is that we don't know if the improvement (if any) is from the practice exam or just from seeing the material again. A better plan would be to have all individuals take the exam as a "pre-test", then have 30 students take the practice exam, while the rest do not. Then we have the students all take the exam again, and we compare the "before" and the "after". - the effect of the confounding variable on the response cannot be separated from the effects of the explanatory variable (factor) Confounding does NOT occur in a completely randomized design!!! Example 4: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed. Is this an experiment? Why or why not? NO, OBSERVATIONAL STUDY What is a potential confounding variable? AGE, YOUNGER PEOPLE MIGHT BE MORE LIKELY TO BE A VICTIM OF VIOLENT CRIMES. Example 5: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured. Is this an experiment? Why or why not? EXPERIMENT BECAUSE WE ARE IMPOSING A TREATMENT What type of design is this? COMPLETELY RANDOMIZED DESIGN Factors? WORD PROCESSING PROGRAM Levels? 4 DIFFERENT WORD PROCESSING PROGRAMS Response Variable? SPEED Is there a potential confounding variable? NO, CONFOUNDING DOES NOT OCCUR IN A COMPLETELY RADOMIZED DESIGN Can this design be improved? Explain. HAVE EACH PERSON DO ALL 4 PROGRAMS BUT RANDOMIZE THE ORDER THAT THEY DO THEM IN.
Example 6: Suppose that the manufacturer wants to test a new fertilizer against the current one on the market. Ten 2-acre plots of land scattered throughout the county are used. Each plot is subdivided into two subplots, one of which is treated with the current fertilizer, and the other with the new fertilizer. Wheat is planted and the crop yields are measured. What type of design is this? Why use this method? MATCHED PAIR DESIGN. BECAUSE THERE ARE PROBABLY DIFFERENCES IN THE SEPARATE PLOTS BUT IF WE DIVIDE EACH PLOT, WE CAN KEEP OTHER VARIBLES THE SAME When does randomization occur? WHEN YOU CHOOSE WHICH SIDE GETS CURRENT FERTILIZER AND WHICH SIDE GETS THE NEW ONE. Randomization reduces bias by spreading any uncontrolled confounding variables evenly throughout the treatment groups. Bias is a systematic error in measuring the estimate. Blocking also helps reduce variability. Is there another way to reduce variability? Variability is controlled by sample size. Larger samples produce statistics with less variability. TARGET IS THE VALUE OF THE PARAMETER. THE REAL VALUE OF THE POPULATION, WHAT YOU ARE TRYING TO FIND OUT. Bias vs. Variability HOGH BIAS AND HIGH VARIABILITY HOGH BIAS AND LOW VARIABILITY LOW BIAS AND HIGH VARIABILITY LOW BIAS AND LOW VARIABILITY