Types of Studies. There are three main types of studies. These are ways to collect data.

Transcription

1 Types of Studies There are three main types of studies. These are ways to collect data. Planned / designed experiments Observational studies Sample survey STA305 - week 1 1

2 Planned / Designed Experiments There are few key features of designed experiments that distinguish it from any other type of study. Independent variables of interest are carefully controlled by the experimenter in order to determine their effect on a response (dependent) variable. Experimental units are randomly allocated to a particular set of experimental conditions in order to avoid introducing subjective bias. Control of independent variables and randomization make it possible to infer cause and effect relationship. Use of replication. STA305 - week 1 2

3 Designed Experiment Example The effectiveness of an essential oil containing antiseptic mouth rinse (Listerine Antiseptic) and an antiplaque/antigingivitis dentifrice (Colgate Total) has been demonstrated in numerous double-blind clinical studies. A six- month clinical trial was conducted to determine their comparative effectiveness. The results of this study were published in The Journal of American Dental Association - may STA305 - week 1 3

4 Methods 360 subjects with mild-to-moderate gingival inflammation and plaque received a dental prophylaxis and began their randomly assigned brushing and rinsing regimen in an unsupervised setting. Subjects brushed for one minute and rinsed with 20 milliliters for 30 seconds twice daily for six months. There were three groups: 1. L (control toothpaste / Listerine rinse) 2. T (Colgate Total toothpaste / control rinse) 3. P (control toothpaste / control rinse) STA305 - week 1 4

5 Summary Toothpaste and mouth rinse used by each participant are controlled by researcher Participants were randomly allocated to one of 3 groups. Replication was used (360 participants) STA305 - week 1 5

6 Observational Studies In some cases, a study may be undertaken retrospectively. In other cases, it may not be possible to control independent variables or to randomize. In observational studies we simply collect information about variables of interest without applying any treatment or controlling for any factors. When factors are not controlled we are not able to infer a causeeffect relationship. STA305 - week 1 6

7 Example - Fidget to Fight Fat Source of example: Macleans January 18, 1999 If a team of American researchers is right, fidgeting could be a key to staying slim. After arranging for 16 volunteers do consume 1,000 extra calories a day for eight weeks, doctors at the Mayo Clinic in Rochester, Minn., saw weight gains ranging from 16 lbs to as few as 2 lbs and attributed the difference to a fidget factor. The low weight-gainers appeared to be doing more throughout the day than the others, said endocrinologist Dr. Michael D. Jensen. It could be fidgeting or moving around or just being a little more restless. Reporting in the journal Science, the doctors said the study seemed to show that men are better than women at burning extra calories. Without conscious effort the four women in the study did the least fidgeting. The factors reported in this article, fidgeting and gender, were not controlled by the researcher they were observed. Participants were not randomly allocated to fidget or to not fidget. STA305 - week 1 7

8 Sample Surveys Sample surveys have much in common with planned experiments. Surveys require existence of physically real population. Survey design includes selection of sample so it is representative of the population as a whole. Accurate comparisons of subgroups can be conducted. Cause of any observed differences cannot be determined. STA305 - week 1 8

9 Example Support for Firearms Registry Source of the example: The Gallup Poll November 27, This year according to a Gallop poll, 76% of Canadians support a national firearms registry, up slightly for 70% last year. 23% oppose registering all firearms with the federal government, and 2% hold no opinion on the question. The results are based on 1,011 telephone interviews with adults 18 or older, conducted on October 17-23, A sample of this size is accurate within a 3.1% margin of errors 19 times in 20. Only one factor was mentioned in this study age. But individuals cannot be randomly allocated to age. Was the sample randomly selected? STA305 - week 1 9

10 Terminology Experimental Units Experimental units are the material to which the levels of the treatment factor are applied. They are the smallest unit of experimental material for which two different units can receive different treatments. Example: if patients are randomly allocated to one of three possible treatments, then patients are the experimental units. Example: if two incentive pay systems are compared by randomly allocating 100 companies to each of the two incentive systems, then the experimental unit is a company. STA305 - week 1 10

11 Factor A factor is an independent variable to be studied in the investigation. It is similar to independent variable in a regression. Studies may include one or several factors. Example: in a study of the effect of price on volume of sales of a luxury item, the factor is price. Example: in a study of the effect of fertilizer on crop of a certain plant, the factor is fertilizer. STA305 - week 1 11

12 Factor Level A factor level is a particular value, or groups of values, of the factor. Example: if temperature is a factor, then 60, 70 and 80 could be levels. Example: in a study of milk consumption, age might be a factor with levels 0-6 years, 7-19 years, years and 51 years. Some factors are not continuous in nature, and can only take on fixed number of values. Example: when studying ulcer drugs, it might be important to treat blood group as factor with levels A, B, O and AB. STA305 - week 1 12

13 Treatment In a study that includes only one factor, the treatments are the factor levels. When a study includes two or more factors, a treatment is a combination of the factor levels. Example: if there are two factors, temperature and pressure, then a treatment is a fixed combination of temperature and pressure levels. STA305 - week 1 13

14 Response Response is the output or measurement to be observed after the treatments have been applied. It is similar to response variable in regression. In some case a response variable is called an outcome measure. STA305 - week 1 14

15 Randomization Introduction The use of randomization to allocate treatments to experimental units (or vice versa) is the key element of well-designed experiment. Random allocation tends to produce subgroups which are comparable with respect to the variables known to influence the response. Randomization ensures that no systematic or subjective bias is introduced in allocation of treatments to experimental units. Randomization reduces the possibility that factors not included in the design will be confounded with treatment. STA305 - week 1 15

16 Example Bicycle Handbrake Testing Bicycle manufacturer would like to test ease of use of 2 handbrake systems (A and B) A course is laid out and subjects recruited to drive through course twice each. At the end of each circuit, each cyclist will be asked to rate ease of use of the braking system. Design components: First time through the course, all rider use braking system A. Second time through the course, all riders use braking system B. Problems: STA305 - week 1 16

17 Example - Lanarkshire Milk Study Lanarkshire milk study is a well-known example of subjective bias introduced into allocation of treatment. Goal was to look for differences in weight gain among children fed with (1) un pasteurized milk (2) pasteurized milk or (3) no milk. The study was conducted through local school system. Initial allocation of children to treatment groups was based on last name. Where there appeared to be imbalance in the number of well-fed / ill-nourished children in a group, teacher allowed to make changes. At the end of the study, it appeared that the changes made by the teachers were influenced by the poverty of some students. No clear-cut conclusions were possible. STA305 - week 1 17

18 More about Bias Bias on the part of study participants or study evaluators can be dealt with by randomization if treatment can be concealed. People participating in clinical trial may experience a placebo effect; if they know they are receiving the new treatment, they might be more optimistic about outcome. Experimenters may show a bias in evaluating outcome if they know which treatment each person has received. In some cases it will not be possible to conceal the treatment. STA305 - week 1 18

19 Mechanics of Randomization There are few methods that could be used to carry out a randomization: - Coin toss or die toss - Computer algorithm (for example, PROC PLAN in SAS) - Random number table The example below illustrates how to use a random number table. The random number table can be found in Appendix A (pages ). STA305 - week 1 19

20 Example Using a Random Number Table Suppose the goal is to randomly assign 30 experimental units to treatment A and 30 to treatment B. Start by numbering experimental units from 01 to 60. Randomly select a starting point in the random number table. Divide the strings of digits into 2-digits numbers. If a 2-digit number is in 01-60, then assign that subject to treatment A. If a 2-digit number is greater than 60, is 00, or has already been drawn, then go on to the next number which is between 1 and 60. Repeat this process until 30 subjects have been selected to receive treatment A. Remaining subjects will receive treatment B. STA305 - week 1 20

21 Replication Important feature of a well-designed experiment is the use of replicates. Since there is variability between experimental units, several units should be studied. The use of replicates allows estimation of experimental error, which is used in determining whether differences are statistically significant. Replicates also allow precise estimation of other parameters of interest (e.g., group means). Note, if measurements are taken on the same experimental unit at different times then these measurements are not independent and are not considered replicates. They are called repeated measurements. STA305 - week 1 21

22 Checklist for Study Planning This material is covered in section 2.2, pages 8-14, of the text. 1. Define the objectives of the study. 2. Identify factors and factor levels and covariates to be included in the study. 3. Identify what are the experimental units. 4. Determine whether there will be any restrictions on randomization (such as blocking), and determine how randomization will be carried out. 5. Determine the most appropriate response measure. 6. Run a pilot experiment 7. Specify the model that will be used to analyze the data and create an analysis plan. 8. Determine the number of experimental units that will be included in the study. 9. Run the study. STA305 - week 1 22

23 Example Evaluation of a Bicycle Skills Training Program for Young Children Source of example: article in Injury Prevention 1998 The abstract of this journal article contains the following summary of the objectives and methods used in the study. Objective: To evaluate the effectiveness of a skills training program in improving safe cycling behavior, knowledge and attitudes in young children. STA305 - week 1 23

24 Methods: Grad 4 children from six elementary schools in East York participated. The intervention - playground based instruction on bicycle handling skills by certified instructors - was allocated to three schools. Altogether 141 children participated: 73 in the intervention group and 68 in the control group, with follow up evaluations available on 117 (83%). The primary outcome was safe cycling behavior (straight line riding, coming to a complete stop, and shoulder checking before a left turn). A self report questionnaire collected data on knowledge and attitudes. Baseline assessments were made in June, with follow up evaluation in September. STA305 - week 1 24

25 Information available in the abstract and other parts of this article can be used to fill in some parts of the study planning checklist. 1. Objective: Bicycle riding is a popular form of exercise, recreation, and transportation. Unfortunately, bike riding often results in serious injury. An instruction program was developed to teach safe bike riding practices to grad 4 students, and the goal of the study was to determine whether this was effective compared to no instructional intervention. 2. Factors: The primary factor of interest is intervention, and 2 levels were studied: instructional program and no instructional program. The study did not focus on schools but they need to be taken into account, since randomization was done at the school level. 3. Experimental Units: The experimental units are the individual grad 4 students enrolled at the 6 participating elementary schools. STA305 - week 1 25

26 4. Randomization: Schools were randomly allocated to either receive the instructional program or not. However, there are no details in the journal article concerning the manner in which this was done (roll a die? random number table? other?) 5. Response Measures: According to the abstract, there were several response measures: straight line riding, coming to a complete stop, shoulder checking before left turn, and responses to a questionnaire concerning knowledge and attitudes. 6. Model and Analysis: The authors stated that a two-sample t-test was used to analyze the data. However, the model is not specified in equation form in the article. STA305 - week 1 26

27 7. Sample Size: 6 schools and 117 Grade 4 students participated in the study. Prior to undertaking this study, the researchers determined the number of schools that would be required in order to meet the study objectives. This is a quote from the journal article regarding the sample size. Sample Size. A priori, the prevalence of safe cycling behavior in the control group was estimated to be 40%. Experimenters wished to have 80% power, at an alpha level of 5%, to detect a twofold difference in the prevalence of safe cycling behavior between the intervention and control groups. For a trial with randomization by individual, a sample size of around 23 per group would be required to detect this effect size. However, because randomization by cluster reduces efficiency, the sample size estimate was increased by a factor of 2.5 (to around 60 per group) based on Cornfields formula. STA305 - week 1 27

28 8. Analysis and Conclusions: The authors summarized the results of their analyses and concluded that the instructional program did not result in an increase in safe biking behavior. Here is a quote from the abstract regarding the results of the analyses and their conclusions. Results: The prevalence of safe cycling behaviors at follow up in the intervention and control groups respectively, were: straight line riding (90% v 88%; p=0.782), coming to a complete stop (90% v 76%; p=0.225), and shoulder checking (0% v 2%; p=1.000). Over time (from baseline to follow up) children in both groups were more likely to maintain straight line riding, less likely to ride on the sidewalk, and less likely to consider that a car had more right to the road. Conclusions: This brief skills training program was not effective in improving safe cycling behavior, knowledge, or attitudes among grade 4 children. STA305 - week 1 28

29 One Factor Experiments Introduction Studies that include only one factor are known as single factor or one factor experiments. At least two factor levels can be included in the study. Factor levels can be chosen according to specific interest. Alternatively, factor levels can be chosen at random from all possible factor levels. Random factor levels will be discussed later. Since there is only one factor, the treatments are the factor levels STA305 - week 1 29

30 Example Suppose that a study is being planned to determine the best amount of cotton to include in a fabric that will be used to make men s shirts. Fabric samples will include either 15% cotton, 20% cotton, or 25% cotton. This is a single factor experiment with the factor being the amount of cotton. There are 3 factor levels: 15%, 20%, 25%. STA305 - week 1 30

31 Notation Factor being considered will be referred to as factor A. Let a denote the number of levels of factor A. Randomly allocate r i experimental units to factor level i, i = 1, 2,, a. Let.. represent the total sample size. Let Y ij denote the response of the j-th experimental unit in the i-th treatment group, i = 1, 2,..., a, j = 1, 2,..., r i The layout of the data would be as follows: STA305 - week 1 31

32 Looking at Differences between Groups The goal of the experiment is to determine whether, on average, treatment groups differ with respect to outcome variable. The first step in answering this question is to look at the data via plots such as box and whisker plots side-by-side histograms Then we need to calculate descriptive statistics such as mean, median range, standard deviation sample quartiles STA305 - week 1 32