SAMPLE SURVEYS, SAMPLING TECHNIQUES, AND DESIGN OF EXPERIMENTS

Transcription

1 7258da_pA01.wiley 7/12/00 6:00 PM Page 682 A SAMPLE SURVEYS, SAMPLING TECHNIQUES, AND DESIGN OF EXPERIMENTS (The following content is not included in this text but is available for download on the Web site at A printed copy is available in the Instructor Solutions Manual and the Student Solutions Manual.) A.1 SOURCES OF DATA CASE STUDY A 1 IS IT A SIMPLE QUESTION? A.2 SAMPLE SURVEYS AND SAMPLING TECHNIQUES A.3 DESIGN OF EXPERIMENTS CASE STUDY A 2 SCIENTISTS PROBE MYSTERY OF THE PLACEBO EFFECT EXERCISES GLOSSARY COMPUTER ASSIGNMENTS

2 7258da_pA02.wiley 7/12/00 6:01 PM Page A1 A.1 SOURCES OF DATA The availability of accurate data is essential for deriving reliable results and making accurate decisions. As the truism garbage in, garbage out (GIGO) indicates, policy decisions based on the results of poor data may prove to be disastrous. Data sources can be divided into three categories: internal sources, external sources, and surveys and experiments. A.1.1 INTERNAL SOURCES Many times data come from internal sources, such as a company s own personnel files or accounting records. A company that wants to forecast the future sales of its products might use data from its own records for previous periods. A police department might use data that exist in its own records to analyze changes in the nature of crimes over a period of time. A.1.2 EXTERNAL SOURCES All needed data may not be available from internal sources. Hence, to obtain data we may have to depend on sources outside the company, called external sources. Data obtained from external sources may be primary or secondary data. Data obtained from the organization that originally collected them are called primary data. If we obtain data from the Bureau of Labor Statistics that were collected by this organization, then these are primary data. Data obtained from a source that did not originally collect them are called secondary data. For example, data originally collected by the Bureau of Labor Statistics and published in the Statistical Abstract of the United States are secondary data. A.1.3 SURVEYS AND EXPERIMENTS Sometimes the data we need may not be available from internal or external sources. In such cases, we may have to obtain data by conducting our own survey or experiment. Surveys In a survey we do not exercise any control over the factors when we collect information. SURVEY In a survey, data are collected from the members of a population or sample with no particular control over the factors that may affect the characteristic of interest or the results of the survey. For example, if we want to collect data on the money various families spent last month on clothes, we will ask each of the families included in the survey how much it spent last month on clothes. Then we will record this information. A survey may be a census or a sample survey. (i) Census A census includes every member of the population of interest, which is called the target population. CENSUS A survey that includes every member of the population is called a census. In practice, a census is rarely taken because it is very expensive and time consuming. Furthermore, in many cases it is impossible to identify each member of the target population. We discuss these reasons in more detail in Section A.2.1.

3 7258da_pA03.wiley 7/12/00 6:01 PM Page A2 (ii) Sample Survey Usually, to conduct research, we select a portion of the target population. This portion of the population is called a sample. Then we collect the required information from the elements included in the sample. SAMPLE SURVEY The technique of collecting information from a portion of the population is called a sample survey. A survey can be conducted by personal interviews, by telephone, or by mail. The personal interview technique has the advantages of a high response rate and a high quality of answers obtained. However, it is the most expensive and time consuming technique. The telephone survey also gives a high response rate. It is less expensive and less time consuming than personal interviews. Nonetheless, a problem with telephone surveys is that many people do not like to be called at home, and those who do not have a phone are left out of the survey. A survey conducted by mail is the least expensive method, but the response rate is usually very low. Many people included in such a survey do not return the questionnaires. Conducting a survey that gives accurate and reliable results is not an easy task. To quote Warren Mitofsky, Director of Elections and Surveys for CBS News, Any damn fool with 10 phones and a typewriter thinks he can conduct a poll. 1 Preparing a questionnaire is probably the most difficult part of a survey. The way a question is phrased can affect the results of the survey. Case Study A 1, which is excerpted from an article published in Psychology Today, shows that writing questions for a questionnaire is a much more complex task than is usually thought. CASE STUDY A 1 IS IT A SIMPLE QUESTION? Even the seemingly simplest of questions can yield complex answers. Do you own a car? asks Stanley Presser, a sociologist at the National Science Foundation in Washington, D.C. That sounds like an awfully simple question. But is it really? What does you mean? Suppose a wife is answering the poll, and the car is registered in her husband s name. How is she supposed to answer? What does own mean? What if the car is on a long-term lease? What does car mean? What if they have one of those new little vans, or a four-wheel-drive vehicle? My God, that sounds like a simple question! You can imagine how diverse the factors become in a more complicated one. Suppose, however, that the question about car ownership had been preceded by a series of related questions: Are you married? Does your spouse drive an automotive vehicle? Is it a car, a van or some other sort of vehicle? Is it leased, or does your spouse own it? Now about you do you own a car? Such a series of questions would serve to clarify the intended meaning of the one about car ownership. Source: Rich Jaroslovsky, What s on Your Mind, America? Psychology Today, July August 1988, Copyright 1988 Sussex Publishers, Inc. Reprinted with permission. Section A.2 discusses sample surveys and sampling techniques in detail. 1 The Numbers Racket: How Polls and Statistics Lie, U.S. News & World Report, July 11, 1988.

4 7258da_pA04.wiley 7/12/00 6:01 PM Page A3 Experiments In an experiment, we exercise control over some factors when we collect information. EXPERIMENT In an experiment, data are collected from members of a population or sample with some control over the factors that may affect the characteristic of interest or the results of the experiment. For example, how is a new drug to be tested to find out whether or not it cures a disease? It is done by designing an experiment in which the patients under study are divided into two groups as follows: 1. The treatment group the members of this group receive the actual drug. 2. The control group the members of this group do not receive the actual drug but are given a substitute (called a placebo) that appears to be the actual drug. The two groups are formed in such a way that the patients in one group are similar to the patients in the other group. This is done by making random assignments of patients to two groups. Neither the doctors nor the patients know to which group a patient belongs. Such an experiment is called a double-blind experiment. Then, after a comparison of the percentage of patients cured in each of the two groups, a decision is made about the effectiveness or noneffectiveness of the new drug. For more on experiments, refer to Section A.3 of this appendix on experimental design. A.2 SAMPLE SURVEYS AND SAMPLING TECHNIQUES In this section we discuss the reasons sample surveys are preferred over a census, a representative sample, random and nonrandom samples, sampling and nonsampling errors, and random sampling techniques. A.2.1 WHY SAMPLE? As mentioned in the previous section, most of the time surveys are conducted by using samples and not a census of the population. Three of the main reasons for conducting a sample survey instead of a census are listed next. Time In most cases, the size of the population is quite large. Consequently, conducting a census takes a long time, whereas a sample survey can be conducted very quickly. It is time consuming to interview or contact hundreds of thousands or even millions of members of a population. On the other hand, a survey of a sample of a few hundred elements may be completed in little time. In fact, because of the amount of time needed to conduct a census, by the time the census is completed the results may be obsolete. Cost The cost of collecting information from all members of a population may easily fall outside the limited budget of most, if not all, surveys. Consequently, to stay within the available resources, conducting a sample survey may be the best approach. Impossibility of Conducting a Census Sometimes it is impossible to conduct a census. First, it may not be possible to identify and access each member of the population. For example, if a researcher wants to conduct a survey about homeless peo-

5 7258da_pA05.wiley 7/12/00 6:02 PM Page A4 ple, it is not possible to locate each member of the population and include him or her in the survey. Second, sometimes conducting a survey means destroying the items included in the survey. For example, to estimate the mean life of lightbulbs would necessitate burning out all the bulbs included in the survey. The same is true about finding the average life of batteries. In such cases, only a portion of the population can be selected for the survey. A.2.2 RANDOM AND NONRANDOM SAMPLES Depending on how a sample is drawn, it may be a random sample or a nonrandom sample. RANDOM AND NONRANDOM SAMPLES A random sample is a sample drawn in such a way that each member of the population has some chance of being selected in the sample. In a nonrandom sample, some members of the population may not have any chance of being selected in the sample. Suppose we have a list of 100 students and we want to select 10 of them. If we write the names of all 100 students on pieces of paper, put them in a hat, mix them, and then draw 10 names, the result will be a random sample of 10 students. However, if we arrange the names of these 100 students alphabetically and pick the first 10 names, it will be a nonrandom sample because the students who are not among the first 10 have no chance of being selected in the sample. A random sample is usually a representative sample. Note that for a random sample, each member of the population may or may not have the same chance of being included in the sample. Four types of random samples are discussed in Section A.2.4 of this appendix. Two types of nonrandom samples are a convenience sample and a judgment sample. In a convenience sample, the most accessible members of the population are selected to obtain the results quickly. For example, an opinion poll may be conducted in a few hours by collecting information from certain shoppers at a single shopping mall. In a judgment sample, the members are selected from the population based on the judgment and prior knowledge of an expert. Although such a sample may happen to be a representative sample, the chances of it being so are small. If the population is large, it is not an easy task to select a representative sample based on judgment. The so-called pseudo polls are examples of nonrepresentative samples. For instance, a survey conducted by a magazine that includes only its own readers does not usually involve a representative sample. Similarly, a poll conducted by a television station giving two separate 900 telephone numbers for yes and no votes is not based on a representative sample. In these two examples, respondents will be only those people who read that magazine or watch that television station, who do not mind paying the postage and telephone charges, or who feel emotionally compelled to respond. To quote Larry King on this subject: All over the board... The 900 telephone number is very popular these days, but viewers should be warned that in the case of political polling it has absolutely no basis in fact. Poor people in the audience can t contribute to the survey, so it s faulty to begin with.... So next time you see a poll based on 900 numbers, treat it as some sort of middle-class amusement and forget about it. ( Larry King s People, News and Views, USA TODAY, July 17, Copyright 1989, USA TODAY. Reprinted with permission.) Another kind of sample is the quota sample. To draw such a sample we divide the target population into different subpopulations based on certain characteristics. Then a subsample is selected from each subpopulation in such a way that each subpopulation is represented in the sample in exactly the same proportion as in the target population. As an example of a quota sample, suppose we want to select a sample of 1000 persons from a city whose population has 48% men and 52% women. To select a quota sample, we choose 480 men from the male population and 520 women from the female population. The sample selected in this way will contain exactly 48% men and 52% women. Another way

6 7258da_pA06.wiley 7/12/00 6:02 PM Page A5 to select a quota sample is to select from the population one person at a time until we have exactly 480 men and 520 women. Until the 1948 presidential election in the United States, quota sampling was the most commonly used sampling procedure to conduct opinion polls. The voters included in the samples were selected in such a way that they represented the population proportions of voters based on age, sex, education, income, race, and so on. However, this procedure was abandoned after the 1948 presidential election in which the underdog, Harry Truman, defeated Thomas E. Dewey, who was heavily favored based on the opinion polls. First, the quota samples failed to be representative because the interviewers were allowed to fill their quotas by choosing voters based on their own judgments. This caused the selection of more upper-income and highly educated people, who happened to be Republicans. Thus, the quota samples were unrepresentative of the population because Republicans were overrepresented in these samples. Second, the results of the opinion polls based on quota sampling happened to be false because a large number of factors differentiate voters, and the pollsters considered only a few of those factors. A quota sample based on a few factors will skew the results. A random sample (that is not based on quotas) has a much better chance of being representative of the population of all voters than a quota sample based on a few factors. A.2.3 SAMPLING AND NONSAMPLING ERRORS The results obtained from a sample survey may contain two types of errors: sampling and nonsampling errors. The sampling error is also called the chance error, and nonsampling errors are also called the systematic errors. Sampling or Chance Error Usually, all samples taken from the same population will give different results because they contain different elements of the population. Moreover, the results obtained from any one sample will not be exactly the same as the ones obtained from a census. The difference between a sample result and the result we would have obtained by conducting a census is called the sampling error, assuming that the sample is random and no nonsampling error has been made. SAMPLING ERROR The sampling error is the difference between the result obtained from a sample survey and the result that would have been obtained if the whole population had been included in the survey. The sampling error occurs because of chance, and it cannot be avoided. A sampling error can occur only in a sample survey. It does not occur in a census. Sampling error is discussed in detail in Section 7.2 of Chapter 7, and an example of it is given there. Nonsampling or Systematic Errors Nonsampling errors can occur both in a sample survey and in a census. Such errors occur because of human mistakes and not chance. NONSAMPLING ERRORS The errors that occur in the collection, recording, and tabulation of data are called nonsampling errors. Nonsampling errors occur because of human mistakes and not chance. Nonsampling errors can be minimized if questions are prepared carefully and data are handled cautiously. Many types of systematic errors or biases can occur in a survey, including selection error, nonresponse error, response error, and voluntary response error. The following chart shows the types of errors.

7 7258da_pA07.wiley 7/12/00 6:02 PM Page A6 Types of errors Sampling or chance error Nonsampling or systematic errors Selection error Nonresponse error Response error Voluntary response error (i) Selection Error When we need to select a sample, we use a list of elements from which we draw a sample, and this list usually does not include many members of the target population. Most of the time it is not feasible to include every member of the target population in this list. This list of members of the population that is used to select a sample is called the sampling frame. For example, if we use a telephone directory to select a sample, the list of names that appears in this directory makes the sampling frame. In this case we will miss the people who are not listed in the telephone directory. The people we miss, for example, will be poor people (including homeless people) who do not have telephones and people who do not want to be listed in the directory. Thus, the sampling frame that is used to select a sample may not be representative of the population. This may cause the sample results to be different from the population results. The error that occurs because the sampling frame is not representative of the population is called the selection error. SELECTION ERROR The list of members of the target population that is used to select a sample is called the sampling frame. The error that occurs because the sampling frame is not representative of the population is called the selection error. If a sample is nonrandom (and, hence, nonrepresentative), the sample results may be quite different from the census results. (ii) Nonresponse Error Even if our sampling frame and, consequently, the sample are representative of the population, nonresponse error may occur because many of the people included in the sample did not respond to the survey. NONRESPONSE ERROR The error that occurs because many of the people included in the sample do not respond to a survey is called the nonresponse error. This type of error occurs especially when a survey is conducted by mail. A lot of people do not return the questionnaires. It has been observed that families with low and high incomes do not respond to surveys by mail. Consequently, such surveys overrepresent middle-income families. This kind of error occurs in other types of surveys, too. For instance, in a face-to-face survey where the interviewer interviews people in their homes, many people may not be home when the interviewer visits their homes. The people who are home at the time the interviewer visits and the ones who are not home at that time may differ in many respects, causing a bias in the survey results. This kind of error may also occur in a telephone survey. Many people may not be home when the interviewer calls. This may dis-

8 7258da_pA08.wiley 7/12/00 6:05 PM Page A7 tort the results. To avoid the nonresponse error, every effort should be made to contact all people included in the survey. (iii) Response Error The response error occurs when the answer given by a person included in the survey is not correct. This may happen for many reasons. One reason is that the respondent may not have understood the question. Thus, the wording of the question may have caused the respondent to answer incorrectly. It has been observed that when the same question is worded differently, many people do not respond the same way. Usually such an error on the part of respondents is not intentional. RESPONSE ERROR The response error occurs when people included in the survey do not provide correct answers. Sometimes the respondents do not want to give correct information when answering a question. For example, many respondents will not disclose their true incomes on questionnaires or in interviews. When information on income is provided, it is almost always biased in the upward direction. Sometimes the race of the interviewer may affect the answers of respondents. This is especially true if the questions asked are about race relations. The answers given by respondents will differ depending on whether the interviewer is white or nonwhite. (iv) Voluntary Response Error Another source of systematic error is a survey based on a voluntary response sample. VOLUNTARY RESPONSE ERROR Voluntary response error occurs when a survey is not conducted on a randomly selected sample but a questionnaire is published in a magazine or newspaper and people are invited to respond to that questionnaire. The polls conducted based on samples of readers of magazines and newspapers suffer from voluntary response error or bias. Usually only those readers who have very strong opinions about the issues involved respond to such surveys. Surveys in which the respondents are required to call 900 telephone numbers also suffer from this type of error. Here, in order to participate, a respondent must pay for the call, and many people do not want to bear this cost. Consequently, the sample is usually neither random nor representative of the target population because participation is voluntary. A.2.4 RANDOM SAMPLING TECHNIQUES There are many ways to select a random sample. Four of these techniques are discussed next. Simple Random Sampling A sample that assigns the same probability of being selected to each member of the population is called a simple random sample. SIMPLE RANDOM SAMPLE A simple random sample is a sample that is selected in such a way that each member of the population has the same chance of being included in the sample. One way to select a simple random sample is by a lottery or drawing. For example, if we need to select five students from a class of 50, we write each of the 50 names on a separate piece of paper.

9 7258da_pA09.wiley 7/12/00 6:05 PM Page A8 Then, we place all 50 names in a hat and mix them thoroughly. Next, we draw one name randomly from the hat. We repeat this experiment four more times. The five drawn names make up a simple random sample. The second procedure to select a simple random sample is to use a table of random numbers. Table I in Appendix D lists random numbers generated by a random process. Suppose we have a group of 400 persons and we need to select 30 persons randomly from this group. To select a simple random sample, we arrange the names of all 400 persons in alphabetic order and assign a three-digit number, from 001 to 400, to each person. Next, we use the table of random numbers to select 30 persons. The random numbers in Table I are recorded in blocks of five digits. To use this table, we can start anywhere. One way is to close our eyes and put a finger anywhere on the page and start at that point. From there, we can move in any direction. We need to pick three-digit numbers from the table because we have assigned three-digit numbers to the 400 persons in our population. Suppose we start at the first block of the 31st row from the top of Table I. The five rows starting with the 31st row from that table are reproduced as Table A.1 here. The first block of five numbers in Table A.1 is We use the first three digits of this block to select the first person from the population. Hence, the first person selected is the one with the number 130. Suppose we move along the row to the right to make the next selection. The second block of five numbers in Table A.1 is The first three digits of this block give 852. However, we have only 400 persons in the population with assigned numbers of 001 to 400. Consequently, we cannot use 852 to select a person. Therefore, we move to the next block of five numbers without making a selection. The third block of numbers is The first three digits of this block are 327. Consequently, the second person selected is the one with the number 327. We continue this process until all 30 required persons are selected. This gives us a simple random sample of 30 persons. Table A.1 Random Numbers Although the table of random numbers given in Appendix D contains only 1485 blocks of fivedigit numbers, we can easily construct a table of as many random numbers as we want using a computer software package such as MINITAB. If we have access to a computer, we can use a statistical package, such as MINITAB, to select a simple random sample. See Section B.14 of Appendix B for explanations and illustrations of how we can draw such a sample by using MINITAB. Systematic Random Sampling The simple random sampling procedure becomes very tedious if the size of the population is large. For example, if we need to select 150 households from a list of 45,000, it is very time consuming either to write the 45,000 names on pieces of paper and then select 150 households or to assign a five-digit number to each of the 45,000 households and then select 150 households using the table of random numbers. In such cases, it is more convenient to use systematic random sampling. The procedure to select a systematic random sample is as follows. In the example just mentioned, we would arrange all 45,000 households alphabetically (or based on some other characteristic). Since the sample size should equal 150, the ratio of population to sample size is 45,000/ Using this ratio, we randomly select one household from the first 300 households in the arranged list either by using the lottery system or by using a table of random numbers. Suppose by using either of these methods, we select the 210th household. We then select every 210th household from every 300 households in the list. In other words, our sample includes the households with numbers 210, 510, 810, 1110, 1410, 1710, and so on.

10 7258da_pA10.wiley 7/12/00 6:05 PM Page A9 SYSTEMATIC RANDOM SAMPLE In systematic random sampling, we first randomly select one member from the first k units. Then every kth member, starting with the first selected member, is included in the sample. Note that systematic random sampling does not give a simple random sample because we cannot select two adjacent elements. Hence, every member of the population does not have the same probability of being selected. Stratified Random Sampling Suppose we need to select a sample from the population of a city and we want households with different income levels to be equally represented in the sample. In this case, instead of selecting a simple random sample or a systematic random sample, we may prefer to apply a different technique. First, we divide the whole population into different groups based on income levels. For example, we may form three groups of low-, medium-, and high-income households. We will now have three subpopulations, which are usually called strata. We then select one sample from each subpopulation or stratum. The collection of all three samples selected from three strata gives the required sample, called the stratified random sample. Usually, the sizes of the samples selected from different strata are proportionate to the sizes of the subpopulations in these strata. Note that the elements of each stratum are identical with regard to the possession of a characteristic. STRATIFIED RANDOM SAMPLE In a stratified random sample, we first divide the population into subpopulations, which are called strata. Then, one sample is selected from each of these strata. The collection of all samples from all strata gives the stratified random sample. Thus, whenever we observe that a population differs widely in the possession of a characteristic, we may prefer to divide it into different strata and then select one sample from each stratum. We can divide the population on the basis of any characteristic, such as income, expenditure, sex, education, race, employment, or family size. Cluster Sampling Sometimes the target population is scattered over a wide geographical area. Consequently, if a simple random sample is selected, it may be costly to contact each member of the sample. In such a case, we divide the population into different geographical groups or clusters and as a first step select a random sample of certain clusters from all clusters. We then take a random sample of certain elements from each selected cluster. For example, suppose we are to conduct a survey of households in the state of New York. First, we divide the whole state of New York into, say, 40 regions, which are called clusters or primary units. We make sure that all clusters are similar and, hence, representative of the population. We then select at random, say, 5 clusters from 40. Next, we randomly select certain households from each of these 5 clusters and conduct a survey of these selected households. This is called cluster sampling. Note that all clusters must be representative of the population. CLUSTER SAMPLING In cluster sampling, the whole population is first divided into (geographical) groups called clusters. Each cluster is representative of the population. Then a random sample of clusters is selected. Finally, a random sample of elements from each of the selected clusters is selected.

11 7258da_pA11.wiley 7/12/00 6:05 PM Page A10 A.3 DESIGN OF EXPERIMENTS As mentioned earlier, to use statistical methods to make decisions, we need access to data. Consider the following examples about decision making. 1. A government agency wants to find the average income of households in the United States. 2. A company wants to find the percentage of defective items produced on a machine. 3. A researcher wants to know if there is an association between eating unhealthy food and cholesterol level. 4. A pharmaceutical company has invented a new medicine for a disease and it wants to check if this medicine cures the disease. All of these cases relate to decision making. We cannot reach a conclusion in these examples unless we have access to data. Data can be obtained from observational studies, experiments, or surveys. This section is devoted mainly to controlled experiments. However, it also explains observational studies and how they differ from surveys. Suppose two diets, Diet 1 and Diet 2, are being promoted by two different companies, and each of these companies claims that its diet is successful in reducing weight. A research nutritionist wants to compare these diets with regard to their effectiveness for losing weight. Following are the two alternatives for the researcher to conduct this research. 1. The researcher contacts the persons who are using these diets and collects information on their weight loss. The researcher may contact as many persons as she has the time and financial resources for. Based on this information, the researcher makes a decision about the comparative effectiveness of these diets. 2. The researcher selects a sample of persons who want to lose weight, divides them randomly into two groups, and assigns each group to one of the two diets. Then she compares these two groups with regard to the effectiveness of these diets. The first alternative is an example of an observational study, and the second is an example of a controlled experiment. TREATMENT A condition (or a set of conditions) that is imposed on a group of elements by the experimenter is called a treatment. In an observational study the investigator does not impose a treatment on subjects or elements included in the study. For instance, in the first alternative mentioned above, the researcher simply collects information from the persons who are currently using these diets. In this case, the persons were not assigned to the two diets at random; instead, they chose the diets voluntarily. In this situation the researcher s conclusion about the comparative effectiveness of the two diets may not be valid because the effects of the diets will be confounded with many other factors or variables. When the effects of one factor cannot be separated from the effects of some other factors, the effects are said to be confounded. The persons who chose Diet 1 may be completely different with regard to age, gender, and eating and exercise habits from the persons who chose Diet 2. Thus, the weight loss may not be due entirely to the diet but to other factors or variables as well. Persons in one group may aggressively manage both diet and exercise, for example, whereas persons in the second group may depend entirely on diet. Thus, the effects of these other variables will get mixed up (confounded) with the effect of the diets. Under the second alternative, the researcher selects a group of people, say 100, and randomly assigns them to two diets. One way to make random assignments is to write the name of each of these persons on a piece of paper, put them in a hat, and then randomly draw 50 names from this hat. These 50 persons will be assigned to one of the two diets, say Diet 1. The remaining 50 persons will be assigned to the second diet, Diet 2. This procedure is called randomization. Note that random assignments can also be made by using the table of random numbers.

12 7258da_pA12.wiley 7/12/00 6:06 PM Page A11 RANDOMIZATION The procedure in which elements are assigned to different groups at random is called randomization. When people are assigned to one or the other of two diets at random, the other differences among people in the two groups almost disappear. In this case these groups will not differ very much with regard to such factors as age, gender, and eating and exercise habits. The two groups will be very similar to each other. By using the random process to assign people to one or the other of two diets, we have controlled the other factors that can affect the weights of people. Consequently, this is an example of a designed experiment. As mentioned earlier, a condition (or a set of conditions) that is imposed on a group of elements by the experimenter is called a treatment. In the example on diets, each of the two diet types is called a treatment. The experimenter randomly assigns the elements to these two treatments. Again, in such cases the study is called a designed experiment. DESIGNED EXPERIMENT AND OBSERVATIONAL STUDY When the experimenter controls the assignment of elements to different treatment groups, the study is said to be a designed experiment. In contrast, in an observational study the assignment of elements to different treatments is voluntary and the experimenter simply observes the results of the study. The group of people who receive a treatment is called the treatment group, and the group of people who do not receive a treatment is called the control group. In our example on diets, both groups are treatment groups because each group is assigned to one of the two types of diet. That example does not contain a control group. TREATMENT AND CONTROL GROUPS The group of elements that receives a treatment is called the treatment group, and the group of elements that does not receive a treatment is called the control group. An example of an observational study. An example of a designed experiment. EXAMPLE A 1 Suppose a pharmaceutical company has invented a new medicine to cure a disease. To see whether or not this medicine is effective in curing this disease, it will have to be tested on a group of humans. Suppose there are 100 persons who have this disease; 50 of them voluntarily decide to take this medicine and the remaining 50 decide not to take it. The researcher then compares the cure rates for the two groups of patients. Is this an example of a designed experiment or an observational study? Solution This is an example of an observational study because 50 patients voluntarily joined the treatment group; they were not randomly selected. In this case, the results of the study may not be valid because the effects of the medicine will be confounded with other variables. All of the patients who decided to take the medicine may not be similar to the ones who decided not to take it. It is possible that the persons who decided to take the medicine are in the advanced stages of the disease. Consequently, they do not have much to lose by being in the treatment group. The patients in the two groups may also differ with regard to other factors such as age, gender, and so on. EXAMPLE A 2 Reconsider Example A 1. Now, suppose that out of the 100 people who have this disease, 50 are selected at random. These 50 people comprise one group, and the remaining 50 belong to the second group. One of these groups is the treatment group, and the second is the control group. The researcher then compares the cure rates for the two groups of patients. Is this an example of a designed experiment or an observational study?

13 7258da_pA13.wiley 7/12/00 6:08 PM Page A12 Solution In this case, the two groups will be very similar to each other. Note that we do not expect the two groups to be exactly identical. However, when randomization is used, the two groups will be very close to exactly similar. After these two groups have been formed, one group will be given the actual medicine. This group is called the treatment group. The other group will be administered a placebo (a dummy medicine that looks exactly like the actual medicine). This group is called the control group. This is an example of a designed experiment because the patients are assigned to one of two groups the treatment or the control group randomly. Usually in an experiment like the one described in Example A 2, patients do not know which group they belong to. Most of the time even the experimenters do not know which group a patient belongs to. This is done to avoid any bias or distortion in the results of the experiment. When neither patients nor experimenters know who is taking the real medicine and who is taking the placebo, it is called a double-blind experiment. For the results of the study to be unbiased and valid, an experiment must be a double-blind designed experiment. Note that if either experimenters or patients or both have access to information regarding which patients belong to treatment or control groups, it will no longer be a double-blind experiment. The use of placebos in medical experiments is very important. A placebo is just a dummy pill that looks exactly like the real medicine. Often, patients respond to any kind of medicine. Many studies have shown that even when the patients were given sugar pills (and patients did not know it), many of them indicated a decrease in pain. Patients respond to placebos because they have confidence in their physicians and medicines. This is called the placebo effect. Note that there can be more than two groups of elements in an experiment. For example, an investigator may need to compare three diets with regard to weight gain for chickens. Here, in a designed experiment, the chickens will be randomly assigned to one of the three diets, which are the three treatments. In some instances we have to base our research on observational studies because it is not feasible to conduct a designed experiment. For example, suppose a researcher wants to compare the starting salaries of business and psychology majors. The researcher will have to depend on an observational study. She will select two samples, one of recent business majors and another of recent psychology majors. Based on the starting salaries of these two groups, the researcher will make a decision. Note that, here, the effects of the majors on the starting salaries of the two groups of graduates will be confounded with other variables. One of these other factors is that the business and psychology majors may be different in regard to intelligence level, which may affect their salaries. However, the researcher cannot conduct a designed experiment in this case. She cannot select a group of persons randomly and ask them to major in business and select another group and ask them to major in psychology. Instead, persons voluntarily choose their majors. In a survey we do not exercise any control over the factors when we collect information. This characteristic of a survey makes it very close to an observational study. However, a survey may be based on a probability sample, which differentiates it from an observational study. If an observational study or a survey indicates that two variables are related, it does not mean that there is a cause-and-effect relationship between them. For example, if an economist takes a sample of families, collects data on the incomes and rents paid by these families, and establishes an association between these two variables, it does not necessarily mean that families with higher incomes pay higher rents. Here the effects of many variables on rents are confounded. A family may pay a higher rent not because of higher income but because of various other factors, such as family size, preferences, place of residence, and so on. We cannot make a statement about the cause-and-effect relationship between incomes and rents paid by families unless we control for these other variables. The association between incomes and rents paid by families may fit any of the following three scenarios. 1. These two variables have a cause-and-effect relationship. Families that have higher incomes do pay higher rents. A change in incomes of families causes a change in rents paid. 2. The incomes and rents paid by families do not have a cause-and-effect relationship. Both of these variables have a cause-and-effect relationship with a third variable. Whenever that third variable changes, these two variables change.

14 7258da_pA14.wiley 7/12/00 6:08 PM Page A13 3. The effect of income on rent is confounded with other variables, and this indicates that income affects rent paid by families. If our purpose in a study is to establish a cause-and-effect relationship between two variables, we must control the effects of other variables. In other words, we must conduct a designed study. CASE STUDY A 2 SCIENTISTS PROBE MYSTERY OF THE PLACEBO EFFECT So you re feeling depressed. Or maybe you re worried about that pain in your chest. Perhaps your asthma is acting up. Well, read this column every word beginning to end. You ll feel better. I guarantee it. You don t believe that? Pity. If you did, you d probably feel demonstrably better. You might even be cured. It s not magic. It s the placebo effect the mysterious ability of our bodies to sometimes heal what ails us, if only we believe. Placebo in Latin means I shall please. In medical research, it refers to a pharmacologically inactive substance like a sugar pill or a sham medical procedure that is administered as a control in testing the effectiveness of a drug or course of action. Walter Brown, clinical professor of psychiatry at Brown University, is at the forefront of research into the placebo effect. He and others are trying to learn why about 30 percent to 40 percent of the people who suffer from conditions ranging from asthma to high blood pressure to depression actually benefit from taking a placebo. Make no mistake: The healing effects of placebos are real and not merely delusions or wishful thinking, Brown and other researchers contend. It s easy to document and prove the effectiveness of placebos, Brown asserts. In one study, for example, researchers in a New York hospital told asthma sufferers that they were going to use an inhalant that would open their airways. That s exactly what happened when these asthmatics used the spray, even though the inhaler contained a placebo. And when another group of 40 asthmatics was warned that the medication in their inhalers would constrict their airways, the treatment did just that for half of the test subjects. Twelve sufferers even had asthma attacks, which were reversed when they inhaled another placebo, Brown said. Test subjects in another study were led to believe that they were consuming an alcoholic beverage when the drink actually was alcohol-free. You guessed it: Depending on how much they drank, their speech became slurred and they showed other telltale signs of tipsiness, Brown said. Others were told their placebo drink contained caffeine. These people showed improved reaction time in lab tests similar to the psychological effects of real caffeine. Perhaps the most startling clinical evidence of the placebo effect is one of the earliest. In the late 1950s, a group of surgeons led by Edmunds G. Dimond of the University of Kansas Medical Center performed a then common surgical procedure on 13 patients to treat angina pectoris, chest pain caused by insufficient blood supply to the heart. On five other patients with the disease, the doctors made only a superficial chest incision but performed no surgery on them. Ten of the 13 patients who underwent actual surgery got better but so did all five of those who were not operated on. (The surgical procedure is no longer done.) Why do placebos work? Nobody knows for sure, Brown said. Researchers suspect that the expectation of relief lowers stress and anxiety, which doctors know not only make you sick, but also are powerful roadblocks to healing. There may be a chemical explanation, as

15 7258da_pA15.wiley 7/12/00 6:08 PM Page A14 well, Brown said: Placebos may trigger the brain to produce endorphins, a powerful natural pain reliever. (The placebo effect also may explain why unorthodox cures such as healing crystals, therapeutic touch, colonic irrigation, and other forms of alternative medicine appear to help some people.) Studies have even found that placebos often behave very much like real drugs sometimes in surprising ways. Test subjects have developed tolerances for the sham medications and required progressively larger doses of the placebo to achieve relief. Others have experienced negative reactions after overdosing on harmless placebos they thought were potent drugs. Some even demonstrated signs of dependence, becoming psychologically addicted, reported physician Richard Letvak in the journal Patient Care. A recent British study revealed that even words can be effective placebos. Half of a test group of 200 patients who had no identifiable disease but complained of pain were told by doctors that they would probably get better in two weeks. Two weeks later, 64 percent of them had. Doctors told patients in the control group that they did not know what was wrong with them, and only 39 percent of these patients recovered, Brown reported last month in the journal Hospital Practice. Source: Richard Morin, Scientists Probe Mystery of the Placebo Effect. Universal Press Syndicate. The Hartford Courant, August 29, Copyright Universal Press Syndicate Reproduced with permission. EXERCISES A.1 Briefly describe the various sources of data. A.2 What is the difference between internal and external sources of data? Explain. A.3 Explain the difference between a sample survey and a census. Why is a sample survey usually preferred over a census? A.4 What is the difference between a survey and an experiment? Explain. A.5 Explain the following. a. Random sample b. Nonrandom sample c. Convenience sample d. Judgment sample e. Quota sample A.6 Explain briefly the following four sampling techniques. a. Simple random sampling b. Systematic random sampling c. Stratified random sampling d. Cluster sampling A.7 In which sampling technique do all elements of a population have the same chance of being selected in a sample? A.8 A statistics professor wanted to find out the average GPA (grade point average) for all students at her university. She used all students enrolled in her statistics class as a sample and collected information on their GPAs to find the average GPA. a. Is this sample a random or a nonrandom sample? Explain. b. What kind of sample is it? In other words, is it a simple random sample, a systematic sample, a stratified sample, a cluster sample, a convenience sample, a judgment sample, or a quota sample? Explain. c. What kind of systematic error, if any, will be made with this kind of sample? Explain. A.9 A professor wanted to select 20 students from his class of 300 students to collect detailed information on the profiles of his students. He used his knowledge and expertise to select these 20 students. a. Is this sample a random or a nonrandom sample? Explain. b. What kind of sample is it? In other words, is it a simple random sample, a systematic sample, a stratified sample, a cluster sample, a convenience sample, a judgment sample, or a quota sample? Explain. c. What kind of systematic error, if any, will be made with this kind of sample? Explain.

16 7258da_pA16.wiley 7/12/00 6:09 PM Page A15 A.10 Refer to Exercise A.8. Suppose the professor obtains a list of all students enrolled at the university from the registrar s office and then selects 150 students at random from this list using a statistical software package such as MINITAB. a. Is this sample a random or a nonrandom sample? Explain. b. What kind of sample is it? In other words, is it a simple random sample, a systematic sample, a stratified sample, a cluster sample, a convenience sample, a judgment sample, or a quota sample? Explain. c. Do you think any systematic error will be made in this case? Explain. A.11 Refer to Exercise A.9. Suppose the professor enters the names of all students enrolled in his class on a computer. He then selects a sample of 20 students at random using a statistical software package such as MINITAB. a. Is this sample a random or a nonrandom sample? Explain. b. What kind of sample is it? In other words, is it a simple random sample, a systematic sample, a stratified sample, a cluster sample, a convenience sample, a judgment sample, or a quota sample? Explain. c. Do you think any systematic error will be made in this case? Explain. A.12 A company has 1000 employees, of whom 58% are men and 42% are women. The research department at the company wanted to conduct a quick survey by selecting a sample of 50 employees and asking them about their opinions on an issue. They divided the population of employees into two groups, men and women, and then selected 29 men and 21 women from these respective groups. The interviewers were free to choose any 29 men and 21 women they wanted. What kind of sample is it? Explain. A.13 Many magazines regularly publish questionnaires and ask their readers to send their responses by mail. The tallied answers are published in the magazine at a later date. What kind of systematic error, if any, is made with this kind of a sample survey? Explain. A.14 A researcher wanted to conduct a survey of major companies to find out what benefits are offered to their employees. She mailed questionnaires to 2500 companies and received questionnaires back from 493 companies. What kind of systematic error does this survey suffer from? Explain. A.15 An opinion poll agency conducted a survey based on a random sample in which the interviewers called the parents included in the sample and asked them the following questions: i. Do you believe in spanking children? ii. Have you ever spanked your children? iii. If the answer to the second question is yes, how often? What kind of systematic error, if any, does this survey suffer from? Explain. A.16 A survey, based on a random sample taken from a borough of New York City, showed that 65% of the people living there would prefer to live somewhere other than New York City if they had the opportunity to do so. Based on this result, can the researcher say that 65% of people living in New York City would prefer to live somewhere else if they had the opportunity to do so? Explain. A.17 A study conducted by the Steering Committee of the Physicians Health Study Research Group in 1988 showed that taking one adult-size aspirin every other day reduces the risk of heart attack. A group of physicians at a research university wanted to investigate whether or not this claim is true. They chose 500 volunteers who offered to be included in the study. Of these 500 persons, 300 volunteered to take one adultsize aspirin every other day. The remaining 200 made up the control group. After two years the physicians compared the heart attack rates for the two groups. a. Is this an observational study or a designed experiment? Explain. b. Is this study a double-blind study? Explain. A.18 Refer to Exercise A.17. Suppose the group of physicians randomly selected 500 persons to be included in the study. Then, of these 500 persons, 300 were randomly selected to make up the treatment group. The remaining 200 made up the control group. The persons in the treatment group were given one adult-size aspirin every other day, and the persons in the control group were given a placebo. The patients did not know what group they belonged to but doctors had access to this information. a. Is this an observational study or a designed experiment? Explain. b. Is this study a double-blind study? Explain. A.19 Refer to Exercise A.18. Now suppose that neither patients nor doctors knew what group patients belonged to.