Chapter Three: Sampling Methods The idea of this chapter is to make sure that you address sampling issues - even though you may be conducting an action research project and your sample is "defined" by your classroom. This chapter should address the larger issues involved, especially in terms of the impact of sampling on research designs and outcomes. Consequently, the sections will focus on the definition of a sample, the strategies used in obtaining samples, as well as differences in perspectives based on research methodology. The discussion for this chapter should provide the reader with a clear understanding of sampling characteristics and techniques. 1. Samples versus Populations A population is composed of the entire group of people that could possibly be included in your study. A sample is a subgroup of individuals selected from that population. Unless the population is small, when you conduct your research you could not possibly study every individual within the potential study population, so you study a subgroup or sample. As researchers choose a sample for study, they need to make sure that the sample is representative of the larger population. When there is a representative sample, the researcher will be able to generalize to the population. Sampling can save time and money! After research is conducted and researchers determine characteristics of the sample, then generalizations can be made about the entire population. (Johnson & Christensen, p.222) It is unrealistic to expect an entire population to participate in a study (unless the population is extremely small) therefore sampling is an accepted alternative. Even when researchers intend to study an entire population, this may not be possible because of people who refuse to participate or because of others who cannot be contacted. Again, a sample is what is needed for the study. 1.1 Population Generalization The main purpose of sampling in quantitative research is so that researchers can generalize about the population. This means that statements about the population can be made based on a study that has been conducted. Researchers must be sure to include a large enough sample before drawing any generalizations. If you choose your sample with an EPSEM method (equal probability sampling method) your sample will have the same characteristics as your population and you can generalize your research findings to the whole population. Your sample is considered a representative sample because it resembles the larger population. An EPSEM is a sampling method in which every member of the population has an equal probability of being included in the study. Any differences between the population and the sample are therefore based on chance, not on researcher bias. This method gives the strongest research design for experimental research. When you have a sample that resembles a population, you can make generalizations from your study of the sample and therefore apply your findings to the larger population. 1.2 Ecological Generalization Ecological Generalization relates to the ability to generalize results from a study to settings and conditions outside of the study. This is very different than population generalizations which apply the results to individuals in the population from which the sample was obtained. In many studies both ecological and population generalizations can apply. Ecological generalizations are most valid when a connection can be made between the study and the new setting. When using sampling, the ultimate goal is to be able to apply the information to other populations. Therefore it is important to note the ecological factors surrounding the study. Many times this information becomes more clear as you read the description of the participants used in the sample. For example, if a study is conducted using low socioeconomic students you may need to ask if they are being studied in an urban or rural setting before being able to apply the findings to your specific classroom. Also, you might look for other influences in the study
environment. These may come from qualitative research supporting any conclusions. For instance, if a classroom of students is being researched with regard to their language acquisition while transitioning to English the researcher may want to examine the applicable information on how the general population surrounding the sample group supports or detracts from this endeavor. In order to make solid ecological generalizations you must examine the conditions surrounding the sampled group. 2. Sampling Strategies Sampling strategies, or selecting the sampling groups, involves the researcher choosing who will participate in the study. This involves careful thought, taking into consideration relevancy between the issue or phenomena under study and the group chosen to be observed. To do this a researcher develops a set of criteria that defines and sets boundaries between who should and should not be selected. Should only boys or only girls be included in the sample? Should the sample include only female Republicans or a mix of both male and female, Democrats and Republicans? Both random and non-random strategies can be used to select the participants in a study. 2.1 Random sampling Random sampling techniques are based on the theory of probability and usually produce good samples [because it] is representative of the population that [is being studied]. (Johnson & Christensen, p.223) In random sampling an equal opportunity is given to all members of a population to be represented. For larger studies, a simple random sample can be accomplished by using a table of random numbers. This computer generated table incorporates no systematic patterns. This method selects participants by matching their number to one from the random number table. Another strategy, systematic sampling, simplifies the random selection process by using a sampling interval. From a random starting point, every kth element in the sample is selected using this process. (k=population size divided by number of participants required.) Care must be exercised in this process to assure that the sampling intervals don't incorporate bias by following an inadvertent cyclical pattern. Another method of random sampling selection is the stratified sampling technique. The stratified method includes preselecting which portion of an entire population is to be studied (i.e. males or females) then, using the table of random numbers or sampling interval, selecting from within this strata of the population. If the samples are selected to parallel the proportions within the total population, this is called proportional stratified sampling. If a researcher wants to purposely select to study a larger proportion than that represented within a population, the method is called disproportional stratified sampling. A caution in using this strategy is that inferences made back to the population as a whole need to be weighted in order to correlate back to the actual proportions. Cluster sampling can also be used to select random participants not as individuals, but as groups like schools, neighborhoods, city blocks, churches, etc. Both one-stage and two-stage cluster sampling can be used. One-stage uses either simple random, systematic, or stratified random sampling to select the participants. Two-stage sampling just takes this process a step further, and again selects from within the first group to obtain the final group to study. To further refine this process, probability proportional to size is used. A random number of individuals from each cluster, large or small, is selected to represent participants from all aspects of the groups under study. 2.2 Non-random sampling One of the most common methods of sampling is a non-random method called convenience sampling. Through this method, one uses the most easily obtained participants or cases for the study. Although this method is nonrandom, it is still commonly used in empirical research. The way this is done is to obtain participants that are
conveniently found. Then, those participants are randomly assigned to two groups one is your control group and one is your treatment group. You carry out your study and compare the outcomes of the two groups. If you have controlled for extraneous variables, the differences in outcomes can be said to be caused by the treatment (for example, a different teaching technique). Although your participants may not be reflective of the entire population about which you might like to make generalizations, this can be improved by further studies with other groups in other locations. Almost all studies done within a classroom are non-random samples. We know that math is a universal language. Last year, at our school, the transitional students scored higher than regular students in that area. In writing, scores were comparably the same, but in reading, transitional students did not do as well as regular. Why is it that they did well in writing but not reading? They all received the same instruction at their level from all subjects in English. It seems likely that transitional students used what they knew in Spanish writing to translate to English but when it came to reading they had difficulties with vocabulary and comprehension so that translation was not possible.a year before, transitional students did not do well in the reading and writing TAKS. Is it because in that year they were segregated from English speaking classmates? This is an example of purposive sampling, where the sample has been chosen because they have the characteristics that the researcher wishes to study. The researcher is comparing different sets of students but they are all transitional students taking the TAKS in English. Under one condition, the students took the English TAKS after spending the year in class with English speaking classmates. Under another condition, the students took the English TAKS after spending a year in class only with other transitional students. These two conditions had two different outcomes. The question is whether this year the students being segregated from English speaking students will score poorly in reading, as the previous segregated group did. These groups are non-random samples so the ability to make generalizations is limited. However, as more data is collected with more groups, the ability to generalize and predict outcomes becomes more possible. Doing this kind of thinking about instructional conditions and learning outcomes can add to our knowledge base if we share our knowledge with others. It can help us as educators to examine different learning conditions and apply them to help our own students succeed. This is type of action research can improve both teaching and learning. This example also brings up a problem with sampling. There are so many extraneous variables when testing in schools, including quality of instruction and testing conditions that even if the hypothesis is proved to be true, can it be repeated? That is the real question with educational research. What happens if the scores this year are comparable with 2 years ago? What happens if the students who are segregated from English speaking classmates this year score higher than said classmates? 2.3 Qualitative sampling In qualitative research the decision of whom or what is going to be studied is based on a defined set of criteria or standards that the sample group must have. These criteria make a distinction between the potential candidates that are going to be included in the study from those who are not going to be part of it. These attributes are referred to as inclusion boundaries. Once these are determined, the researcher can start the process of selecting the sample. The sampling strategy used in qualitative research is known as criterion-based selection or purposeful sampling. These terms are used interchangeably and define the population or cases that meet the criteria set for the sample and consequently the purpose of the research. Different factors can affect the selection of the sample, for instance availability of the potential participants, and the costs of the logistics of finding and recruiting them. All these variables have to be taken into account in order to select the best candidates as well as meeting the cost constraints established for the study. As an addition to the information located above, qualitative sampling contains about nine different sampling types. A shorten recap of the nine sampling types are the following: Comprehensive Sampling: This type includes everything in the case to be researched. This type of sampling is
expensive and not recommended unless the reaserch is being conducted for a small population where everything is close enough to be researched. Maximum Variation Sampling: This form of sampling describes a wide range of cases being researched. One purpose for the use of this form would be so you can say that you researched everything and nothing was left out. Another reason would be for the researcher to look for a pattern among the sample being researched. Homogeneous Sample Selection: This kind of sampling would usually be used by focus groups because they can get a small group of people and research a common interest among them. The researcher would benefit from this type of sampling because they would get a greater understanding of how the people in the group thought about the topic. Extreme-case Sample: The sample would consist of choosing the extremes of a certain topic and researching it. The purpose for the research would be to gather rich information about the topic. This kind of research would consist of comparing and contrasting the two extremes of a certain topic. Typical-case Sample: This form is exactly what the name states, researching what is believed an average case. Critical-case Sample: This type of sampling consist of selecting a case that can make a point or deals with something really important. The guide for this type would be if I can do it, then so can you. Negative-case Sample: When selecting this type of sampling, the researcher actually disconfirms himself or herself in regards to the expectation of the case. The original idea or expectation is found to not be true and revision would be needed. Opportunistic Sample: This sample is one of the simplest ones because the case is usally chosen through opportunity. Mixed Purposeful Sample: The final sample refers to joining of two or more sample strategy. The purpose for this sample would be when the reasearcher has multiple data sources. 3. Sample Size A rule of thumb in random sampling size is the larger the better. Greater accuracy in results and in drawing inferences from results can be achieved through larger sampling sizes. Although including whole populations is the ideal way to insure zero sampling errors, this is not feasible either from a monetary standpoint or considering the amount of time it would take a researcher to gather this quantity of data. Thus the accepted practice of using representative sample groups has evolved within the research community. If the testing population as a whole is small, then to obtain reliable data, the researcher should select a large percentage of that population. In more homogenous testing groups, a smaller percentage can be chosen and should exhibit representative characteristics. However, when a researcher wants to extract more categories from the data, a larger sample size is needed. Also when a researcher predicts that the effect of the dependent or independent variable will be somewhat weak, a larger sampling group needs to be established. This will address the random error effect. Conversely the more well defined the sampling method, the smaller the sample size needs to be. Buffering your sampling group with extras to account for attrition and disqualifications is always a good idea so your final group will be sufficient to define the outcomes of your research. A good guide in choosing the sampling size can be gained from reviewing similar research done in the area of your research.
3.1 Quantitative versus Qualitative Quantitative research has a much heavier burden than qualitative research when it comes to determining the sample group and assuring its similarities to the population. Quantitative research is about stating a particular hypothesis and then testing it by collecting data. Qualitative research explores and describes what is observed and then generates a theory based upon those observations. The very nature of the data for quantitative research requires it to be much more concerned about sampling issues and techniques. 3.2 Generalizability The main objective when choosing a sample group for a research study is finding one which is comparable to the population. Generalizability only exists if that comparison is valid. Generalizability is the researchers' ability to make statements about the population dependent upon the findings of their sample group. All findings and conclusions made by the researchers concerning their particular research study will be applied to the population for their study. This will be valid if the sample is large enough and is similar enough to the population. There are time and money constraints which keep researchers from studying the entire population, so they must try their best to collect data from a sample group which represents that entire target population. 3.3 Type I and Type II error In an empirical study, it is assumed that there is no relationship between the variables in the study. This is called the Null Hypothesis. The researcher assumes that the Null Hypothesis is true unless there is evidence from the study to show that it is not true. A Type I Error occurs when the researcher concludes that the Null Hypothesis is false (rejects the Null Hypothesis) but it is true. A Type II Error occurs when the researcher concludes that the Null Hypothesis is true but it is actually false. 4. Sampling Bias and Error A statistic is a numerical characteristic of a sample. For example let s say 80% of people are visual learners -this is a numerical characteristic of a sample. A parameter is a numerical characteristic of a total population. So, the actual number of people who are visual learners would be the parameter. We can never really know the actual parameter of a population. That s why sample data is collected so that we can make estimates of the population parameter. A sampling error is the difference between the value of a sample statistic and the corresponding population parameter. There is always a sampling error, meaning that the sample statistic will sometimes be a little larger than the population parameter and will sometimes be a little smaller. (Johnson & Christensen, p.224) A sampling error is usually something that cannot be controlled by the researcher, there are sampling techniques that can be used to decrease the probability of sampling error. In research it is costly and difficult to achieve large sample size, however usually the larger your sample size the smaller the chances of having a sampling error. 4.1 Sampling Bias Nonrandom samples are said to be bias samples because they are almost always systematically different from the population on certain characteristics. (Johnson & Christensen, p.223) Would the following situation fall into the category of a sampling bias? The question this year at our school is: Is it probable that Transitional Bilingual students will do as well as Regular students in the English Reading and Writing TAKS like last year? This year Transitional students are again segregated from Regular students (same situation as two years ago); therefore leaving them at a disadvantage when it comes to taking the TAKS in English. They are receiving instruction in English but are not culturally integrated with fourth grade peers and other English native speakers. According to Random sampling students fall in the same category because they are all fourth grade students taking English TAKS.
Since the researcher selects the participants in a study, if there is a sampling bias in a study it would be considered the researcher s error. When selecting a sample, a researcher must try to represent the larger population or element to the best of his or her ability. In the example given above random sampling is used because overall achievement of fourth graders is what a researcher is trying to gauge. Since most fourth grade populations in Texas include regular and transitional students, this would be a sample of the actual population, regardless of the language diversity within the sample group. I certainly feel that this represents sampling bias because the students do not share the same mastery of the language. Therefore, the population has a key characteristical difference within which can affect the outcome of a study. 4.2 Sampling error The difference between a sample statistic and the population parameter is the sampling error. For example, if a researcher is studying the learning type (visual, auditory, kinesthetic, etc.) of a population and finds that 75 % are visual learners and the actual value is 72% of the population are visual learners, the difference is the sampling error. With different studies, even if they are studying the same phenoma or occurrence, the sampling error will vary. Consistently the sampling error will differ, but should not be too large or too small. Small or large sampling errors indicate sampling bias.(johnson and Chrisensen, p. 224) 5. Summary Given the variety of educational backgrounds, academic levels and social identifiers such as gender, age, race, and socioeconomic status of students across the globe, developing appropriate sampling technique in educational research is a top priority. Researchers must be cognoscente of their targeted population and choose the most appropriate sampling philosophy to support their particular research design. As discussed the above sections, the sample population in the research needs to be representative of populations in other areas of the globe in order to best support the research findings. Therefore, the researcher must take special care when choosing research candidates, sample size, and sampling strategy. In most cases, researchers choose to study a sample group which is representative of the total population. In this way, the researcher can generalize the entire population. It is important that the research includes a large enough sample before drawing generalizations. Sampling strategies can occur either randomly or non-randomly. In qualitative sampling, the choice of participants is limited to a defined set of criteria that everyone in the sample group must possess. In selecting sample groups, the rule of thumb is the larger the better. This assures greater accuracy in results and in drawing the conclusions from the results. It is much more important for researchers conducting quantitative research to be concerned with sampling issues and techniques. Errors can occur in empirical studies. These can be Type I or Type II errors. It is also possible for sampling errors or sampling bias to occur. It is important for the researcher to recognize any and all problems which can arise in choosing their samples and in making conclusions and generalizations after the study is concluded.