Qualitative and Quantitative Approaches Workshop. Comm 151i San Jose State U Dr. T.M. Coopman Okay for non-commercial use with attribution

Transcription

1 Qualitative and Quantitative Approaches Workshop Comm 151i San Jose State U Dr. T.M. Coopman Okay for non-commercial use with attribution

2 This Workshop This is a research skill workshop. This workshop provides an overview and information references that supply critical knowledge to help you succeed in this course. This workshop covers the differences between qualitative and quantitative research approaches. Instead of a task, this workshop is assessed via a quiz.

3 Quantitative vs. Qualitative Why does it matter? Broadly, many students think quantitative research means numbers and qualitative research is, well, without numbers. Or worse, the first is objective and the second subjective. However, the differences are much more distinct and involve different epistemologies - or world views.

4 Quantitative vs. Qualitative The world view behind the quantitative approach is usually either positivist (the idea that reality is singular, a priori, and objective independent of the knower) or post-positivist (The idea that the physical and social worlds are composed of complex phenomena that exist independently of individual perception, but human beliefs about these phenomena are inevitably multiple, partial, approximate, and imperfect). This researcher objectively observes, but neither participates nor influences what is being studied. The world view behind the qualitative approach is broadly interpretivist - the idea that realities are unique, plural, simultaneous, and local phenomena. They are accomplished between human beings through their symbolic practices of expression and interpretation. This researcher participates closely with what (or whom) is being studied and believes this is the best way of knowing.

5 Quantitative vs. Qualitative Here is a brief comparison between the two approaches. Types of question Qualitative probing research question Quantitative non-probing hypothesis Sample size small large Types of analysis subjective, interpretive statistical, summation Ability to replicate low high Type of research exploratory description or causal World view objectivity is not possible objectivity is possible

6 Quantitative vs. Qualitative Fundamentally, quantitative research is focused on the how whereas qualitative research is more interested in the why. Careers have been ruined and much ink and some blood has been spilling in what is called the paradigm wars. Such conflicts are counter-productive. The emerging view is that different tools and ways of knowing work best for different questions and often a combination of approaches yields the best results. Generally, if you want to know about large phenomena (how many people watch The Daily Show) then a quantitative approach would be best, but if want to know why people watch or what they get out of it, then a qualitative approach may be more suitable.

7 Tool Box In this class we take a tool box approach. That is, the proper tool for the proper job. First, understand that this is a mini-research project. Any full-scale research project would require much more data collection than is feasible for one class. For our purposes, let s look at this from the standpoint of the course research project. For each research project, you will use two qualitative and one quantitative method. To keep things simple, everyone project will utilize a quantitative survey and qualitative interviews and focus groups. First a quick overview.

8 Basics of Quantitative Research There are some basic terms and uses of statistics in quantitative analysis that should be utilized in your research. The Mean (average): the sum of value divided by the number of values. This can be skewed by excessive outliers. For our purposes, this is useful in comparison to other measures. Median: This represents the middle of the distribution. This is less skewed by outliers. To calculate, distribute all numbers evenly based on value. If you have an odd number, take the average of the middle two numbers. Mode: the mode is the most frequent occurring number in any distribution (it can also be thought of as the most popular option). How would you use this information? For example, the median household income for 2010 in the US was $49, 445 while the mean was $67, 530. This indicates that income distribution is skewed towards the high end. Another example could be grades in this class. The mean grade in this course last term was 71.83% or a C-. However, the median grade was 87.05% or a B+. What does this tell us? The skew is towards the higher end. In this case, a few very low scores brought down the average, but overall grades were higher.

9 Basics of Quantitative Research Two important issues in quantitative analysis are reliability and validity. Reliability is basically how consistent your findings are. This is why you would create a survey and distribute the same survey to everyone. Everyone takes that same survey. Another example would be in coding your data. If you are in a team, you should test code several of the same surveys to ensure you are coding for the same things and that your interpretations and categorizing of the data is consistent (inter-coder reliability). This is why it is important to to use a key or code book the explicitly defines all terms and what goes into each category. Even if you are working alone, it is easy to get data drift into other categories, so a key is crucial for consistent coding which equals good data. Validity refers to accuracy. That is, if your measures measure what they are supposed to measure (expected results). The analogy is a broken scale, which is consistent, but inaccurate. One way to achieve this is to use measures that have been used in other studies (criterion validity). This way, results can be compared to the literature. Content validity is if, for example, the survey asks questions that the survey taker can reasonably answer. That is, it measures what it is supposed to.

10 Basics of Qualitative Research Qualitative research does not have a lot of the specific technical terminology as quantitative research - this is by design - but there are some concepts that are important. First is that it is important that the voice of those you may study (participants) comes through in your data collection and analysis. This involves thick description of what is observed and generous use of direct quotes to illustrate points. The trustworthiness of qualitative research is not assessed the same way as quantitative research (validity, reliability, and objectivity). Qualitative research can be assessed via credibility, transferability, dependability, and confirmability. The credibility is assessed via the designing of data collection strategies that are able to adequately solicit the representations, but also to design transparent processes for coding and drawing conclusions from the raw data. Transferability refers to the extent to which the researcher s working hypothesis can be applied to another context. Dependability is determined by checking the consistency of the study processes, and confirmability is determined by checking the internal coherence of the research product, namely, the data, the findings, the interpretations, and the recommendations. The materials that could be used in these audits include raw data, field notes, theoretical notes and memos, coding manuals, process notes, and so on.

11 Variables Another aspect of quantitative and qualitative analysis is the idea of variables. A variable is basically just one aspect or element of a study - male and female are demographic variables. The key to research is discovering what variables could effect the outcome of study. There are two types of variables, independent and dependent. In quantitative analysis the independent is considered the cause while the dependent is the affect. In qualitative analysis, a variable is broadly any unit of analysis. Qualitative analysis does not seek to address cause and effect relationships. For example, in a class, the independent variable is the course design and dependent variable is the grade the student earns. As you may know, it is not that simple since there are a wide variety of other variables that can impact a students grade ranging from life issues (job loss, illness) to academic preparation (poor writing skills coming into the class) to students not completing assignments or failing to put effort into their work. This is something that makes assessing teachers performance very difficult. For our purposes, the idea of the dependent and independent variables leads us to look at a wide variety of elements that may impact communication behavior. For example, using Facebook. You may think that younger people are more likely to use Facebook (age being the independent variable) and many students are shocked when they discover that some of their peers do not use Facebook. The independent variables (use is dependent) could be lack of time, some consider it a waste of time or that it takes too much time, perceived negative impact on academic performance, its coercive nature, lack of privacy and security and each of these is based on different personal experiences with the service. Bottom line, simplistic cause and effect arguments are rarely useful. Behavior and people are complex and cultures are complex systems.

12 Scales Measurement Scales in Statistical Analysis In statistical analysis there are four basic levels of measurement. You might use statistical analysis in both qualitative and quantitative research. A Nominal Scale (mode) is the weakest form of statistical measurement. Researchers use a nominal scale to classify observations with no intention to order or rank the findings in level of importance. Such observations include highlighting the color of eyes, race, religion, nationality, and the like. An Ordinal Scale (median, percentage) incorporates the nominal scale, but seeks to rank responses with some "greater than" or "less than." For instance, a research questionnaire might be designed to learn how much adults enjoy using Facebook or the results of a NASCAR race might be listed in the order of finish. Interval Scales and Ratio (mean*) are a third form of statistical measurement. The first characteristic of interval and ratio scales is that the level of significance is treated in terms of known and equal intervals (eg. months, pounds). The second characteristic of these levels or scales is that they are quantitative in nature. * also includes a wide variety of other statistical analyses not covered here

13 Structural Constraints While we are usually looking at human communication behavior, human are constrained in the social, cultural, and physical structures they exist in. This impacts behavior because there are actual and normative (social) limits to what we can and cannot do in any given situation. For example, if you want to examine students use of cell phones in class, you need to take into account if the professor prohibits cell phone use. Otherwise the lack of cell use in a given class may lead you to draw false conclusions. Likewise, the lack of internet access will limit someone s internet use. These constraints may be hidden - even from those impacted by them. Normatively, we are constrained by the social and cultural forces around us. This often involves some sort of cultural coercion. For example, when all your friends are on Facebook the choice of joining is less voluntary. Often association is the key element of behavior. Those who share a network where a majority of members cheat in school (or are teen mothers or are over weight) greatly increases the odds of you engaging in the same behaviors. Often a structural constraint is one of several variables that impact behavior. For example, while news directors use a variety of subjective judgements in what goes on the evening news, she/he is also constrained by time and the demands of the medium (visual).

14 Combining Methods For this class, we use a quantitative method to go broad and two qualitative methods to go deep. Quantitative Qualitative Survey Interviews or Focus Group

15 Example Perhaps you want to know how people use Facebook within the context of romantic relationships. Qualitatively, you could ask How do people use Facebook in maintaining romantic relationships? Quantitatively, you could include a hypothesis such as, Increased Facebook use significantly predicts jealousy in romantic relationships. * * actual study in CyberPsychology & Behavior Journal - it s true!

16 Piloting The last think you want to do is to take a method into the field with testing it out. A test for a method is called a pilot study. This is small scale test to make sure the method will work and generate usable data. This is often just a matter of writing clear and concise questions. Just because it makes sense to you does not mean it will make sense to everyone else. I require that all methods (survey, interview, and focus group questions and activities) be piloted. This may seem like a pain, but at the end of every class the most common comments are (1) I wish I could done more pilots and (2) I am so glad I piloted!

17 Strategy Each method generates data that should be used to improve other data collection methods. For example, you ask people in your survey if they use Facebook to coordinate school group activities and no one does, it is likely that asking interviewees about this behavior will be a waste of time. One problem is often you do not know what you do not know. So, starting off with a focus group is great way to see how what you discovered in the lit manifests on the ground. So I suggest this strategy: 1. Pilot Focus Group(s) and revise 2. Focus Group(s): code data to get a good grasp of the topic. 3. Pilot surveys and interviews and revise. Schedule interviews. 4. Distribute survey (it will take some time to hit required # of respondents) and conduct your interviews while it runs.

18 Task For this workshop follow the link and take the Qualitative and Quantitative worksop quiz.