Chapter 2: Research Methods in I/O Psychology Research a formal process by which knowledge is produced and understood Generalizability the extent to which conclusions drawn from one research study spread or apply to a larger population 3 goals of science: description, prediction, explanation The Empirical Research Process Steps in conducting empirical research: 1. Statement of the problems 2. Design of research study 3. Measurement of variables 4. Analysis of data 5. Conclusions from research And back to #1. This is an important feedback factor: results from step five influence the first step in future studies Statement of the Problem Theory a statement that proposes to explain relationships among phenomena of interest Inductive method a research process in which conclusions are drawn about a general class of objects or people based on knowledge of a specific member of the class under investigation After conducting research on a topic, researchers may propose a theory about why the behavior occurs Deductive method a research process in which conclusions are drawn about a specific member of a class of objects or people based on knowledge of the general class under investigation Here, the researcher forms a theory first and then tests the theory by collecting data Accurate theory: data will support it; inaccurate theory: data will not support it Psychology is difficult to investigate people are far too variable Lewin a theory is useful for conducting research; it synthesizes info, organizes it into logical components, and directs the researcher s efforts in future studies Skinner too much effort is spent on proving theories; productive research doesn t require a theory extreme empiricism Chan researchers become too committed to proving their theories and become blinded to info that doesn t conform to the theory they want to believe A theory is an important way to specify research questions, but it is only one of many ways to formulate a research problem Design of the Research Study Research design a plan for conducting scientific research for the purpose of learning about a phenomenon of interest Research strategies may be compared on these two important dimensions: 1. The naturalness of the research setting 2. The investigator s degree of control over the study Internal validity the degree to which the relationships evidenced among variables in a particular research study are accurate or true o The extent to which the results are attributed to the variables investigated rather than to other possible explanations External validity the degree to which the relationships evidenced among variables in a particular research study are generalizable or accurate in other contexts o The extent to which findings from a study are relevant to individuals and settings beyond those specifically examined o Synonymous with generalizability If a study lacks internal validity, it can have no external validity Naturalness of the Research Setting Studying the problem in the environment in which it naturally occurs is desirable because we don t want the research strategy to distort the phenomenon under study Hawthorne study was conducted in a natural environment
Some studies don t need to be conducted in a natural setting though, because the behavior under investigation is assumed to be independent of the setting Ex. A study to test whether people react faster to red or green lights Degree of Control High degree of control controlling the exact amount of lighting in the Hawthorne study Low degree of control you can t control the age of the people in your class Low control is endemic to the questionnaire research method Primary Research Methods Primary research methods a class of research methods that generates new info on a particular research question No one method is perfect; none offers a high degree of both naturalism and control Four primary research methods: 1. Laboratory Experiment Laboratory experiment a type of research method in which the investigator manipulates independent variables and assigns subjects to experimental and control condition Conducted in a contrived setting; unnatural Researcher has a high degree of control The lab setting must mirror certain dimensions of the natural environment where the behavior normally occurs; omits those conditions that wouldn t be present Random assignment to enhance control and facilitates drawing causal inferences Study: effects of alcohol intoxication on visual-motor performance Error rates were dramatically higher under conditions of alcohol consumption Under the effects of alcohol, some people became more cautious and sacrificed speed for fewer errors Reduced speed of response may decrease errors, but it also may prevent engaging in needed defense maneuvers 2. Quasi-experiment Quasi-experiment a type of research method for conducting studies in field situations where the researcher may be able to manipulate some independent variables Less control Participants do not perceive the setting as having been created to conduct research Random assignment of study participants is often not possible in a field setting o Leads to less generalizable conclusions Study: how a one-day training program on goal setting affected the job performance of pulpwood workers o Results showed that the crews who were trained to set production goals for themselves harvested significantly more wood than the other crews o This study supported the use of goal setting in an industrial context o Major strength of the study: it was real; not in a lab o Weakness: some workers decided not to participate; few I/O psychologists are able to influence a company to change its work operations for research purposes 3. Questionnaire Questionnaire a type of research method in which subjects respond to written questions posed by the investigator Rely on individuals self-reports as the basis for obtaining info Classified as a non-experimental research method since no independent variables are controlled Most frequently used in I/O psychology Study on college-aged students and older students to indicate the degree to which they view testing for illicit drug use as justified in each of 35 different jobs
o Results indicated that the degree to which different jobs involved danger to the worker, coworkers, or the public was most strongly correlated to the acceptability of employee drug testing o Responses by both students were similar Practical limitations: people unwilling to complete questionnaires and return it to researcher (return rate is usually less than 50%) raises the question of how representative/unbiased the responses are for the group as a whole Questionnaires administered through the Internet are a more efficient means of obtaining data than via the mail o Internet = efficient means of collecting survey data Another limitation: the truthfulness of the responses given by respondents o Questions perceived to be sensitive or threatening are more likely to produce distorted responses They re still used extensively in I/O psych 4. Observation Observation a method in which the investigator monitors subjects for the purpose of understanding their behavior and culture Used when the research is examining overt behaviors Not used frequently in I/O psych because it requires a lot of time and energy Study: behaviors that differentiate effective and ineffective work supervisors o Results indicated that the primary behavior that differentiated them was the frequency with which they monitored their employees performance o Effective managers spent more time sampling their employees work Webcams may be useful for observational studies in the future Laboratory experiment Quasi-experiment Questionnaire Observation Control High Moderate Low Low Realism (naturalness of Low High Moderate High setting) Cross-Cultural I/O Psychology Motivational process: setting goals for yourself, directing own behavior, making evaluations o Highly individualistic (US) Eastern cultures motivation is an exchange process between a supervisor and subordinate In the US, questionnaires are a very popular approach for conducting research o Not for Russians, they worked on a questionnaire as a group Secondary Research Methods Primary research gathers/generates new info on a particular research question Secondary research methods a class of research methods that examines existing info from research studies that used primary methods Meta-analysis a quantitative secondary research method for summarizing and integrating the findings from original empirical research studies into a single result/outcome o Increases the likelihood of achieving more accurate conclusions than could be reached in an individual study by reducing errors of measurement o Involves subjective decisions determining which studies to include (only good quality ones); file drawer effect studies that yield negative results are not published (becomes biased toward the direction of positive outcomes) Level of analysis the unit or level (individuals, teams, organizations, nations, etc.) that is the object of the researchers interest and about which conclusions are drawn from the research o Issue: be careful not to meta-analyze studies focusing on different topics (level) Meta analysis is a popular research method in I/O psych Qualitative Research Qualitative research a class of research methods in which the investigator takes an active role in interacting with the subjects he/she wishes to study Implies the absence of quantitative research, which is not true Three purposes for conducting a scientific study: personal, practical, and research Psychology embraces pluralism there are different ways of living the human experience
Ethnography a research method that utilizes field observations to study a society s culture o Ethnographers try keeping an open mind preconceived notions can bias the findings Emic the insider s view emphasizes knowledge derived from the participants awareness and understanding of their own culture Etic external view emphasizes knowledge derived from the perspective of a detached objective investigator in understanding a culture Most ethnographers begin their research from the emic perspective and then make sense of it from the etic perspective need both I/O psych has been slow to adopt qualitative research Need both qualitative and quantitative Measurement of Variables o Variable an object of study whose measurement can take on two or more values o Quantitative variables objects of study that inherently have numerical values associated with them Ex. Weight, age, time o Categorical variables objects of study that do not inherently have numerical values associated with them Ex. Gender, race Can be coded to have numerical meaning o Variables Used in I/O Psychological Research Independent variables a variable that can be manipulated to influence the values of the dependent variable chosen by the experimenter Dependent variables those whose values are influenced by the independent variable the object of the researcher s interest The same variable can be used as the dependent or the independent variable depending on the goals of the study Predictor variables those used to predict or forecast a criterion variable o Similar to independent variables Criterion variables the focal point of a research study; it is forecasted by a predictor variable o Similar to dependent variables When scores on one variable (predictor variables) are used to predict scores on a second (criterion variables) Independent and dependent variables are used in the context of experimentation Predictor and criterion variables are used in any research where the goal is to determine the status of the subjects on one variable (criterion) as a function of their status on another (predictor) Independent variables are associated with making causal inferences; predictor variables may not be Analysis of Data o Statistics are used to make sense of the data o Descriptive statistics a class of statistical analyses that describe the variables under investigation describe data o Distributions and Their Shape Normal or bell-shaped many variables in psych research are distributed normally (with the most frequently occurring scores in the middle) The more variables, the smoother the distribution appears Skewed negatively skewed (tail of the distribution is in the negative direction) easy test; positively skewed (tail is in positive direction) hard test o Measures of Central Tendency 1) Learn the shape, 2) find the typical score using one of three measures depending on the shape Mean the average of a distribution of numbers Most common measure Formula: =
o Median the midpoint of all the scores in the distribution; 50% above, 50% below Best measure of C.T. for skewed distributions that contain some extreme scores (the mean would be greatly affected) Mode the most frequently occurring number in a distribution Least common measure Has practical purpose where whole numbers are more clear for a given concept In a normal distribution, mean, median, and mode are all equal In a skewed distribution, the mean and median are pulled toward the tail of the distribution Measures of Variability Variability the dispersion of numerical values evidenced in the measurement of an object or concept shows how representative the mean is as a measure of CT Range a descriptive statistical index that reflects the dispersion in a set of scores; arithmetically, the difference between the highest and lowest score Standard deviation a statistic that shows the spread or dispersion of scores around the mean in a distribution of scores Formula: s = 68% of scores fall within +/- 1 standard deviation of the mean 95% of scores fall within +/- 2 standard deviations of the mean o Correlation Correlation coefficient a statistical index that reflects the degree of relationship between two variables ranges from 1.00 to +1.00 Tells is two things about the relationship between two variables: the direction of the relationship and its magnitude The direction of a relationship is either positive (both increase/both decrease) or negative (one increases, other decreases) The magnitude of the correlation is an index of the strength of the relationship Correlation of.80 is stronger than.30 Magnitude and direction are independent; a correlation of -.80 is just as strong as +.80 The stronger the relationship between the two variables, the tighter is the spread of data points around the line of best fit that runs through the scatter plot o Determining Causality A correlation coefficient does not permit any inferences to be made about causality (whether one variable caused the other to occur) Laboratory experiment can identify causal relationships the causal-based conclusions are limited to only the variables examined in the study Statistical techniques with non-experimental methods (e.g., questionnaires) can make causal inferences controversial because people say human behavior is too complex to be unraveled by statistical calculations Correlations reveal statistical associations among variables; correlations do not reveal causal relationships Conclusions from Research o It is unwise to implement any major changes based on the results of only one study o Factor determining the boundary conditions for generalizing the conclusions from a research study to a broader population: The representativeness of the research subjects must use different groups of people o Researchers build on each others work Ethical Issues in Research APA code of ethics was created to protect the rights of subjects and to avoid the possibility of unqualified people conducting research Participants in psychological research are granted five rights: 1. Right to informed consent Right to know the purpose of the research Right to decline or withdraw at any time
Informed of any risks 2. Right to privacy Right to limit the amount of info revealed about oneself 3. Right to confidentiality Right to decide to whom they will reveal personal info leads to more honest responses when you guarantee confidentiality 4. Right to protection from deception Deception refers to a researcher intentionally misleading a participant about the real purpose of the research To use deception, they must show that the research outweighs the harm and that it can t be studied in any other way Deception is a last resort 5. Right to debriefing Debriefing occurs at the end to answer the participants questions, to remove harmful effects, to leave them with a sense of dignity To explain the importance of the research If these are violated, subject to professional censure and possible litigation Researcher is faced with more problems if participants are employees of companies o Role conflict researcher must comply with both company and professional standards o Sometimes must compromise between needs to client (the company) and maintaining confidentiality of the employee (the thief) Academic-Based and Practitioner-Based Research Pfeffer reported a 50% reduction in death rates from heart disease because physicians have implemented the findings from medical research Unlike medicine, there isn t a strong connection between I/O psychological research findings and the management of organizations Scientist-practitioner gap the difference between scientific findings on organizations and their management versus how organization are actually managed Reasons as to why the gap exists: o Academic researchers have over-emphasized the importance of theory in determining what constitutes useful research o The lack of publications that serve to translate complex scientific findings into more readily understandable terms (e.g., for managers) A scientist prefers descriptive findings ( what is the statistical relationship between X and Y ); a practitioner (manager) prefers prescriptive findings ( if X occurs, then do Y ) how to take action I/O Psychology and the Economy When the economy if contracting, business activity slows, and research by I/O psychologists is different: 1. The direction of the research follows the general edict of cutting back on expenses 2. In cutting back on expenses, money spent on research is often among the first items to be reduced in a budget 3. Unless the organization has a continuous large-scale need for I/O psychologists, it may be cheaper for them to use the temporary services of a consultant Case Study Questions: 1. What research method should Robin use to test her idea? 2. If this idea were tested using a laboratory or quasi-experiment method, what variables should be eliminated or controlled in the research design? 3. If this idea were tested with a questionnaire, what questions should be asked? 4. If this idea were tested with the observation method, what behaviors would you look for? 5. What other variables might explain the employees attitude toward Mr. Kast?