REVIEW: Monday: Jenna discussed sampling issues in survey research (probability sampling, types of surveys, how to create an effective survey) Thurs: In section, you will develop observational codes and also learn how to formulate survey questions. These are useful skills! APPLIED RESEARCH: You ll be asked to create surveys for student and community organizations and in your future jobs! Program evaluation for HIV education in Africa (Bill Gates Foundation) Correctional facilities and therapy (State of Washington) Community social worker (City of Seattle) Usability research (Microsoft) Marketing research (Starbucks) Remember that applied research can both contribute to basic knowledge and help solve real-world problems (TAs: Jennifer, Kristie, Cory) CONTAMINATION : Some people erroneously think that applied research is not real research because it has too many confounds ( the effects are contaminated. ) One difference between applied and basic research is in the trade-off between internal and external validity. As internal validity increases, external validity usually decreases. Some basic research privileges internal validity more than external validity. Some applied research privileges external validity over internal validity.
NONEQUIVALENT CONTROL GROUP DESIGNS Nonequivalent control groups are defined by 2 characteristics: 1. rely on non-manipulated IVs (definition of nonequivalent groups) 2. groups experience different events EXAMPLE 1: Violeta Amarova Does teaching style influence the self-esteem of experienced and nonexperienced dancers? IV1: Teacher (manipulated-- Encouraging versus Discouraging) IV2: Dancers (non-manipulated-- Experienced versus Non-experienced) DV: Self-esteem This is a P X E design. This is NOT a non-equivalent control group because dancers are matched by experience, and then randomly assigned to teacher condition (equal number of experienced dancers in Encouraging Teacher and of non-experienced dancers in Discourage Teacher conditions). EXAMPLE 2: Little League Baseball Does coaching training influence the self-esteem of children in Little Leagues? 8 coaches in League 1 received coaching effectiveness training (reinforcing effort, not just performance); 10 coaches in Leagues 2 and 3 did not receive any training. The coaches did not differ in experience or age. N=152 Little League boys from all three leagues were interviewed in the preseason and postseason. During the interview, the boys completed a self-esteem questionnaire (scores ranged from 14 to 56 where a higher score indicates higher self-esteem). IV1: Coach (manipulated Training versus No Training) IV2: League (non-manipulated) DV: self-esteem score
In an experimental design, coaches from the SAME league would be randomly assigned to training. In this design, it wasn t practical, and a nonequivalent control group design ensured no leakage within the league. Boys in the experimental group (with coaches who received effectiveness training) and in the control group (with coaches who did not receive effectiveness training) did NOT statistically differ in preseason or postseason self-esteem. No differences in won-loss records across leagues. However, upon further analyses, the researchers found that boys with low SE showed increases in SE in the experimental group, whereas boys with low SE did not show any increases in SE in the control group. 50 49 48 47 Mean SE Score 46 45 44 Control Trained 43 42 41 40 Preseason SE Self-Esteem Scores Postseason SE At the end of the season, the researchers also found that boys in the experimental group reported greater 1) fun playing baseball, 2) liking for their coach, 3) coach s teaching ability, 4) liking for teammates, compared with the control group.
NONEQUIVALENT VS. NONEQUIVALENT CONTROL GROUPS: Nonequivalent groups rely on non-manipulated IVs. Natural groups are a kind of nonequivalent group design. So are single-factor nonequivalent groups designs (brain-injured versus non-injured Ps) P X E factorial designs (gender and vessel noise effects on echolocation abilities among killer whales) correlational research (Psych 209 survey) Nonequivalent control groups are defined by a 2nd characteristic: 1. rely on non-manipulated IVs (same as nonequivalent groups) 2. groups experience different events Brain Injury Study Killer Whale Study (lecture) Psych 209 Survey Violeta Study Little League Study Non-manipulated Different Design IV event? Brain injury No Nonequivalent Gender No Nonequivalent Studiousness No Nonequivalent Dancer Experience No Nonequivalent League Yes (Coach) Nonequivalent Control
PROGRAM EVALUATION: (PUT THIS ON YOUR RESUMES!) 1. Planning for Programs Needs Analysis: Is there a population that can benefit from a proposed program? What is the problem to be solved? Use census data for demographics, survey of available resources, survey of potential users, focus groups. EXAMPLE: Du Pont Company (p. 373): surveyed employee health records (absence, disability, illness) over 15-year period. Lowestranking and least well-paid workers had the highest illness burden. 2. Monitoring Programs Formative Evaluation Is the program resulting in progress? Is the program being implemented correctly? Collect information about how the program is being used, use pilot studies. EXAMPLE: The Oprah Winfrey Leadership Academy For Girls. Oprah Winfrey spent $40 million dollars to build a school for disadvantaged girls in a small town near Johannesburg, South Africa. Oprah wanted to create a safe, nurturing environment for girls to promote leadership. The school is on 22 lush acres and spread over 28 buildings. The complex features oversize rooms done in tasteful beiges and browns with splashes of color, 200-thread-count sheets, a yoga studio, a beauty salon, indoor and outdoor theaters, hundreds of pieces of original tribal art and sidewalks speckled with colorful tiles. Parents have complained that the maximum number of visitors per student is four so in some cases the entire family can't visit at once. Family members only get a chance to visit the girls once a month. So that means if a girl has five siblings, the only way she can see all of them is over a period of three months.
3. Weighing Costs Cost-effectiveness Analysis What is the estimated cost of a program? Are there two programs that are equally effective, but that differ in cost? Use nonequivalent control group designs. EXAMPLE: Comparison of 2 worksite wellness programs (p. 376). Company 1 spent an average of $39.28/employee to build a state-of-theart fitness center. Company 2 spent an average of $30.96/employee to focus on health education, follow-up, and special attention to at-risk employees. Risk reduction after program was 36% in Company 1 and 48% in Company 2. Company 2 s program is more effective and less expensive. 4. Evaluating Outcomes Summative Evaluation Is the program effective, overall? Use quasi experiments and surveys. EXAMPLE: Course Evaluations! Course Goals: 1) Recognize the field of psychology as a diverse field of scientific inquiry. Understand and appreciate the hallmarks of scientific thinking: its basis in both empirical and rational modes of understanding the social and natural world. Acknowledge the uncertainty associated with the conclusions that can be drawn from scientific research and employ skeptical inquiry as a consumer of scientific research. 2) Understand as one of the major tasks of psychology the measurement of theoretical, and often intangible, constructs (e.g. intelligence, empathy, conformity, self-confidence) by the use of operational definitions. Recognize that theoretical constructs can often be operationalized or measured in multiple ways.
3) Understand the role of hypothesis testing in theory building. Recognize the wide variety of study designs and experimental methods available to researchers and understand that the use of these different methods often depends on the question posed by researchers (e.g. correlational vs. experimental). 4) Develop skills related to reading scientific literature. Be able to read about a scientific study and understand the measurement issues and research design of that study. Be able to identify the hypotheses or predictions made by the researcher(s) and develop a keen eye for identifying potential sources of bias and limitations of studies. 5) Search for and evaluate the scientific literature in psychology and be able to explain the differences between scientific and/or popular sources. Recognize the features of and the rationale behind scientific writing in contrast to popular sources. Identify the common parts of a research article and why the structure is useful for research and the dissemination of scientific knowledge. 6) Build a foundational understanding of the role of statistical methods in psychological research by being able to collect, manage, and summarize data both visually (by making graphs and tables in Excel) and numerically (by being able to compute basic descriptive statistics). Use scientific writing to represent the scientific method by summarizing articles or proposing a research project. 7) Show appropriate concern for honesty and due care in behavioral research. Understand basic guidelines for conducting ethical research. (FRIDAY S LECTURE AND NEXT WEEK) 8) Apply basic research methods knowledge by being able to generate an empirical question, operationalize variables of interest, select an experimental design, describe results, and discuss the implications of your findings.