Research Methodology Samples, Sample Size And Sample Error Sampling error = difference between sample and population characteristics Reducing sampling error is the goal of any sampling technique As sample size increases, sampling error decreases 1 2 How Big Is Big? Estimating Sample Size The goal is to select a representative sample Larger samples are usually more representative But larger samples are also more expensive And larger samples ignore the power of scientific inference Generally, larger samples are needed when Variability within each group is great Differences between groups are smaller Because As a group becomes more diverse, more data points are needed to represent the group As the difference between groups becomes smaller, more participants are needed to reach critical mass to detect the difference 3 4 Variables Variables Research involves trying to determine the relationship between two or more variables. The researcher asks: What is the effect of X on Y? How is X related to Y? Some examples might be: IQ Anxiety Academic success Gender Income Recidivism rates 5 6 1
Variables Types of variables In order to study variables, we must define them clearly, and have some way to measure them. In research we call this the process of creating operational definitions. For example: Treatment outcomes may be measured by changes in behavior (going to the store alone) or changes on a scale (such as an anxiety scale). 7 Independent A variable that is manipulated to examine its impact on a dependent variable. Manipulated by the researcher. Treatments of conditions under control of the researcher. Independent of the outcome. Presumed to cause, effect or influence the outcome. 8 Types of variables What Makes Good Variables? Dependent A variable that indicates whether the treatment or manipulation of the independent variable had an effect. Dependent on the independent variable(s). Independent variable is not confounded Levels do not vary systematically with other variables Depends on the experimental treatment. Depends on how the independent variable(s) are managed or manipulated. The outcome of a research study. Dependent variable is sensitive to changes in the independent variable 9 10 Types of variables Variance Extraneous A variable that is related to the dependent or independent variable that is not part of the experiment. Differences among phenomena - Without differences and variation there is no way to determine relations among variables. 11 In research, means of different experimental groups are compared to study relations. 12 2
Error Variance Error Variance Is the fluctuation or varying of measures due to chance (random variance). Sources of Error Sampling taking a portion of the population and considering it to be representative. Variation in measures due to the usually small and self-compensating fluctuations of measures. 13 14 Sources of Error Sources of Error Within the Individual General Level of ability Test taking skills Ability to understand instructions Specific Ability related to the trait being assessed. Test-taking skills specific to the test. Temporary Health, fatigue, emotional strain, motivation. 15 Outside the Individual Test Administration Factors Conditions of testing Interaction between examiner and test taker. Bias in grading. Rater Bias. 16 Interpretation of data and reports Research Biases - Placebo s Research report (different formats) - Researcher Bias Deliberate or inadvertent bias in which data are misanalyzed or participants are treated differently over and above any planned differences in treatment. 1. Intro or Rational 2. Review of Literature 3. Method (research design) 4. Results and discussion 5. Summary 17 - Single / Double Blind Studies - Hawthorne Effect Refers to conditions under which performance in an experiment is affected by the knowledge of participants that they are in an experiment. - Rater Bias - Pilot Study 18 3
Ethical Considerations Protection from harm Minimize the risk to participants in the study, including, if needed, follow-up contact to resolve potential negative consequences associated with their participation. Research Design is a plan, structure, strategy of investigation conceived to obtain answers to research questions and to control variance. Debriefing Is an ethically required process where the researcher explains the nature and purpose of his/her research to the participants. 19 20 Research Designs Good research does not give us answers only new questions! 21 True Experimental To investigate possible cause-andeffect relationships by exploring one or more experimental groups to one or more treatment conditions and comparing the results to one or more control groups not receiving the treatment (random assignment being essential). 22 Treatment / Control Group Basic Research Designs Treatment Group - Is the set of participants in an experiment that receives the treatment/independent variable. Control Group - Is the set of participants in an experiment that does not receive the treatment/independent variable. Data from the control group are compared against the experimental group to evaluate the potential effect of an Historical To reconstruct the past objectively and accurately. Descriptive To describe systematically a situation or area of interest factually and accurately. Developmental To investigate patterns and sequences of growth and/or change as a function of time. Case and Field To study intensely the background, current status, and environmental interactions of a given social unit: an individual, group, institution, or community. independent variable. 23 24 4
Basic Research Designs Correlational To compare variables to determine a relationship of + /-. Ex-Post Facto To investigate possible cause-and-effect relationships of past events. Quasi Experimental Is a type of experimental research design that contains a variable that cannot be directly manipulated or controlled be the researcher. Survey Uses questionnaires to collect information from a large number of 26 individuals. 25 Hypothesis Reflects the general problem under study Restates the general problem in a form that is precise enough to allow testing Null Hypothesis States that there is no relationship between the independent and dependent variables under study H o : µ 1 = µ 2 H o : Null hypothesis µ 1 : Theoretical average of population 1 µ 2 : Theoretical average of population 2 Purpose Of Null Hypothesis A starting point for analysis Accepted as true absent other information Assumes that chance caused any observed differences Provides a benchmark for comparison 27 28 The Research Hypothesis A statement of inequality A relationship exists between the independent and dependent variables H 1 : X 1 X 2 H 1 : Research hypothesis X 1 : Theoretical average of population 1 : Theoretical average of population 2 X 2 Directional vs. Nondirectional Research Hypothesis Nondirectional Research Hypothesis Groups are different, but direction is not specified H 1 : X 1 X 2 Directional Research Hypothesis Groups are different, and direction is specified H 1 : X 1 > X 2 H 1 : X 1 < X 2 29 30 5
Purpose Of Research Hypothesis Differences Between Null & Research Hypothesis Directly tested during research process To compare against null hypothesis Null Equality between variables Refers to population Indirectly tested Stated using Greek symbols (µ) Implied Research Inequality between variables Refers to sample Directly tested Stated using X Roman symbols ( ) Explicit 31 32 What Makes A Good Hypothesis? Working With The Hypothesis Is stated in declarative form Posits a relationship between variables Reflects theory or literature Is brief and to the point Is testable If the hypothesis is confirmed Plan new research If the hypothesis is refuted Try to understand what other factors might be important 33 34 2009 Pearson Prentice Hall, Salkind. Scientific Proof Science does not Prove anything It only gives evidence to except or not except a given hypothesis 4 - Purposes of Statistics 1. To reduce large quantities of data to manageable and understandable form. 2. To aid in the study of populations and samples. 3. To aid in decision making. 36 35 4. To aid in making reliable inferences from observational data. 6
2 - Types of Statistics 1. Descriptive Describes the data (ex. Mean). 2. Inferential Makes statements beyond the data. Measurement Mean The average of a set of scores. Find the sum of all scores and divide by the number of scores (N). Median The midpoint of a distribution Half of the scores fall above the median and half fall below it. 37 38 Mode The most frequently occurring value Standard Deviation t & z - tests The positive square root of variance. How scores vary around the mean squared. t-test Determines a significant difference between: Only two sample means. (small sample) 39 It s the average distance that each score is form the mean. 40 z-test Determines a significant difference between: Only two sample means. (large sample) ANOVA (similar to a t-test) Chi Square Two or more groups (from sample population) are given a common test. Looks at interactions between groups (variance) and within groups (variance). Uses frequency counts (not scores). Differences in observed and expected frequency. Can not be any interaction must be independent. 41 Shows interactions. 42 Looks at variability between groups compared to within groups. 7
Correlational Looks at relationships between two variables Graphs Techniques which may be employed to organize and summarize a collection of test scores to represent data graphically (visually). Visual ways to represent data. 43 44 Types of Graphs - Histogram Line graph Bar graph Probability Purpose of Analysis 45 The probability that an event (object or person) will occur is equal to the relative frequency of occurrence of that event (object or person) in the population under consideration. Example Population = 100 30 females 70 males Prob. of selecting a male at random drawing 70/100=.70 46 Reduces data to intelligible and interpretable form so that the relations of research problems can be studied and tested. Positive results Negative results Supports hypothesis Interpret according to theory. Are harder to interpret than positive ones. Questions - Why What s wrong with the prediction 47 48 8
Next Week Mid-Term Exam Week 6 & 7 Appointments What to bring to appointment Week 8 Final Exam Week 9 Spiral Bound Proposal Due Turnitin Receipt 49 9