Variable Measurement, Norms & Differences 1 Expectations Begins with hypothesis (general concept) or question Create specific, testable prediction Prediction can specify relation or group differences Different Types of Variables Subject Situation Response 1
Different Types of Definitions Subject Situation Response Enjoyment 1 (none) to 5 (lots) Conceptual Operational Issues in Measurement Ways of measuring Scales of measurement Measurement strategies Measurement accuracy Validity Reliability Basic descriptions Qualitative Operational Measures Descriptive Combines variables Contextualized Quantitative Objective Separates variables Decontextualized 2
Scales of Measurement Ratio Interval Ordinal Nominal Apply to quanitative measures Ratio Scales Levels are ordered Proportional relationship between levels It is possible to have none Example: exam scores 98 96 95 92 90 90 89 89 89 88 Interval Scales Levels are ordered Mathematical (not proportional) relationship between levels Cannot have none of it Example: temperature 98 96 95 92 90 90 89 89 89 88 3
Ordinal Scales Levels are ordered No mathematical relation between levels No set zero No fixed start or end point Example: year in college Freshman Sophomore Junior Senior Likert Scales Special type of ordinal scale Ordered rating categories Example: 1 2 3 4 5 Not at all A Lot Nominal Scale Levels are not ordered No relationship between levels Discrete categories Example: candies Lollipops Candy Bars Gum 4
And the type of scale is... Dollars Ratio DRDP Interval Number of books Ordinal Body Mass Index Likert Types of fruit Nominal Weight in pounds Categories on the food pyramid Parent satisfaction from 1 to 4 Appropriateness of books on the ECERS Measurement Strategies Direct test Observation Content Analysis Surveys Surveys Measure ideas rather than behavior Ease data collection Subject to error Double-binds Demand characteristics Social desirability Reactivity Order effects 5
Quality of Measures 16 Measurement Error Goal = assess concept Error = assess something else Source of error: Operational definition doesn t match validity Operational definition isn t consistent reliability Operational Quality - Validity Construct validity Measure assess concept Measure doesn t assess anything else Internal validity Study measures cause No other explanation is suggested External validity Study applies to real world Results apply to population of interest 6
Construct Validity Face validity (subjective) Criterion validity (objective) Predictive (in future) Concurrent (between groups) Convergent (between measures) Discriminant (between measures) Operational Quality - Reliability Test-retest Alternate forms Internal consistency Split-half Cronbach s alpha (α) Ranges from 0 to 1.60 is acceptable.80 is good Inter-rater Measures of Preschool Quality ECERS Strong correlation among scales 37 items Likert scale from 1 (inadequate) to 7 (excellent) α =.96 Caregiver Interaction Scale 26 items Likert scale from 1 (not at all) to 4 (very much) α =.93 Early Childhood Observation Form 25 items 3, 4, or 5 point Likert scales, didactic to child centered α =.92 Adult Involvement Scale 6 nominal categories Used % of time minimally or more involved Inter-rater reliability 7
Your Turn What do we know about reliability and validity of child outcome measures? 22 Describing a Group (statistics ) 23 Statistics Mathematical techniques for collecting, analyzing, interpreting, and presenting numerical data 2 Types: Descriptive Inferential 8
Goals of Research Description Prediction Causation Descriptive Statistics Inferential Statistics Describing a Group Norms (Central Tendencies) Individual Differences (Variability) Measures of Central Tendency Mean Median Mode 9
Mean Most familiar Mathematical average Procedure: Add up all scores Divide by # of scores Example: Scores: 5 5 5 4 4 4 3 3 3 2 2 2 2 2 1 Mean= (5+5+5+4+4+4+3+3+3+2+2+2+2+2+1) 15 Mean= 3.13 Median Exact middle of group Half the scores are higher Half the scores are lower Procedure: Rank order all scores Find middle score Example: Scores: 5 5 5 4 4 4 3 3 3 2 2 2 2 2 1 Mean= 3.13 Median= 3 Mode Most common score Procedure: Count frequency of each score Find most common score Example: Scores: 5 5 5 4 4 4 3 3 3 2 2 2 2 2 1 Mean= 3.13 Median= 3 Mode= 2 10
Measures of Variability Range: distance from smallest score to largest score Standard Deviation: how normal distant scores are Differences in Range Differences in Standard Deviation 11
Can describe group with numbers or with pictures 34 Frequency Distributions Three styles Pie graphs Bar graphs histograms Line graphs frequency polygons Select based on Type of scale Number of variables Point to be emphasized 2003-04 OCDE Enrollment by Ethnicity non Hispanic) 37.4 White (non( Hispanic) 37.4% African American Asian 1.9% 12.4% Hispanic or Latino 43.9% Other, Multiple and No Response 4.4% California Department of Education, Educational Demographics Unit 12
2003-04 OCDE Enrollment by Ethnicity 50 Proportion 40 30 20 10 0 African American Asian Hispanic or Latino Other / Multiple White California Department of Education, Educational Demographics Unit Receptive Language Girls Boys Not Yet 13% Emerging 6% Not Yet 16% Emerging 7% Fully 53% Almost 28% Fully 50% Almost 27% Receptive Language 60% 50% 40% Girls Boys 30% 20% 10% 0% Not Yet Emerging Almost Fully 13
Receptive Language 60% 50% Girls Boys 40% 30% 20% 10% 0% Not Yet Emerging Almost Fully ECERS Scores Cost, Quality, Outcomes (1999) Graphs vs. Numbers 42 14
Children s Receptive Vocabulary Cost, Quality, Outcomes (1999) Children s Receptive Vocabulary Cost, Quality, Outcomes (1999) Our Article 15
Frequency Graphs Single-variable Y-axis = count/ amount X-axis =variable 2 variable Y-axis = 1 st variable X-axis = 2 nd variable Y Axis = Amount/Count 60% 50% 40% Girls Boys 30% 20% 10% 0% Not Yet Emerging Almost Fully Frequency Graph And Line Graph Y Axis = 2 nd Varialbe Line Graph Not Frequency Graph 16