Step 3-- Operationally define the variables B. Operational definitions 3. Evaluating operational definitions-validity f. Reactivity Measurement changing the variable used to situation, awareness, learning, demand characteristics 4. Evaluating operational definitions-ethics Consequences for participants Step 3-- Operationally define the variables-summary A. Conceptual definitions of variables clear on what the variables are B. Operational definitions of variables 1. How to measure/manipulate the variable 2. Reliability the same, internally consistent meaning 3. Validity Is it what you intend? related to other measures of the same thing related to related things unrelated to different things appropriateness, reactivity minimize harm 4. Evaluating operational definitions-ethics 1
I. Case study A. What case studies are in depth information-few subjects B. Reasons for using a case study design 1. Generating hypotheses lots of information-possible relations 2. Rare and unique phenomena rare or variable 3. Intervention effectiveness for individual I. Case study C. Advantages and disadvantages Advantages: Rich information Disadvantages: Generalizing 2
A. Self-report measures -- awareness B. Designing a questionnaire 1. Open- or closed-ended questions Open ended questions - free response hard to summarize & compare Closed ended questions - constrain responses easy to score B. Designing a questionnaire 1. Open- or closed-ended questions Likert scale -- rating scale Indicate the extent you show these: 1. feelings of choking 4. unable to relax 2. shaky 5. numbness/tingling 3. feeling hot 6. nervous 0 1 2 3 not at severely, I can all barely stand it 3
B. Designing a questionnaire 1. Open- or closed-ended questions Thurston Scale -- Sequence of statements Which of the following reflects your view on government assistance? a. all welfare should be abolished b. welfare recipients should have to work for benefits c. welfare recipients should have to work for benefits, unless they have disabilities or small children d. welfare programs should provide job training & encourage, but not require, work e. all poor people have a right to government assistance B. Designing a questionnaire 2. Decreasing biases interpret one way not leading a. Clear unbiased questions Reverse wording Social desirability b. Response biases 4
B. Designing a questionnaire 3. Reliability and validity data Test-retest reliability Internal consistency reliability Relations to measures of related concepts Relations to other measures of the concept & external criteria 4. Instruments with known properties existing measures C. Samples 1. Probability samples Representative sample Simple Random Sample Stratified Random Sample Cluster Sample 5
C. Samples 2. Non-probability samples Haphazard/Convenience Samples, Quota Samples possibility of biased sample 3. How much the sample matters Matters a Lot describe the population known influences of culture or group Matters Less relating 2 variables no reason to expect influences of culture or group (basic process) III. Observations-people as measuring instrument A. Naturalistic vs. systematic observations Naturalistic observation - unsystematic Systematic observation - planned test hypotheses 6
III. Observations-people as measuring instrument B. Designing observational measures 1. Identify & define behaviors- What behaviors matter? Specific definitions - inter-rater reliability 2. Pick a coding method live or video running record event recording time sampling III. Observations-people as measuring instrument B. Designing observational measures 3. Validity concerns Reactivity Expected relations and independence Previously validated system 4. Reliability Coders training inter-rater reliability before and during 7
IV. A Comparison of Observations and Questionnaires Observations Researcher defines Questionnaires Subject interprets Behavior occurs Subject reports occurrence Behavior that is seen Subject aware of behavior Observer bias Response biases Fewer subjects More subjects Based on few instances Based on many instances More detail More generality Descriptive Statistics I. Why statistics are necessary? A. What is a believable difference? Treatment A: 75% improve (9 of 12 subjects) Treatment B: 67% improve (8 of 12 subjects) 8
Descriptive Statistics II. Basic statistical concepts A. Types of scales--classes of measures 1. Nominal- names, no order 2. Ordinal- rankings: order, unequal steps (single ratings are ordinal, summed ratings treated as interval) 3. Interval & Ratio-order and equal steps most flexible statistics Descriptive Statistics II. Basic statistical concepts B. Describing data 1. Frequency Distributions a. conventions: vertical - frequency, horizontal - DV b. Shapes Normal Distribution Rectangular Distribution Positive Skew Negative Skew 9
Descriptive Statistics II. Basic statistical concepts B. Describing data 2. Central Tendency a. Mode-most frequent score nominal data 5 people with depression 7 people with panic disorder 3 people with ADHD Mode is panic disorder Descriptive Statistics II. Basic statistical concepts B. Describing data 2. Central Tendency ordinal & skewed interval b. Median-half scores above, half below rank order, then find middle 1,2,3,4,4,10 median=3.5 mode=? 2,3,3,5,8,9,12 median=? 2,3,3,5,8,9,100 median=? not influenced by extremes 10
Descriptive Statistics II. Basic statistical concepts B. Describing data 2. Central Tendency interval or ratio c. Mean-average Add scores, divide by number of scores sum of the scores X = = number of scores 2,3,3,5,8,9,12 median =5 mean = 42/7 = 6 2,3,3,5,8,9,100 median =5 influenced by extremes mean =130/7 = 18.57 n X 11