Methods in Perception. Qualitative Methods in Perception. Qualitative Observation. Methods in Perception Research

Transcription

1 Methods in Perception Methods in Perception Research Chapter 1: Fundamentals of Sensory Perception Qualitative: Getting the big picture. Quantitative: Understanding the details Threshold-seeking methods Magnitude estimation Many others, often similar to those used in other areas of psych. 1 2 Qualitative Observation Thatcher Illusion Qualitative observation uncovered the phenomenon Much quantitative work now trying to find out why it happens & what it means Schwaninger, A., Carbon, C.C., & Leder, H. (2003). Expert face processing: Specialization and constraints. In G. Schwarzer & H. Leder (Eds.), Development of Face Processing, pp Göttingen: Hogrefe. Qualitative Methods in Perception Also called phenomenological or naturalistic observation methods Relatively non-systematic observation of a given perceptual phenomenon or environment (e.g., an illusion) Yields a verbal description of one s observations, (possibly with some simple numerical assessment) First step in any study of any perceptual phenomenon. Gives the big picture 3 4

2 Qualitative Methods in Perception Example: Famous perception researcher Jan Purkinje noticed that his flower bed looked light red/dark green during the day but dark red/light green at twilight This phenomenological observation led to the hypothesis of 2 visual systems, & ultimately to an understanding of the functions of rods and cones 5 6 Quantitative Methods Quantitative Methods in Perception Threshold seeking methods measure a physical quantity representing a limit of perceptual ability (i.e., a threshold) Measured in physical units (meters, decibels, parts-per-million, candelas of light, etc.) Absolute threshold - smallest detectable physical quantity (e.g., 2 db, 3.57 grams...) Difference threshold - smallest detectable difference between two physical quantities 7 8

3 Quantitative Methods in Perception Thresholds are defined for a given level of response accuracy Typically we speak of the 50% threshold, meaning the physical quantity detectable (absolute threshold) or the physical difference detectable (difference threshold) 50% of the time But a threshold can be defined for any level of accuracy (e.g., 75%, 83.6%...) 9 Threshold-seeking Methods Classic methods (Fechner, 1850 s) Method of adjustment: Quick and dirty Method of limits: Easy on observer, fairly fast and accurate Method of constant stimuli: Very slow but very accurate Adaptive methods: Fast, very accurate, but can be difficult for untrained observers 10 Method of Adjustment Stimulus intensity is adjusted (usually by the observer) continuously until observer says he can just detect it Threshold is point to which observer adjusts the intensity Repeated trials averaged for threshold Fast, but not always accurate, due to inherently subjective nature of adjustment Method of Adjustment Instructions: Adjust the intensity of the light using the slider until you can just barely see it Photometer Reading: cd/m

4 Method of Limits Stimuli of different intensities presented in ascending and descending order Observer responds to whether she perceived the stimulus Cross-over point (between yes, I see it and no, I don t ) is the threshold for a sequence Average of cross-over points from several ascending and descending sequences is taken to obtain final threshold. Method of Limits (descending sequence) Instructions: For each light intensity, indicate whether you can detect it. Photometer Reading: cd/m Method of Limits (ascending sequence) Instructions: For each light intensity, indicate whether you can detect it. Example Data From Method of Limits Why ascending and descending sequences? Why different starting points? Photometer Reading: cd/m

5 Method of Constant Stimuli 5 to 9 stimuli of different intensities are presented many times each, in random order The intensities must span the threshold, so must know approx. where it is a priori. Multiple trials (often 100 s) of each intensity are presented Threshold is the intensity that results in detection in 50% of trials Method of Constant Stimuli Instructions: For each light intensity, indicate whether you can detect it cd/m 2 Photometer Readings Question Here are some data from a participant in the MoCS. Can you estimate her 50% threshold? Stimulus Intensity cd/m 2 % of stims detected The Psychometric Function (Stimulus Intensity cd/m 2 ) 20

6 To calculate thresholds we use curve-fitting techniques to fit a sigmoidal (=s-shaped) function to the data (green dots). This is called a psychometric function (grey line). It links physical stimulus intensity to performance Debate exists over which kind of function--cumulative Normal, Weibull, Logistic, etc.-- is theoretically best, but in practice differences are minor Calculating 50% Threshold Intensity Math Review: Logs Remember exponents? 2 2 = 4 Logarithms are the opposite log2(4) = 2 What exponent will turn the base--usually 2, 10 or e--into the operand? What is the log10(100)? log2(8)? Khan Academy: Math Review: e (or Euler s Number ) A mysterious constant that just pops up everywhere in nature. e loge is called the natural logarithm, often symbolized as ln One also sees e x, where x can be quite a complex expression. exp(x) is also used. Learn to use your calculator to do logarithms and work with e Math Review: Inverse Functions If a function F takes input X and returns Y then inverse F takes input Y and returns X Example: Y = 2X inverts to X = ½Y The Weibull: y = exp(-(x/b) a ) The inverse Weibull x = b(-ln(1-y)) (1/a) 23 24

7 Math Review: Free Parameters Free parameter: Part of a math function that is adjusted so that the function fits data Example: The intercept (a) and slope (b) of a line are free parameters when fitting a regression line: y = a + bx; y = 0 +2x y = 10 +1x y = 0 +1x Math Review: Free Parameters y = exp(-(x/5.0)1.0) y = exp(-(x/1.5)1.0) y = exp(-(x/1.0)1.0) y = exp(-(x/0.5)1.0) In the Weibull (and its inverse) a and b are free parameters. a is the offset (in this example a = 1 in all cases) b is the slope (here varying from 0.5 to 5.0) The Weibull Function y = exp(-(x/b) a ) x is stimulus intensity (in some positive physical unit) y is predicted probability of stimulus detection (from 0 to 1) a is the offset and b is the slope a and b are free parameters whose values are chosen so that the curve best fits the data. These values are determined by curve-fitting algorithms whose details are beyond the scope of the course. Calculating 50% Threshold Intensity Via Inverse Weibull x = b(-ln(1-y)) (1/a) To figure out the threshold, we need to figure out what the right values of a and b are for the Weibull that best fits our data. We will then enter the percentage threshold we are seeking (e.g.,.5 for 50% threshold) into the inverse Weibull (above) to determine the associated stimulus intensity x 27 28

8 Calculating 50% Threshold Intensity Open the excel file "WeibullFit.xlsx" Enter your data in two columns: X (stimulus intensity) in column A Y (proportion detected) in column B Click the Tools menu and select "Solver ". Note that you may have to first activate the solver. See the document on the class website called Loading MS Solver Add. In the solver window, click Options, then enter 10 in max time, then click Okay. Click "Solve" in the window that appears. After a moment, the Fit Parameters will appear in cells G2 (slope, or A) and G3 (offset, or B) So for our data, the inverse Weibull is x =.603 (-ln (1-y) ) (1/4.08) Plug in the desired probability of 0.5 and to a threshold of cd/m 2 Inverse Weibull step-by-step x =.603 (-ln (1-0.5) ) (1/4.08) x =.603 (.6931) (1/4.08) x =.603 (.6931) (.245) x =.603 (.9141) x =.5512 Y"(Data)" 1" 0.9" 0.8" 0.7" 0.6" 0.5" 0.4" 0.3" 0.2" 0.1" 0" 0.00" 0.20" 0.40" 0.60" 0.80" 1.00" X" "Data" "Fit" Inverse Weibull step-by-step Example for Self-Test Here are some data from a participant in the MoCS. What is her 50% threshold? What about her 83% threshold? x =.603 (-ln (1-0.5) ) (1/4.08) x =.603 (.6931) (1/4.08) x =.603 (.6931) (.245) x =.603 (.9141) x =.5512 Y"(Data)" 1" 0.9" 0.8" 0.7" 0.6" 0.5" 0.4" 0.3" 0.2" 0.1" 0" 0.00" 0.20" 0.40" 0.60" 0.80" 1.00" X" "Data" "Fit" Stimulus Intensity % of stims detected % =c.bj 83% = d.fc 31 32

9 Questions What is a free parameter? What is log 10(10000)? 33 Adaptive Methods Examples: Staircase, QUEST, PEST, etc. Stimulus first presented at an arbitrary level If observer perceives it, intensity is reduced by a predetermined step-size If observer does not perceive stimulus, intensity is increased This is repeated until several reversals are obtained. Threshold is average of reversal points. 34 Reversal Points Stimulus Intensity N N N N N N Y Y Y N N N Y Y Y N N Y Y N Observer Response Do you see it? Threshold = ( ) / 6 = N N N N N N Y Y Y N N N Y Y Y N N Y N N But why is it called the Staircase Method? 35 36

10 Stimulus Intensity Example for Self-test N N N N N Y Y Y N N N N Y Y Y N N Y N Y Observer Response Do you see it? Calculate the threshold for this run by this observer. Adaptive Methods Similar to method of limits, but we don t run the same series of levels each time, instead adapting each series based on previous performance. A number of specific procedures exist, which modify how the adaptation is done (size of steps, etc.): Staircase, QUEST, PEST, etc. Very efficient, because almost all trials are close to the threshold and therefore informative. But best used with trained psychophysical observers (can be frustrating for untrained Ss) 37 ebh.f 38 Forced Choice Variations Some threshold-seeking methods can be hampered by response bias. Some participants have a lax criterion and tend to say yes, I see it a lot. Some participants have a strict criterion and tend to say no, I don t see it a lot. One way to mitigate this problem is to use forced-choice methods. X-Alternative Forced Choice Variations In a forced-choice task, a participant is given several options to choose from: One contains the stimulus and the others don t. The participant is asked, for example: Which of the two boxes has a light in it? Note that a forced choice method is a modification of (addition to) the thresholdseeking methods we ve looked at so far

11 X-Alternative Forced Choice Variations Any number of alternatives can be offered, but usually 2 to 8 are given. A 2AFC is a two-alternative forced choice procedure, for example. The number of alternatives affects the chance level performance (= 100% / A) 2AFC Method of Limits (descending sequence) Instructions: For each light intensity, indicate which side it s on. Photometer Reading: cd/m Threshold-finding With Other Senses The same methods can be used with a wide variety of sensory qualities. Different physical units are used depending on the modality: Sound amplitude (decibels) Touch pressure (pascals) Smell/Taste intensity (parts-per-billion) Questions What is an absolute threshold? Name several methods for measuring absolute thresholds. What is an xafc procedure? Why use one? Describe generally how the method of limits works

12 Difference Threshold Smallest intensity difference between two stimuli a person can detect Produces a Just Noticeable Difference (JND) in degree of subjective sensation Same psychophysical methods are used as for absolute threshold As magnitude of stimulus increases, so does difference threshold ( I) Weber s Law Weber s Law describes the relationship between stimulus intensity (I) and difference threshold ( I) as follows I / I = K or I = I K This holds true for many senses and many physical quantities across a wide range of moderate intensity levels (but see Steven s Power Law) Weber s Law 47 48

13 Method of Adjustment for Difference Threshold Instructions: Adjust the intensity of the test light using the slider until you can just barely see the difference between it and the standard light Standard Light: Photometer Readings 0.6 cd/m 2 2AFC Method of Limits for I Instructions: For each set of lights, indicate which pair (right or left) differ. Test Light: cd/m Standard Light (Constant) Test Light (Adjustable) Example for Self-Test 20 I t Q: If your Weber fraction (k) is.05, and the standard light s intensity (I s ) is 20, what level will the test light intensity (I t )have to be raised to in order for it to be just noticeably different from the standard? A: It will have to be higher by the I; That is, I t = I s + I I t = I s + (I s k) I t = 20 + (20.05) = 21 where I = I s k 51 Standard Light (Constant) Test Light (Adjustable) 50 I t Q: If your Weber fraction (k) is.20, and the standard light s intensity (I s ) is 50, what level will the test light intensity (I t )have to be raised to in order for it to be just noticeably different from the standard? 52 fj.j

14 Questions Beyond the Threshold A participant can just tell the difference between lights of 100 cd/m 2 and 112 cd/m 2. Therefore, she should be able to just tell the difference between 200 cd/m 2 and cd/m 2. What is this participant s Weber fraction for light intensity? 53 Beyond Thresholds What do we know about the relationship between subjective sensation and objective intensity at levels above threshold? Classic work done by Fechner, who derived Fechner s Law from Weber s work. This has since been largely supplanted by Steven s Power Law. Fechner s Law From Weber s findings, Fechner derived the idea that subjective sensation (S) related to stimulus intensity according to: S = k ln(i/i0) k = empirically-determined free parameter Recall that ln means log to base e I = stimulus intensity I0 = stimulus intensity at absolute threshold 55 56

15 Fechner s Assumption Fechner s Assumption Fechner assumed that each time you went up by one difference threshold (or, subjectively, one JND), that that related to a an equal jump in subjective intensity. Intuitively appealing, but turns out to be wrong, as Stevens later showed Fechner s Assumption Fechner s Law Subjective Sensation (S) I I I Works fairly well for some modalities (loudness, weight), but not others (electric shock) that show response expansion More closely models the response of individual neurones than people. Still, your music player s volume control is modelled after Weber s and Fecher s laws Objective Stimulus Intensity (I) 59 60

16 Magnitude Estimation Technique pioneered by Stevens to examine the relationship between subjective perception and objective stimulus intensity at easily perceived levels All stimuli are well above threshold Observer is given a standard stimulus and a value for its intensity (e.g., see this light? this is a 5. ) Observer assigns numbers to the test stimuli relative to the standard (e.g., that looks about twice as bright, I ll call it a 10. ); Magnitude Estimation Relationship between intensity and perception usually shows either: Response compression: As intensity increases, the perceived magnitude increases more slowly than the intensity Response expansion: As intensity increases, the perceived magnitude increases more quickly than the intensity Magnitude Estimation Relationship between intensity and perceived magnitude is a power function Stevens Power Law S = k I b (amazingly simple!) Note how for b < 1, we get an approximation of Fechner s law (red line) Magnitude Estimation S = k I b S = perceived magnitude I = physical intensity k and b are empiricallydetermined free parameters Note that k is not the Weber Fraction 63 64

17 The 3 Laws of Psychophysics Weber s law relates two physical units: Standard stimulus intensity and difference threshold Fechner s law relates a physical unit (stimulus intensity) with subjective sensation (or so he thought). The above two derive from one another directly, and are sometimes called the Weber-Fechner law Steven s Power Law expands on Fechner s Law, covering stimuli that show response expansion as well as more closely modelling human responses. Questions Steven s Power Law: S = k I b Given that for bitterness b=2 and k=2, what will sensory magnitude (S) be for stimulus intensity (I) of 2 ppm? What if I is doubled to 4 ppm? What does this mean about the relationship between stimulus intensity and sensory magnitude in this case? Sensitivity & Signal Detection Theory Sensitivity & Signal Detection Theory Sensitivity (d ) Criterion (c) Will spend some time on this, as it is used in many fields (though with different jargon): Diagnostics Inferential statistics Human factors engineering 67 68

18 Absolute Thresholds Ain t So Absolute Thresholds shift for many reasons unrelated to actual sensitivity. An especially problematic factor is criterion (tendency to say yes a lot or no a lot). For example, an individual doing Method of Limits could just say yes to all stimuli (lax criterion) and look like he s incredibly sensitive! xafc methods are one way to deal with this, but another is to measure Sensitivity instead of threshold. Sensitivity Sensitivity is symbolized d' ( dee-prime ). Measure of one s ability to detect a given signal (= a stimulus), usually at low intensity. Arises from Signal Detection Theory (more later) How might we measure sensitivity? (a bad way of) Measuring Sensitivity Present stimulus 100 times and note % of times participant says he detects it? Problem: P who says yes all the time will do very well. (the very problem we are trying to avoid!) Need measure that reflects ability to discriminate between signal present and signal absent (a good way of) Measuring Sensitivity Present stimulus (signal) on only half of the trials, (test trials). Present no stimulus (noise) on other half of trials, (catch trials). Note that unlike threshold experiments, we present only one stimulus at one intensity. Many trials are presented. For each, the participant says yes, the stimulus is there or no, it isn t

19 Four Possible Results on Each Trial Also Known As... The stimulus is really... The difference is really... Present Absent Present Absent Participant says... Yes, I see it No, I don t Hit Miss False Alarm Correct Rejection Statistical test says... Yes, it s there No, it s not True Positive False Negative (type II error) False Positive (type I error) True Negative Some Example Results Participant says... Albert Yes, I see it No, I don t The stimulus is really... Present (n =100) Absent (n=100) Participant says... Claire Yes, I see it No, I don t Participant says... Present (n =100) Yes, I see it No, I don t The stimulus is really... Absent (n=100) Benny The stimulus is really... Present (n =100) Absent (n=100) Sensitivity The results of such an experiment yield: proportion of hits (P h= N hits / N testtrials) proportion of FAs (P fa= N fa / N catchtrials) For example, for Albert: P h= 90 / 100 =.9 Pfa= 20 / 100 =.2 Note that we could calc proportions of misses and correct rejections too, but these are redundant (Pm = 1-Ph; Pcr = 1-Pfa) 76

20 Questions What is Benny s proportion of hits (P h)? What is his proportion of FA s (Pfa)? How do sensitivity experiments differ in method from threshold-seeking methods? Sensitivity Perfect participant: Ph = 1, Pfa= 0 Participant just guessing: Ph =.5, Pfa=.5 Worst possible participant (perfectly backwards) : P h = 0, P fa= 1 In calculating sensitivity, we want to reward hits and punish FAs, so we could just use Basic Sensitivity : BS = P h - P fa Sensitivity BS equals: Perfect participant: P h of 1 - P FA of 0 = 1 Participant guessing: Ph of.5 - PFA of.5 = 0 Backward participant: Ph of 0 - PFA of 1 = -1 So BS seems to work. However, for obscure statistical reasons, BS is, well, B.S. Instead we calculate d' = z(p h) - z(p FA) Careful! This means function z, do NOT multiply z by PFA) Converting the proportions of hits and FAs to z-scores yields a more valid result. d is measured in standard deviation units. How to calculate the z scores? In Excel, use norminv(p, 0, 1) Table A5.1 from MacMillan & Creelman Or the unit normal table from any stats textbook

21 Normal distribution for finding d' (based on Macmillan & Creelman (2005), Signal Detection: A User s Guide) Sensitivity Link back to data (#91) d' = z(ph) - z(pfa) Albert: P h =.9 z = 1.28; P fa =.2 z = d = (-0.84) = 2.12 Ben: P h =.8 z = 0.84; P fa =.1 z = d = (-1.28) = 2.12 Claire: P h =.9 z = 1.28; P fa =.3 z = d = (-0.52) = Zero and One What to do with proportions of 0 and 1? These yield z scores of and + There are many ways of getting around this (MacMillan & Creelman, 2005, chp. 1). For our purposes, just substitute values of 0.01 and 0.99, respectively. Doubting Dan Participant says... Yes, I see it No, I don t The stimulus is really... Present (n =100) Absent (n=100) Participant says... Flawless Fran Yes, I see it No, I don t Participant says... Present (n =100) Easy Eddie Yes, I see it No, I don t The stimulus is really... Absent (n=100) The stimulus is really... Present (n =100) Absent (n=100)

22 Sensitivity Questions d' = z(ph) - z(pfa) Dan: P h = 0 z = -2.33; P FA = 0 z = d = (-2.33) = 0 (usual minimum) Eddie: P h = 1 z = 2.33; P FA = 1 z = 2.33 d = = 0 (usual minimum) Fran: P h = 1 z = 2.33; P FA = 0 z = d = (-2.33) = 4.66 (conventional max.) Example for self-test: Zack does a sensitivity experiment with 20 test trials and 20 catch trials. He gets 10 hits and 5 false alarms. What are his Ph and Pfa values? What is is d? d =.fg Ph =.ej Pfa=.be Criterion The flip side of sensitivity is criterion or response bias. How do we measure a person s tendency to say yes or no? There are several measures, but the simplest is: c = [z(ph) + z(pfa)] / -2 The lower c, the more a person s tendency to say yes. When c<0, yes > no. When c>0, yes < no. Criterion c = [z(p h) + z(p FA)] / -2 Albert: P h =.9 z = 1.28; P FA =.2 z = c = ( ) / -2 = -.22 [tends to yes ] Ben: P h =.8 z = 0.84; P FA =.1 z = c = ( ) / -2 =.22 [tends to no ] Claire: Ph =.9 z = 1.28; PFA =.3 z = c = ( ) / -2 = -.38 [tends to yes ] Link back to data (#91) 87 88

23 Isosensitivity Curves (a.k.a. ROC curves ) A range of Ph and Pfa values can produce the same d. Ph Pfa z(ph) z(pfa) d c This occurs when sensitivity remains the same, but criterion shifts 89 Isosensitivity Curves Plotting results for different criterion levels at the same sensitivity yields an isosensitivity curve (aka, ROC curve) Hit Rate ROC Curve d' = 1.09 d = 1.09, criterion = d = 1.09, criterion = 0.30 d = 1.09, criterion = False Alarm Rate 90 Hit Rate (Ph) Correct Rejection Rate (Pcr) False Alarm Rate (Pfa) 91.5 Miss Rate (Pm) Why Does Criterion Shift? Many reasons, but an important one is the payoff matrix. If the cost of a false alarm, or reward for a correct rejection, is great, one will tend to make one s criterion stricter (more no ) If the cost of a miss, or reward for a hit, is raised, one will tend to make one s criterion laxer (more yes ) 92

24 Example of a Pay-off Matrix Leading to a Lax Criterion Radiologist Says... Yes, I see it No, I don t Present (n =100) Patient Saved Patient Dies Cancer is... Absent (n=100) Unnecessary Additional Testing Patient Goes Home Participant says... Lax Yes, I see it No, I don t Participant says... The stimulus is really... Present (n =100) Neutral Yes, I see it No, I don t Absent (n=100) +100$ -$10 -$10 +10$ The stimulus is really... Present (n =100) Participant says... Strict Absent (n=100) +10$ -$10 -$10 +10$ Yes, I see it No, I don t The stimulus is really... Present (n =100) Absent (n=100) +10$ -$10 -$ $ Questions Dr. X tests a new field diagnostic procedure by applying it to 50 individuals known to have an illness. He finds that it correctly labels all 50 of them as ill, whereas his old procedure labeled only 40 of them as such. He concludes that the new procedure is better. Is his conclusion sound? Why or why not? Questions Why would one be interested in measuring changes in criterion? True or False: As one travels up and to the right along an isosensitivity curve, criterion rises. True or False: A higher criterion value indicates a stronger tendency to say yes, the stimulus is present 95 96

25 Signal Detection Theory The concepts of sensitivity and criterion arise out of Signal Detection Theory (SDT) SDT suggests that any attempt at detecting a signal (stimulus) has to contend with competing noise Noise in this sense is random variations from the environment (e.g., literal noise) or from within the detector (e.g., neuron chatter) SDT Example: You re in the shower and expecting a call. Signal: Phone ringing Noise: Sound of shower (plus all other sources of sound), as well as your own internal neuron chatter. If the phone isn t ringing, you have just noise. If the phone is ringing, you have signal+noise. d is essentially a measure of how similar the signal is to the noise for you subjectively Probability Distributions of Noise and Signal+Noise Probability That a Given Perceptual Effect is Due to N or S+N noise only phone + noise noise only phone + noise Probability Probability not at all a bit kinda a lot unmistakably not at all a bit kinda a lot completely How much it sounds subjectively like a phone How much it sounds subjectively like a phone 99 Link back to matrix 100

26 Different Criteria One Can Adopt Probability noise only Lax phone + noise Result: Pfa =.50 Ph =.95 d' = 1.64 Different Criteria One Can Adopt Probability noise only phone + noise Strict Result: Pfa =.05 Ph =.50 d' = 1.64 not at all a bit kinda a lot completely not at all a bit kinda a lot completely How much it sounds subjectively like a phone How much it sounds subjectively like a phone Increasing d means increasing the distance betwen the probability distributions. Note how, with a greater d, a criterion somewhere in the neutral area will produce very few misses or false alarms. Decreasing d means decreasing the distance between the probability distributions. Note how, with a smaller d, many misses and/or false alarms result, regardless of where the criterion is placed. Probability noise only d (large) phone + noise Probability d (small) noise only phone + noise not at all a bit kinda a lot completely not at all a bit kinda a lot completely How much it sounds subjectively like a phone 103 How much it sounds subjectively like a phone 104

27 Questions In terms of the SDT, what would be the effect of turning up the phone volume on the probability curves? What effect would this have on d? What about turning down the shower? A guard is watching the woods for visible signs of intruders. What are some likely sources of noise in his situation? For More Info... A nice primer: Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory measures. Behavioral Research Instruments, Methods and Computers, 31, The bible of SDT: Macmillan, N. A., & Creelman, C. D. (2005). Detection Theory: A User's Guide (2nd ed.). Mahwah, N.J.: Lawrence Erlbaum Associates