Sources (highly recommended!) Suggestions for Measuring Psychometric Functions. Why use adaptive procedures? Agenda. An example

Transcription

1 Sources (highly recommended!) Suggestions for Measuring Psychometric Functions Brian P. Keane UMDNJ RWJ Medical School, UBHC Rutgers University Center for Cognitive Science Wichmann & Hill (2001). The psychometric function: I. Fitting, sampling and goodness of fit. Perception & Psychophysics, 63(8), Klein, S.A. (2001). Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63(8), Kingdom, F.A.A., & Prins, N. (2011). Psychophysics: A practical introduction. Boston: Academic Press. Prins, N. & Kingdom, F.A.A. Palamedes: Matlab routines for analyzing psychophysical data. Why use adaptive procedures? Get the most accurate information on one or more aspects of performance (threshold/slope/upper and lower asymptotes) with the fewest trials. Agenda Quickly review basics of psychometric function (PF) What are the different parts of the PF and what do they mean? Discuss common decisions when fitting a function What psychometric shape to assume? What is the appropriate range and spacing of stimulus intensity? When to use MOCS? When to use adaptive procedure? Discuss adaptive procedures QUEST (Watson & Pelli, 1983) Psi method (Kontsevich & Tyler, 2000) Up/down staircases Some tips along the way The psychometric function: A refresher An example Task: Orientation discrimination Psi Probability of a correct (or positive) response x Signal intensity Alpha Position of curve along abscissa (here, at 75%) Beta Slope (although depends on form of psychometric function) Gamma Lower asymptote value (here, guess rate); Lambda 1 upper asymptote (finger error rate) F psychometric function (here, logistic) Cameron, Tai, & Carrasco, 2002

2 Samples of Cumulative Normal PFs Sample of Cumulative Normal PFs Palamedes: StimLevels = [0:.01:4]; pcorrect = PAL_CumulativeNorm all([ ], StimLevels); plot(stimlevels,pcorr ect); Alpha = 1, 2 Beta =1,2,.05 An example: Zhou et al., 2008 The meaning of the PF parameters depend on the set up Task: Discriminate fat/thin illusory shapes or fat/thin amodal shapes Task Difficulty: Larger rotations make task easier Harder Easier The psychometric function: A refresher Sample of cumulative normal PF Zhou et al., 2008 Palamedes: StimLevels = [0:.01:4]; pcorrect = PAL_CumulativeNorm all([ ], StimLevels); <--Fatter Thinner plot(stimlevels,pcorr ect); Alpha PSE Beta slope (or sensitivity) Gamma Lower asymptote (here, lapse error rates for fat shapes); Lambda 1 upper asymptote (here, lapse error rates for thin shapes) F psychometric function (here, cumulative Gaussian) Gamma=0,.05

3 Choices: Which psychometric shape? Cumulative Gaussian y = gamma + (1 gamma lambda)*.5*erfc( beta.*(x alpha)./sqrt(2)); Advantage: Threshold is at curve center (e.g., 75%) Advantage: Beta = inverse of standard deviation Logistic Advantage: Threshold is at curve center (e.g., 75%). Log Weibull (Gumbel) Commonly mislabeled as Weibull (Kingdom & Prins, 2011) Weibull Disadvantage: Changing alpha also changes slope of curve even when beta is held constant. Hyperbolic Secant (rare) Choices: Stimulus range and spacing? Range of x: just above chance to just below 100%. Spacing: Logarithmic or not? Logarithmic is usually best since function often decelerates Zhou et al., 2008 MOCS or adaptive procedure? MOCS is appropriate when: Shape of function is not well known Multiple aspects of the psychometric curve need to be estimated (e.g., alpha, beta, lambda). Goal is to detect small differences in slope Disadvantage: Requires ~40 trials for 5 or 6 stimulus levels. Adaptive procedures are most appropriate when: Lots of conditions Limited number of trials (as with patients) Only want to know threshold or slope. What kind of adaptive method? Options: QUEST (Watson & Pelli, 1983) Psi Method (Kontsevich & Tyler, 1999) Down/up staircase methods (Levitt, 1971) Others that I won t discuss Adaptive Probit Estimation (Watt & Andrews, 1981) PEST (e.g., Pentland, 1980) ZEST (King Smith & Pierce, 1994) QUEST (Watson & Pelli, 1983) On each trial, a PF is fit to all data collected for condition. Fitted PF is used to select stimulus intensity for next trial. Estimates only threshold. Assumes a slope of 2 (in Palamedes) and a Gumbel function. Assumes a lapse error rate (i.e., 2%). Advantage: Apparently robust to poor estimates of slope. Palamedes toolbox functions: Uses mean of posterior probability distribution (King Smith et al., 1994) Uses uniform prior probability distribution (Alcala Quintana & Garcia Perez, 2004) Psi Method (Kontsevich & Tyler, 1999) On each trial, selects stimulus to efficiently estimate threshold and slope. Updates two dimensional posterior probability distribution Selects stimulus for next trial so that expected entropy in posterior distribution will be minimized Requires ~30 trials for reasonable threshold estimate ~250 trials needed for similarly precise slope

4 Psi Method probably the best algorithm for the task and should be looked at carefully (Klein, 2001, p. 1422). arguably the most efficient of the adaptive methods which target both an estimate of the location and the slope (Kingdom & Prins, 2011, p. 143). Down/up staircases Varieties 2 down, 1 up: 70.7% accuracy 3 down, 1 up: 79.4% accuracy 3 down, 1 up (not necessarily consecutive): 75% (Zwislocki & Relkin, 2001) Reversal the sign of the signal change reverses Experiment stops after a fixed number of reversals (e.g., 8) Typical: Average an even number of reversals (e.g., 6), after throwing out the first 2 or 4. 3 down, 1 up Down/up staircases Task: Do you hear a tone? Leek, 2001 Advantages Easy to implement Robust and almost as good as maximum likelihood methods (Klein, 2001) Appropriate inertia (good if threshold is not stationary). Tips If using 2AFC, then don t use 2 down, 1 up (Leek, 2001) Make sure that step size is not too small and that starting point is not too far from threshold (otherwise inefficient) Consider using transformed up down method (begin with 1 down/1 up until first reversal is reached). Klein (2001): When determining threshold, consider averaging stimulus intensity at all levels after tossing out first several reversals. Interleave staircases, so regularities in staircase are less transparent. General tips when estimating PF Try to use more than 2 response alternatives [T]he 2AFC task has a flaw whereby a series of lucky guesses at low stimulus levels can erroneously drive adaptive methods to levels that are too low. Some adaptive methods with a small number of trials are unable to fully recover from a string of lucky guesses early in the run. One solution is to allow the subject to have more than two responses. For reasons that I have never understood, people like to have only two response categories in a 2AFC task. (Klein, 2001, p. 1436) Try to use only a single interval. Yes/no tasks are not any more biased and they can allow more trials to be run in a session. (Klein, 2001) (spatial) 4AFC tasks may be much more efficient at least for naïve observers (Jakel & Wichmann, 2006) Fitting curves for MOCS Curve fitting toolbox Lsqcurvefit yhat=gamma + (1 gamma para(3))*(1 exp( (stimlevels/para(1)).^para(2))); LB=[ Inf 0 0]; %lower bound for fitted vars; UB=[Inf Inf lambda_cap]; %upper bound for fitted vars [para sse residual exitflag] = lsqcurvefit(functiontype, parainit',stimlevels,accuracy, LB, UB); Palamedes toolbox PAL_PFML_Fit [para LL exitflag] = PAL_PFML_Fit(stimlevels,accuracy*num_trials, num_trials*ones(size(stimlevels)),... [parainit(1) parainit(2) gamma parainit(3)], [ ], functiontype, 'lapselimits',[0 lambda_cap]);

5 Formulae for psychometric shapes Cumulative Gaussian y = gamma + (1 gamma lambda)*.5*erfc( beta.*(x alpha)./sqrt(2)); Logistic y = gamma + (1 gamma lambda)*(1./(1+exp( 1*(beta).*(x alpha)))); Log Weibull (Gumbel) y = gamma+(1 gamma lambda).*(1 exp( 1.*10.^(beta.*(x alpha)))); Weibull y = gamma + (1 gamma lambda)*(1 exp( 1*(x./alpha).^beta)); Hyperbolic Secant (rare) y = gamma + (1 gamma lambda)*((2/pi).*atan(exp((pi/2).*beta.*(xalpha))));