Decision Processes in Memory & Perception

Transcription

1 Decision Processes in Memory & Perception outline PSYCH 720 Contemporary Problems Jan-Feb, 2018 my background: where I m coming from psychophysics classical threshold theory signal detection theory 1 2 Sensation vs. Perception Is consciousness a useful concept for studying perception? tion. It has been maintained by its followers generally that psychology is a study of the science of the phenomena of consciousness. It has taken as its problem, on the one hand, the' analysis of complex mental states (or processes) into simple elementary constituents, and on the other the construction of complex states when the elementary constituents are given. The world of physical objects (stimuli, including here anything of such a 'will o' the wisp' character. One can assume either the presence or the absence of consciousness anywhere in the phylogenetic scale without affecting the problems of behavior by one jot or one tittle; and without influencing in any way the mode of experimental attack upon them. On the other hand, I cannot for one moment assume that the paramecium 3 4

2 Psychophysics can refer to a set of methods, specific scientific goals/ areas, and/or to a general approach to studying behaviour characterized by: - simple tasks (often to measure thresholds) - careful control (and detailed description) of stimuli - explanations that emphasize stimulus characteristics and simple internal mechanisms/processing simplify the job of the homunculus Centrality of the stimulus in psychophysics responses. In a system of psychology completely worked out, given the response the stimuli can be predicted; given the stimuli the response can be predicted. Such a set of statements is crass and raw in the extreme, as all such generaliza- To what extent can we base explanations of behaviour on the stimulus? How much can/should we rely on unobservable internal states/variables? 5 6 Absolute Detection Threshold for Vision Are 2 lights the same or different? results of colour matching experiments yielded important insights into processing Frequency-of-seeing curves from Hecht, Shlaer, & Pirenne (1942) Rod receptor responses to individual photons. Baylor, Lamb, & Yau (1979) S (blue) cones M (green) cones L (red) cones Denis Baylor, Proctor Lecture, 1987 spectral sensitivity curves measured in behavioural experiments agree closely with physiological measurements of cone responses 7 8

3 Is psychophysics important for real world problems? Why emphasize thresholds when much of perception is supra-threshold? What can be learned from such simple tasks? binary diagnostic problems Was a stimulus presented? Did I hear this item before? Was that a word or a non-word? Is a cancer present in this patient? Will this individual commit a violent act? Does this luggage contain explosives? Is this aircraft safe to fly? Will the Dow Jones go up today? Is this assembly-line item flawed? Does this tax return justify an audit? 9 10 implications simple detection task performance depends on two factors - quality of evidence - rule for generating response how can we untangle them? need a theory of the underlying evidence & decision rule - N.B. this is always the case, even if we are not aware of our theoretical assumptions perhaps psychophysical tasks can be used to study decision making in important, naturalistic tasks % SEEN responses % yes responses stimulus strength 11 12

4 Classical Threshold Theory Psychometric Function Sharp boundary between always conscious and always unconscious events Herbart (1824): the threshold of consciousness is the boundary which an idea appears to cross as it passes from the totally inhibited state into some degree of actual ideation. See Corso, J.F. Psychological Bulletin, % stimuli seen Stimulus intensity Observed Psychometric Function Explanation of Smooth Curve Top performance often below 100% - mistakes (i.e., pressing wrong button) Smooth, gradual shape; not sharp - Moment-to-moment fluctuations in threshold False alarms: observers see stimuli on blank/catch trials - guessing Gaussian distribution of threshold states predicts cumulative normal form of psychometric function

5 Predicting % Seen What about weak stimuli? Gaussian distribution shows probability of threshold being a particular value % seen for each stimulus i equals the % of trials on which threshold is less than or equal to i So, psychometric function should trace out a cumulative normal distribution Most psychometric functions level out at about 1-5% seen they typically do not go to zero Why? Perhaps threshold varies a lot, and there is a nonzero probability of threshold being very low But, that can t be the whole story, because observers make seen responses on catch trials So, some seen responses must be due to guessing Guessing reflects non-sensory effects Guessing Does Happen P(stimulus) alters behaviour known for a long time that guessing does occur non-sensory factors can contribute to guessing - P(stimulus) - Payoff Matrix 19 20

6 Payoff Matrix values/rewards for each stimulus-response combination Effect of Payoff Matrix Response yes no stimulus 5-2 no stimulus -2 5 P( seen ) when stimulus was presented. P( seen ) when stimulus was not presented Summary of Classical Theory Correction for Guessing Classical theory assumes that evidence is binary above threshold: always seen below threshold: never seen also assumes decision rule is binary above threshold: say seen below threshold: guess, with some P of seen Classical (high-threshold) theory assumes that False Alarms are always guesses (i.e., that FA & Hits arise from qualitatively different processes) P(seen) is a combination of true seen responses and guesses on true not seen trials so, P(seen) is combination of sensory and non-sensory factors How can we extract true P(seen) from measured P(seen)? subtraction P(seen) = P(seen*) + FA{1-P(seen*)} P(seen) = measured probability of seen 1-P(seen) = measured probability of not seen P(seen*) = true probability of seen response FA = false alarm rate = P(seen) on catch trials = guessing rate 1-FA = true probability not seen responses P(seen*) = {P(seen) - FA} / {1-FA} 23 24

7 ROC Predictions P(seen) = P(seen*) + FA{1-P(seen*)} Equation specifies a relation between P(seen), or HITS, to FA So, theory makes a prediction about ROC ROC: Receiver Operating Characteristic ROC: Relative Operating Characteristic relates p(seen stimulus), or HITS, to p(seen no stimulus), or FA Specifically, P(seen) should be a linear function of FA True Hits = 0.9 (measured hit rate) True Hits = 0.1 (guessing rate) P(seen) = P(seen*) + FA{1-P(seen*)} y = b + x*m (b & m are constants) Family of ROC curves predicted from high-threshold theory. Each line represents prediction for observers that differ in sensitivity {i.e., different P(seen*)}. Each line represents an iso-sensitivity curve and shows how the observed hit rate changes as a function of guessing when sensitivity is the same ROCs on z-axes Each probability, e.g. P(HIT) and P(FA), can be transformed into a z score For reasons that will become obvious, it is often convenient to plot ROCs after the proportions have been transformed into z scores On z-score axes, the ROC curves predicted by threshold theory look like this... ROCs generated by high-threshold theory for different values of P(seen*) 27 28

8 simpler correction for guessing a simpler, more common form to correct for guessing is to subtract FA from HITS P(seen*) = P(seen) - FA = HITS - FA this formula also relates HITS & FA, and so predicts the form of ROC double threshold theory (Swets, 1986) internal events associated with A that always exceed high threshold (say A ) events associated with B that never exceed a low threshold (say B ) events due to either A or B that lie in-between thresholds (guess) 29 ROC for double-threshold model Family of ROC curves predicted from double-threshold theory. Each line represents prediction for observers that differ in sensitivity {i.e., different P(seen*)}. Each line represents an iso-sensitivity curve and shows how the observed hit rate changes as a function of guessing when sensitivity is the same. 30 ROC for double-threshold model other detection scores Family of ROC curves predicted from double-threshold theory. Each line represents prediction for observers that differ in sensitivity {i.e., different P(seen*)}. Each line represents an iso-sensitivity curve and shows how the observed hit rate changes as a function of guessing when sensitivity is the same. 31 there are other ways of correcting for guessing all attempt to remove the effects of guessing to obtain a pure measure of sensitivity NONE of the measures are assumption-free - all make predictions about ROC curves - all make assumptions about the nature of internal responses (i.e., evidence) and rules for translating them into overt responses most make predictions that are qualitatively similar to ones made by high-threshold and double-threshold theories - Swets (1986; Psychological Bulletin) 32

9 Empirical ROC curves In a variety of tasks, empirical ROC differ qualitatively from the ones predicted by threshold theories (Swets, 1986) - Empirical ROCs are curved when plotted on linear axes (the predictions are straight lines) - and straight on z-axes (when the predictions are curved lines). ROCs plotted in z coordinates from 4 Ss in a visual detection study (Swets, Tanner, Birdsall, 1955) ROCs plotted in z coordinates from 1 subject for old-new word recognition memory. Word was shown once or twice (r=1 or 2) during study phase (Egan, 1958). ROCs plotted from 15 Ss in an old-new odor recognition memory experiment measured at 3 retention intervals (Rabin and Cain, 1984)

10 ROCs plotted for 2 Ss in dot location categorization task (Lee 1963). Z (H) Z (FA) ROCs from 2 pigeons in a wavelength discrimination task. Each curve shows the probability that a given number (i) of response or fewer were made to a reinforced (S D ) and non-reinforced (S λ ) wavelength (Blough, 1967) Z (H) Z (H) Z (FA) Z (FA) ROC from two groups of 6 radiologists in a lesion detection study. The ROCs were gathered from two different image modalities (CT= computed tomography; RN - radionuclide scans). A z is the area under the z- transformed ROC, and s is the slope of the best-fitting line. Data from Swets et al. (1979), Assessment of diagnostic technologies, Science, 205, p ROC from 10 cytotechnologists who viewed approximately 6,000 individual cell photomicrographs to discriminate between abnormal and normal cells in screening for cervical cancer. From Bacus et al. (1984), Malignant cell detection and cervical cancer screening, Analytical and Quantitative Cytology, 6, p

11 threshold models fail empirical ROC do not match predictions this failure means that threshold-based correction-forguessing schemes DO NOT WORK - measures of performance are not pure - observed differences in sensitivity might be due to bias - also, real differences in sensitivity might be missed this failure to correct for guessing is particularly important for studies of aging, because young and old observers often differ in response bias need a different theory signal detection theory assumes that evidence, internal response, takes the form of a continuous variables (the decision variables) decision variables are continuous, random variables - Gaussian assumption - constant variance model variable variance model - decision variables form a decision space a criterion divides decision space into distinct regions - internal responses leading to response A - internal responses leading to response B response bias reflects the location of the criterion sensitivity reflects the overlap between the distributions (d ) signal detection theory P(False Alarms) FA equals area under target-absent distribution to the right of the criterion

12 P(Hits) Calculating d from raw data d = (μ1- μ2)/σ We don t know the parameters of underlying distributions, but we do know the Hit and False Alarm rates. Hit & False Alarms can be converted into distances using the relation between cumulative proportions and z-scores HITS equals area under target-present distribution to the right of the criterion Calculate distance between means & criterion d = z(h) - z(fa) ƒ(n) ƒ(sn) z(cr) = -z(fa) -z(miss)=z(h) ƒ(n) ƒ(sn) µ 1 µ 2 β = ƒ(sn)/ƒ(n) µ 1 µ 2 The standard distance between u1 and Beta is related to P(CR) & P(FA). The standard distance between u2 & Beta is related to P(Miss) & P(Hit). d = z(h) + -z(fa) β = ƒ(sn)/ƒ(n) = ƒ(h)/ƒ(fa) 47 48

13 sensitivity reflects distribution overlap Adjusting the location of the criterion changes the HIT and FALSE alarm rates in a particular way to generate predictions about ROC curves. d = (μsn - μn) / σ differences in d d is constant for all points on each line 51 52

14 Linear ROC on z axes Non-constant variance model Slope = 1 d =z(h)-z(fa), so z(h) = d + z(fa) d is constant for all points on each line Non-constant variance model predicts ROCs that are asymmetrical in linear coordinates (left) and have a slope that is less than 1 in z coordinates (right) Signal Detection Theory Information from 2nd choices Which of the 4 locations contained the target? Predicts correct shape of ROC: - curvilinear ROCs in linear coordinates - linear ROCs in z coordinates - this means that measure of sensitivity (d ) is approximately invariant despite changes in response bias/criterion Also, correctly relates performance (e.g., % correct) in different tasks (e.g., m-alternative forced-choice tasks, detection vs. identification) 4 AFC detection task: List 1st & 2nd choices How should you do this task? SDT: pick 2 most-likely locations High-Threshold theory: pick location that is above threshold and then guess randomly On incorrect trials, is 2nd choice above chance? Yes: information about target location is contained in below threshold responses consistent with idea that internal response is graded/continuous 55 56

15 Confidence Ratings Confidence Ratings Generate an ROC convert ratings into yes/no responses c 1, c 2, c 3 & c 4 c 2, c 3 & c 4 P(H) c 3 & c 4 c 4 P(FA) Rating scale with N levels requires (N-1) criteria. 57 Treat rating task like a yes/no task. First count a 5 rating as a yes and 1-4 as no. Next count 4 & 5 as yes and 1-3 as no ; then count 3-5 as yes and 1-2 as no. Finally, count 4-5 as yes and 1 as no. 58 ROCs from confidence intervals are lawful though not always the same as those generated by varying p(stim) and/or payoff matrix. Clearly, Ss have access to graded information in internal response Z Normal Deviate 0 Psychonomic Bulletin & Review 2003,10 (3), Uncertainty in pigeons LESLIE M. SOLE, SARA J. SHETTLEWORTH, and PATRICK J. BENNETT University of Toronto, Toronto, Ontario, Canada P N (A) Normal Deviate Normal Deviate PcN(A) f oe a p(correct) : filled p(safe key): open Observer 1 Rating Data x Yes-No Data 0.01 _L PN(A) Observer 4 Rating Data x Yes-No Data P N (A) meta-cognition in pigeons? Application of SDT to situations involving uncertainty & meta-cognition 59 60

16 The Neural Basis of Decision Making Joshua I. Gold 1 and Michael N. Shadlen 2 In the world In the brain summary Context e.g., instruction Contextual cues and prexisting knowledge Two possible states {up,down } One state holds e.g., up Information flow for each decision Consequence of action & state (4 possible) One action e.g., answer up Sensory data x = { x 1, x 2,... } Useful form of evidence e Decision variable l 12(e) Peh e.g., y... ( 1 ) Peh ( 2 ) or loglr 12 log[ l 12(e) ] Apply decision rule e.g., choose left up if l 12(e) criterion Motivation to perform the task Consideration of two propositions (hypotheses) h 1:up or h 2 : down Statistical knowledge likelihoods: Peh ( i ) priors: P( h i ) values: v( H j h i ) Establish decision rule based on goals signal detection theory accounts for many phenomena - shape of ROCs - accuracy on many psychophysical tasks - graded information content of internal responses SDT & psychophyical tasks may help to understand decision processes in higher-order tasks Evaluation Experience payoff or cost value or utility v ij {1,2} Annual Review of Neuroscience, 2007, 30,