Brook's Image Scanning Experiment & Neuropsychological Evidence for Spatial Rehearsal Psychology 355: Cognitive Psychology Instructor: John Miyamoto 04/24/2018: Lecture 05-2 Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren t needed to view the slides. You can disable or delete the macros without any change to the presentation.
Outline Brandimonte's Mental Subtraction Experiment Mental Subtraction + Articulatory Suppression Done! Brook's Image Scanning Experiment: Interference between VSP & PL Neuropsychological evidence for spatial rehearsal in the VSP Similarities and differences between PL and VSP Purpose of Brook's Image Scanning Experiment Psych 355, Miyamoto, Spr 18 2
Next: Brook s Image Scanning Experiment Brooks, L. R. (1968). Spatial and verbal components of the act of recall. Canadian Journal of Psychology, 22, 349-368. Purpose #1: To show that there are 2 different short-term stores: VSP and PL Purpose #2: To show that... Performing two different tasks in VSP interfere with each other. Performing two different tasks in PL interfere with each other. Performing one task in VSP and a different task in PL does not cause as much interference Image Scanning Experiment Psych 355, Miyamoto, Spr '18 3
Image Scanning (Brooks) Experiment has 4 conditions. 2 types of stimulus: Diagrams or Sentences 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode Stimulus Types Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 * There was actually a third response type, tapping, but it will not be discussed here it turned out not to be very informative. Point Out that Goldstein Textbook Omits the Sentence Stimuli Psych 355, Miyamoto, Spr '18 4
Image Scanning (Brooks) Experiment has 4 conditions. 2 types of stimulus: Diagrams or Sentences 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode Stimulus Types Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Same Slide with Remark that Goldstein Omits Sentence Stimuli Psych 355, Miyamoto, Spr '18 5
Image Scanning (Brooks) Experiment has 4 conditions. 2 types of stimulus: Diagrams or Sentences 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode Stimulus Types Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Goldstein's discussion of this study omits the Sentence stimuli. Only the diagram stimuli are discussed in the textbook. Explain Diagram Stimulus x Vocal Response Psych 355, Miyamoto, Spr '18 6
Image Scanning (Brooks) Experiment has 4 conditions. 2 types of stimulus: Diagrams or Sentences 2 types of responses: Pointing or Vocal Response* Dependent variable = time to perform a task Next: Explain the different stimulus types and response modes Response Mode Stimulus Types Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Next Slide Psych 355, Miyamoto, Spr '18 Explain Diagram Stimulus x Vocal Response 7
Image Scanning (Brooks, cont.) Condition: Diagrammatic Stimulus & Vocal Response Memorize the figure. Then it is removed. Start your scan at the star and (mentally) move clockwise around the figure. Say "OUT" when you reach an outside corner.. Say "IN" when you reach an inside corner. Correct response: out, out, out, in, in, out, out, in, out, out, out Dependent variable: Response time (time to complete the task) 2 x 2 Exp Design: Identify Combination of Diagram Stim & Pointing Resp Psych 355, Miyamoto, Spr '18 8
Image Scanning (cont.) Next: Diagrammatic stimulus combined with pointing response: Stimulus Type Response Mode Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Next Slide Previous Slide Psych 355, Miyamoto, Spr '18 Condition: Diagram Stimulus with Pointing Response 9
Image Scanning (Brooks, cont.) Condition: Diagrammatic Stimulus & Pointing Response Stimulus Memorize the figure. Then it is removed. Same as before Start your scan at the * and (mentally) move clockwise around the figure. Same as before. Point to "Out" when you reach an outside corner. Point to "In" when you reach an inside corner.. Correct response: (See diagram) out, out, in, in, out, out, in, out, out, out Dependent variable: Response time (time to complete the task) Respond by pointing to letters on this sheet. Psych 355, Miyamoto, Spr '18 Transition to Next Condition: Sentence Stimulus & Vocal Response 10
Image Scanning (cont.) Explain sentence stimulus combined with vocal response: Stimulus Type Response Mode Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Next Slide Psych 355, Miyamoto, Spr '18 Condition: Sentence Stimulus & Vocal Response 11
Image Scanning (Lee Brooks, cont.) Condition: Sentence Stimulus & Vocal Response Memorize the sentence. Then it is removed. Sentence Stimulus A bird in hand is worth two in the bush. Now work from the beginning to the end of the sentence. Say "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun. Correct response: no, yes, no, yes, no, no, no, no, no, yes Dependent variable: Response time (time to complete the task) Transition to Condition with Sentence Stimulus & Pointing Response Psych 355, Miyamoto, Spr '18 12
Image Scanning (cont.) Condition: Sentence Stimulus & Vocal Response Stimulus Response Mode Type Pointing Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Next Slide Sentence Stimulus x Pointing Response Psych 355, Miyamoto, Spr '18 13
Image Scanning (Lee Brooks, cont.) Condition: Sentence Stimulus & Pointing Response Sentence Stimulus: A bird in hand is worth two in the bush. Memorize the sentence. Then it is removed. Work from the beginning to the end of the sentence. Point to "yes" each time you encounter a noun; say "no" when you encounter a word that is not a noun.. Correct response: no, yes, no, yes, no, no, no, no, no, yes Respond by pointing to letters on this sheet. Psych 355, Miyamoto, Spr '18 Image Scanning Results 14
Results: Image Scanning (Lee Brooks) Stimulus Type Pointing Response Mode Vocal Diagrams 28.2 11.3 Slower Faster Sentences 9.8 13.8 Faster Slower Same Slide: Why is Pointing Slower than Vocal For Diagrams Psych 355, Miyamoto, Spr '18 & Opposite for Sentences? 15
Results: Image Scanning (Lee Brooks) Stimulus Type Pointing Response Mode Vocal Diagrams 28.2 11.3 Slower Faster Sentences 9.8 13.8 Faster Slower Why is diagram/pointing slower than diagram/vocal? Why is sentence/vocal slower than sentence/pointing? Discussion of Results Psych 355, Miyamoto, Spr '18 16
Why is Diagram/Pointing Slower than Diagram/Vocal? Stimulus Type Pointing Response Mode Vocal Diagrams 28.2 11.3 Slower Faster Sentences 9.8 13.8 Diagram/Pointing: The stimulus and response compete for a common cognitive resource, representation in VSP. Information processing bottleneck causes slower response. Diagram/Vocal: Stimulus & response use separate cognitive resources. No bottleneck. Why is sentence/vocal slower than sentence/pointing? Psych 355, Miyamoto, Spr '18 17
Why is Sentence/Vocal Slower than Sentence/Pointing? Stimulus Type Pointing Response Mode Vocal Diagrams 28.2 11.3 Sentences 9.8 13.8 Faster Slower Sentence/Vocal: Again, stimulus and response compete for a common cognitive resource, representation in PL. Information processing bottleneck causes slower response. Sentence/Pointing: Stimulus & response use separate cognitive resources. No bottleneck. Brook s Image Scanning Experiment Supports Existence of Visuospatial Component to WM Psych 355, Miyamoto, Spr '18 18
Brooks' experiment supports the existence of a visuospatial sketch pad (VSP) because... If there were only one STM store (not separate PL & VSP), then the effect of the pointing response would be the same in the visuospatial task and the verbal task. Similarly for the effect of the verbal response. WM model assumes multiple memory stores this lets WM predict conflicts between similar mental codes. The preceding experiment illustrates a basic principle: A response in one modality (verbal or spatial) will interfere more with a memory representation in the same modality than with a memory representation in a different modality. Comment re the Response Sheet for the Pointing Response Psych 355, Miyamoto, Spr '18 19
Response Sheet for the Pointing Response in Brooks' Image Scanning Experiment Condition: Diagrammatic Stimulus & Pointing Response Stimulus Response sheet as displayed in Goldstein Table 5.2 (p. 139) Respond by pointing to letters on this sheet. Same Slide with Comment that the Response Sheet on Right is Correct Psych 355, Miyamoto, Spr '18 20
Response Sheet for the Pointing Response in Brooks' Image Scanning Experiment Condition: Diagrammatic Stimulus & Pointing Response Stimulus Response sheet as displayed in Goldstein Table 5.2 (p. 139) The actual response sheet looked more like this. Why use the irregular display as on the right? Respond by pointing to letters on this sheet. Spatial Rehearsal in Visual Working Memory Psych 355, Miyamoto, Spr '18 21
Spatial Rehearsal in Working Memory What Is It? Spatial Rehearsal maintaining information about a location in working memory Terminology PFC DLPFC Prefrontal Cortex (the front of the frontal lobe of the brain) Dorsolateral Prefrontal Cortex (along the upper outside of the PFC) Delayed-Match-To-Sample (Monkey Study) Psych 355, Miyamoto, Spr '18 22
Delayed-Match-To-Sample Paradigm (a) Cue (b) Delay (c) Reward (a) (b) (c) Monkey sees where food is located. 10 second delay without visual input. Monkey reaches for food (or makes a mistake and reaches in the wrong place). Goldstein refers to this experimental task as the "delayed response task." See Figure 5.23, p. 143. Same Display Single Cell Recordings Find Neural Correlates of (a), (b) & (c) Psych 355, Miyamoto, Spr '18 23
Delayed-Match-To-Sample Paradigm (a) Cue (b) Delay (c) Reward Single-cell recording in monkey PFC shows a correlation between neural activity & the three stages of this task. Some neurons become active during the initial placement of the food. Some neuron become active only during the delay period (spatial rehearsal). Some neurons become active only when the response is made. Same Display Monkeys with PFC Lesions Cannot Do This Task Psych 355, Miyamoto, Spr '18 24
Delayed-Match-To-Sample Paradigm Right Wrong (a) Cue (b) Delay (c) Reward Monkeys with frontal lesions cannot do this task. Human infants cannot do task until about 12 months old. Psych 355, Miyamoto, Spr '18 Multiple Location / Eye Movement Version of Delayed Response Task 25
Monkey Performance on Delayed Eye-Movement Task Same idea as the delayed-match-to-sample task (preceding slide), except that there are multiple locations to remember, and the response is an eye movement. See Goldstein, Figure 5.24 (p. 144). + + + Target Stimulus Delay Period Eye Movement Response The monkey's task is to see the target, wait until a response is permitted, and then move his eyes to the target position. He gets rewarded with a squirt of apple juice. Psych 355, Miyamoto, Spr '18 Single-Cell Recordings When Perceiving the Cue; Maintaining Attention; Responding 26
Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1993). Dorsolateral prefrontal lesions and oculomotor delayedresponse performance: Evidence for mnemonic "scotomas." Journal of Neuroscience, 13, 1479-1497. Neurons in DLPFC show sustained activity during delay period that is selective of the particular location to be remembered. Errors occur when this sustained activity fails to initiate or decays before end of delay period. Specific Lesions Cause Location-Specific Inability to Memory Loss Psych 355, Miyamoto, Spr '18 27
error Lesion Lesions introduced into spatial memory areas cause location-specific inability to retain spatial information during the delay period. (Mnemonic scotoma). Perception for these locations is not impaired, i.e., this is not a blind spot. Memory for other locations is not impaired. Neural Mind Reading Psych 355, Miyamoto, Spr '18 28
Tuesday, April 24, 2018: The Lecture Ended Here Psych 355, Miyamoto, Spr '18 29
Neural Mind of Perceptual Processing Neural Mind Reading - guessing what someone was thinking from the neural response. Kamitani, Y., & Tong, F. (2005). Decoding the visual and subjective contents of the human brain. Nature Neuroscience, 8, 679 685. Neural mind reading of perceptual processing: fmri used to guess which pattern a subject is viewing On each trial, the subject views the pattern on left or the pattern on the right: OR Researcher tries to guess the stimulus based on fmri image taken while subject views the stimulus. Psychologists achieve accuracy of 75% - 100% depending on the stimulus pair. Guessing rate would be 50%. Neural Mind Reading of Rehearsal in VSP Psych 355, Miyamoto, Spr '18 30
Neural Mind Reading of Rehearsal in VSP Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458, 462 465. On each trial, the subject views a sequence of screens. 2 11 seconds Target 1 Target 2 Remember This Target 2 fmri Recording Test Stimulus Is Test Stimulus Clockwise or Counterclockwise? Researcher tries to guess the image held in WM based on fmri image taken while subject retains the memory of the cued stimulus. Psychologists achieve accuracy of 80% (average). fmri successfully detected VSP rehearsal of the target image. Graph Showing Brain Activity in Visual Cortex While Retaining Orientation Info Psych 355, Miyamoto, Spr '18 31
fmri Measure of Brain Activity During Stimulus Trial Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458, 462 465. Stimulus Sequence The graph (b) shows brain activity during the retention period (and after the retention period - not relevant to our current topic). BOLD contrast is strong in targeted areas of visual cortex during the retention period (11 seconds), even though there is no visual stimulus. Brain Activity in Visual Cortex During 11 Second Retention The fact that the activity was in the visual cortex supports interpretation that what we are observing is spatial rehearsal. Psych 355, Miyamoto, Spr '18 Summary: Neuropsych Evidence for WM Components 32
Psych 355, Miyamoto, Spr '18 33 Table Comparing PL to VSP - Overview of Layout Neuropsychological Evidence for WM Components There is a short-term memory store (VSP) that is specific for visual imagery and spatial representations. Evidence for spatial rehearsal in VSP Single-cell recordings in monkey Imaging studies of phonological loop: No one brain locus shows increased activity during rehearsal. Multiple brain areas that are associated with language show increased activity. Evidence from cognitive impairments
Summary: Comparison Between PL & VSP Phonological Loop Visuospatial Sketchpad Common Characteristics Common Characteristics Differences Differences Psych 355, Miyamoto, Win '13 34 Comparison of PL & VSP - END
Summary: Comparison Between PL & VSP Phonological Loop Limited capacity Information is retained by an active process (verbal rehearsal) Information is highly accessible Multiple verbal tasks interfere with each other. Visuospatial Sketchpad Limited capacity Information is retained by an active process (manipulation of mental imagery) Information is highly accessible Multiple visual tasks interfere with each other. Verbal Task + Visual Task produces less interference than Verbal Task + Verbal Task or Visual Task + Visual Task PL processes linguistic information; VSP processes visual imagery and spatial information. Different areas of the brain show enhanced activity during verbal and visual/spatial rehearsal. Psych 355, Miyamoto, Spr '18 35 END