Control of visuo-spatial attention Emiliano Macaluso
CB demo
Attention Limited processing resources Overwhelming sensory input cannot be fully processed => SELECTIVE PROCESSING
Selection via spatial orienting time Shifts of gaze and/or covert attention to select relevant locations Sequential processing of relevant information for in-depth analysis
OVERVIEW BEHAVUIOURAL PARADIGMS Visual search Spatial cueing NEURAL SUBSTRATES Hemi-spatial Neglect Electrophysiology (non-human primates) ATTENTION NEWORKS IN HUMANS Dorsal and ventral attention networks Source-Site model of attention control Interactions between networks nodes
Behavioral paradigms: visual search CONJUNCTION SERACH POP-OUT (FEATURE) SERACH o x o x o x o o o o x x x o o Search for the red X x o o x x o o o x o x o o x o Result: => Pop-out search is fast and does not depend on the number of distractors (stimulus-driven orienting) => Conjunction of features requires topdown attention to the items
Behavioral paradigms: spatial cueing (Posner 80) ENDOGENOUS ATTENTION: based on voluntary decisions (e.g. strategy, goals, knowledge) EXOGENOUS ATTENTION: based on incoming signal (e.g. sudden onsets, high contrast) Top-down control by internal signals Bottom-up control by external signals Standard spatial cueing paradigms > cue + + x target + x
Behavioral paradigms: spatial cueing (Posner 80) Valid trial Fixation time X CUE TARGET
Behavioral paradigms: spatial cueing (Posner 80) Valid trial Invalid trial Fixation time X CUE TARGET X
Behavioral paradigms: spatial cueing (Posner 80) Valid trial Invalid trial Fixation CUE time X TARGET X Result: Covert discrimination is faster and more accurate for valid trials than invalid trials Interpretation: - Benefits of endogenous attention at the attended side (valid trials) - Costs of stimulus-driven shift of attention from cued to target location (invalid trials)
Neural substrates: hemi-spatial neglect Failure to orient (attend) towards the contralesional side of space Typically following lesion of the right ventral fronto-parietal network
Neural substrates: hemi-spatial neglect Visual search Spatial cueing Search for T target among L-distractors Leftward bias in overt spatial orienting Selective deficit for left invalid trials (i.e. cue right, target left) => disengagement deficit
Neural substrates: electrophysiology (non-human) The main players: LIP (lateral intraparietal sulcus) FEF (frontal eye-feields) Frontal node Visual areas: V1-V2-V4 MT Parietal node
Neural substrates: electrophysiology (non-human) Activity in parietal cortex (LIP) reflects attention, rather than movement preparation Small sensory response: stimulus irrelevant / unattended Large response, before the eye-movement: stimulus is relevant/attended! Large response, with arm rather than eye-movement: stimulus still relevant/attended Eg. Colby and Goldberg 1999
Neural substrates: electrophysiology (non-human) Attention modulates sensory responses in the visual cortex Neuron in area V2 Neuron in area V4 Review Treue 2001 Moran and Desimone, 1985 Note: visual input does not change, what changes is the focus of covert attention
preferred orientation Neural substrates: electrophysiology (non-human) Competition (bottom-up) between two stimuli in the neuron s RF reduces the response to the preferred stimulus Top-down attention modulate these competitive interactions, e.g. => Based competition model of attention (Desimone 1998)
Tasks and substrates: summary Visual search and spatial cueing Attention selects relevant information Multiple types of signals: endogenous (voluntary) & exogenous (stimulus-driven) Neural Substrates Neglect: leftward bias after lesion of the right fronto-parietal cortex Attention modulates activity in: fronto-parietal cortex (FEF and LIP) & visual occipital cortex
OVERVIEW BEHAVUIOURAL PARADIGMS Visual search Spatial cueing NEURAL SUBSTRATES Hemi-spatial Neglect Electrophysiology (non-human primates) ATTENTION NEWORKS IN HUMANS Dorsal and ventral attention networks Source-Site model of attention control Interactions between networks nodes
Selection via spatial orienting time Shifts of gaze and/or covert attention to select relevant locations Sequential processing of relevant information for in-depth analysis
Attention network for attention shifting => predictable shifts of attention: voluntary/endogenous control? Frontal cortex (FEF) FEF IPS Parietal cortex (IPS) => The dorsal fronto-parietal network Corbetta et al. 1993
Attention shifting: endogenous vs. exogenous control Search: goal-driven (endogenous) orienting Eye-position (n. fixations) + Serial processing during search Target :
Attention shifting: endogenous vs. exogenous control Search: goal-driven (endogenous) orienting Eye-position (n. fixations) + Serial processing during search Target : Track: stimulus-driven (exogenous) orienting Eye-position (n. fixations) +
Attention shifting: endogenous vs. exogenous control Search: goal-driven (endogenous) orienting IPS/PPC FEF => The dorsal fronto-parietal network Vis Track: stimulus-driven (exogenous) orienting TPJ => The ventral fronto-parietal network (posterior node only, here)
Attention shifting: spatial cueing Result: Covert discrimination is faster and more accurate for valid trials than invalid trials Interpretation: - Benefits of endogenous attention at the attended side (valid trials) - Costs of stimulus-driven shift of attention from cued to target location (invalid trials)
fmri using spatial cueing paradigms Central detection Blocked design: valid & invalid vs. central attention Parietal cortex (P) Frontal eye-field (F) Superior temporal sulcus (ST) Temporo-occipital junction (TO) Precuneus (PCU) => The dorsal fronto-parietal network, but endo/exo mixed-up Gitelman et al. 1999
Task analysis Multiple processes occur during this type of task Left Right CUE-related voluntary (endogenous) processes (voluntary shift + sustained delay)
Task analysis Multiple processes occur during this type of task Left Time Left Right Right Valid trial: Facilitation sensory processing CUE-related voluntary (endogenous) processes (voluntary shift + sustained delay)
Task analysis Multiple processes occur during this type of task Left Time Left Right Right Valid trial: Facilitation sensory processing CUE-related voluntary (endogenous) processes (voluntary shift + sustained delay) Left Right Time Invalid trial: Spatial re-orienting ( stimulus-driven )
fmri of spatial re-orienting Event-related fmri: direct comparison invalid > valid trials Valid trial Invalid trial > + x x > + cue-related (endogenous) effects will cancel out Arrington et al. (2002) Corbetta and Shulman (2002) Macaluso et al. (2002) TPJ: temporo-parietal junction & IFG: inferior frontal gyurs => The ventral fronto-parietal network (exogenous control)
Facilitation at the attended side Pioneering PET study of endogenous spatial attention (Heinze at al.,1994) Stimuli on both sides, sustained (blocked) attention to one side => Modulation of contralateral visual activity during endogenous control
% signal change Cue-related, preparatory activity Participants are CUED to one location, prior to the onset of the target stimulus Dorsal fronto-parietal network Preparatory Attention Attention + Stimulation Cue Stimulation Time => Preparatory shifts of baseline activity, in dorsal FP-network Kastner et al. 1999
% signal change % signal change Cue-related, preparatory activity Dorsal fronto-parietal network Preparatory Attention Attention + Stimulation Cue Stimulation Time Sensory visual cortex V 4 Time (sec) => Preparatory shifts also in VISUAL occipital cortex => Possible role of back-projections Kastner et al. 1999
Visuo-spatial attention control: interim summary Multiple processes for attentional control Endogenous orienting (cue) : Dorsal fronto-parietal Network Exogenous re-orienting (invalid trials) : Ventral fronto-parietal Network Sensory facilitation (valid trials) : Occipital Cortex Cue-related preparatory activity: Dorsal FP-net + occipital cortex Dorsal FP-net Ventral FP-net Visual, sensory
Visuo-spatial attention control: interim summary Left Right Invalid trial Spatial re-orienting => Ventral FP-net Macaluso et al., 2002; Macaluso et al., 2003
Visuo-spatial attention control: interim summary Valid trial Left Right Modulation of sensory input => sensory areas Invalid trial Spatial re-orienting => Ventral FP-net Macaluso et al., 2002; Macaluso et al., 2003
Visuo-spatial attention control: interim summary Valid trial Left Right Preparatory activity Modulation of sensory input => sensory areas Invalid trial => Dorsal FP-net Spatial re-orienting => Ventral FP-net Macaluso et al., 2002; Macaluso et al., 2003
Visuo-spatial attention control: interim summary BUT ALSO : Left Right 4 2 0 Preparatory activity Attend: Target: -2 L R L R Present ABSENT Preparatory Effects in Sensory Areas => Dorsal FP-net Macaluso et al., 2002; Macaluso et al., 2003
SITE SOURCE Source-Site model of attention control Dorsal Fronto-Parietal Network (PPC, FEF) Ventral Fronto-Parietal Network (TPJ, IFG) Sensory Areas (Occipital Cortex) Modified from Corbetta & Schulman, 2002; see also Kastner et al., 1999; plus Macaluso & Driver, 2000
SITE SOURCE Source-Site model of attention control Top-down Control (endogenous) Dorsal Fronto-Parietal Network (PPC, FEF) Ventral Fronto-Parietal Network (TPJ, IFG) Preparation Expectation Intention Sensory Areas (Occipital Cortex) External world Modified from Corbetta & Schulman, 2002; see also Kastner et al., 1999; plus Macaluso & Driver, 2000
SITE SOURCE Source-Site model of attention control Top-down Control (endogenous) Dorsal Fronto-Parietal Network (PPC, FEF) Bottom-up Control (exogenous) Ventral Fronto-Parietal Network (TPJ, IFG) Preparation Expectation Intention Sensory Areas (Occipital Cortex) Salience Relevance External world Modified from Corbetta & Schulman, 2002; see also Kastner et al., 1999; plus Macaluso & Driver, 2000
Control of visuo-spatial attention Attention is a dynamic mechanism, NOT the function of a single brain area!!! Any direct evidence for dynamic changes within the Source-Site Network? ==> Use fmri to study task-dependent changes of functional connectivity between brain areas
y: right M1 Inter-regional connectivity Correlation between "BOLD in area A and BOLD in area B Example (at Rest): - Seed in left M1 => reveals right M1 2 Predictor / seed : BOLD in the left M1 12 01 Correlation / connectivity -10 0 5 10 15 20 25 30 35 40 45-1 2 0 5 10 15 20 25 30 35 40 45 2 0 0 Data: "here BOLD in right M1" -2 0 5 10 15 20 25 30 35 40 45-2 0 5 10 15 20 25 30 35 40 45 scans 2 1.5 1 0.5 0-0.5-1 -1.5-2 -1-0.5 0 0.5 1 1.5 X: left M1
Changes of inter-regional connectivity Spatial cueing paradigm Cue: Left / Right Target: Left / Right Invalid - Valid Inferior Parietal Cortex Target Right - Left Left Occipital Cortex Indovina & Macaluso, 2004
Changes of inter-regional connectivity Spatial cueing paradigm Cue: Left / Right Target: Left / Right Inferior Parietal Cortex Invalid trials Valid trials Invalid - Valid Inferior Parietal Cortex Occipital Cortex Occipital Signal Target Right - Left Increased connectivity between occipital areas and inferior parietal cortex during invalid trials Left Occipital Cortex
Bilinear state equation state changes intrinsic connectivity m external inputs system state direct inputs Cu z B u A z m j j j ) ( 1 m nm n m n m j j nn j n j n j j nn n n n u u c c c c z z b b b b u a a a a z z 1 1 1 11 1 1 1 1 11 1 1 11 1 modulation of connectivity state changes intrinsic connectivity m external inputs system state direct inputs Cu z B u A z m j j j ) ( 1 m nm n m n m j j nn j n j n j j nn n n n u u c c c c z z b b b b u a a a a z z 1 1 1 11 1 1 1 1 11 1 1 11 1 modulation of connectivity Z 1 Z 2 a 12 a 21 b 2 21 u 1 u 2 c 11 "stimuli" "context" The influence one system exerts over another (Friston et al., 1993) => directional / causal effects A-priori specification of stimuli / task (design) and network s areas Effective connectivity
Causal influences, during attention control Spatial cueing task with valid and invalid trials judge orientation of the patch circled in green Vossel et al. 2012
Causal influences: top-down effects on visual cortex Top-down modulation from IPS (dorsal FP-net) to VIS cortex boosts contralateral & suppresses ipsilateral
Causal influences: bottom-up effects for invalid trials Bottom-up activation from VIS to TPJ (ventral FP-net) by invalid targets exogenous signals that further propagate to dorsal regions
Control of visuo-spatial attention Attention is a dynamic mechanism, NOT the function of a single brain area!!! Additional evidence for interactions between the nodes of the attention networks: A. Effect of frontal stimulation on visual activity in occipital cortex B. Functional connectivity between dfp and vfp in Neglect
Interactions between FEF and VIS: electrophysiology The main players: LIP (lateral intraparietal sulcus) FEF (frontal eye-feields) Visual areas: V1-V2-V4 MT
Interactions between FEF and VIS: electrophysiology Micro-stimulation of FEF => record visual responses in V4 Visual activity in V4 Motor effect after suprathresholds FEF stimulation Moore and Armstrong, 2003
Interactions between FEF and VIS: electrophysiology Enhanced visual responses in V4, with sub-threshold FEF stimulation (no eye-movement) = preferred stimulus in the neuron s RF FEF as a source of the modulatory (top-down) influences on visual cortex Moore and Armstrong, 2003
Interactions between FEF and VIS: TMS in humans TMS on FEF, while recording visual responses with fmri
Interactions between FEF and VIS: TMS in humans FEF stimulation enhances visual responses (only at peripheral locations) FEF as a source of (top-down) influences on visual cortex!
Interactions between dfp and vfp: Neglect Typically following lesion of the right ventral fronto-parietal network => Dorsal network relatively spared, but
Interactions between dfp and vfp: Neglect Lesion anatomy (n = 11; purple, damaged in one to three patients; blue, damaged in four to seven patients) dorsal FP, green ventral FP, red Test pair-wise connectivity (correlation) within- and betweenattention control networks + link behavioral deficits in spatial cuing task He et al., 2007
Interactions between dfp and vfp: Neglect Reduced connectivity within the dorsal FP: L-R pips which correlates with the behavioral reorienting deficit! => Dysfunction of the intact dorsal (endogenous) system He et al., 2007
Interactions between dfp and vfp: Neglect Reduced connectivity within the ventral FP: L-R STS Acute: L VF, Invalid which also correlates with the behavioral reorienting deficit! He et al., 2007
Interactions between dfp and vfp: Neglect Links between reduced connectivity in the two systems Possible role of the MFG as a communication hub between the dorsal and the ventral attention systems He et al., 2007
An updated model of dfp-vfp control Corbetta and Shulman 2002 Corbetta et al., 2008 BUT see also (e.g.) Vossel et al, 2012.
Summary Multiple processes for attentional control Endogenous orienting (cue) : Dorsal fronto-parietal Network Exogenous re-orienting (invalid trials) : Ventral fronto-parietal Network Sensory facilitation (valid trials): Occipital Cortex Cue-related preparatory activity: Dorsal FP-net + occipital cortex Source-Site Model Endogenous control: Top-down signals (dorsal network) Exogenous control: Bottom-up signals (ventral network) Dynamic changes of connectivity between areas ( source ) and sensory areas ( site ) Dynamic interactions within/between dorsal and ventral source regions
CONCLUSIONS Attention selects relevant information Endogenous & exogenous signals Primary role of the fronto-parietal cortex (dfp & vfp) Source-site model of attention control Interactions between the nodes of the attention system Attention is a dynamic mechanism, NOT the function of a single brain area!!!