Visual Processing of Bodies Corps et cognition: l'embodiment Corrado Corradi-Dell Acqua corrado.corradi@unige.ch Theory of Pain Laboratory Summary Visual Processing of Bodies Category-specificity in ventral visual system FFA: localization and functional properties EBA and FBA: localization and functional properties Body Parts organization in the human brain Multiple Body Representations The Body is: 1. An egocentric reference frame for directing our actions in the external space Body Schema 2. An object situated in the external space with invariant properties Body Image/ Body Structural Description/ 1
The Body is something we see The body is a highlyfamiliar stimulus that it is seen in everyday contact with other members of one s family, friends, colleagues, etc.. Two-way visual system Mishkin M, & Ungerleider LG (1982). Behav Brain Res Goodale MA & Milner AD (1992). Trends in Neuroscience Infamous fmri Throughout the scientific community, neuroimaging techniques such as fmri have been often criticized as: They didn t lead to the expected theoretical advance in our understanding of brain cognition They blocked ongoing psychological research, as most of the resources have been devoided to fmri. One (of few) notable exceptions: the visual ventral system is organized in a (pseudo) category-selective fashion. 2
Lateral Occipital Cortex Malach et al. (1995). PNAS Lateral Occipital Cortex Malach et al. (1995). PNAS V1 LOC Lateral Occipital Cortex Grill-Spector et al. (2001). Vision Res. 3
Not any stimulus, but a specific stimulus Bodies Faces Buildings/ Places Kanwisher & Dilks (2013). Organization of ventral visual pathways in humans Summary Visual Processing of Bodies Category-specificity in ventral visual system FFA: localization and functional properties EBA and FBA: localization and functional properties Body Parts organization in the human brain IT cortex of macaque Single cells in macaque s IT are sensitive to face stimuli in a category-selective fashion De Simone R et al. (1984). Journal of Neuroscience 4
IT cortex of macaque Wachsmuth E et al. (1994). Cerebral Cortex Intracranial Recordings ~170-200 ms Subdural electrods from epileptic patients with pharmacologically intractable epileptic seizures Allison et al. (1999). Cerebral Cortex McCarthy et al. (1999). Cerebral Cortex N170 component EEG MEG Kanwisher et al. (2002). Nat Neurosci Thierry et al. (2006). NeuroImage 5
Prosopoagnosia Acquired inability to recognize faces from vision (including close family members) Recognition triggered by alternative means: (ex.: voice) Good visual acuity Adequate semantic information about the people who cannot be recognized While traveling, I had a stopover at O'Hare and I was approached by a stranger in the lounge area. It took 10-15 seconds of casual conversation before realizing who it was. It was my brother. Prosopoagnosia Sometimes Prosopoagnosia is observed following very focal lesions Wada & Yamamoto (2001). JNNP IT cortex, a human homologue? 6
Fusiform Face Area Kanwisher N et al. (1997). Journal of Neuroscience Fusiform Face Area Non sensitive to objects vs. non-objects Sensitive to faces > object Kanwisher N et al. (1997). Journal of Neuroscience Fusiform Face Area Popular control: houses and locations Evidence for different anatomical specificities within the ventral occipital cortex Kanwisher N et al. (1997). Journal of Neuroscience 7
FFA vs. PPA PPA FFA Fusiform Face Area Non sensitive to any body part Sensitive to faces > other body parts Kanwisher N et al. (1997). Journal of Neuroscience Fusiform Face Area Not sensitive to attentional allocation per se 1-back memory task applied on control stimuli leads to weaker activity than the same task applied on faces Kanwisher N et al. (1997). Journal of Neuroscience 8
Fusiform Face Area Same or Different? Vuilleumier P et al. (2001). Neuron Fusiform Face Area FFA PPA Vuilleumier P et al. (2001). Neuron Highly reproducible across individuals Kanwisher N et al. (1997). Journal of Neuroscience 9
Core Face Network Not only FFA Kanwisher N & Yovel G (2006). Phil. Trans. Roy. Soc. B. Configural Processing of Faces Same or different? Featural information: information from the constituent parts of the face (i.e., eyes, nose, and mouth) Configural information: spatial relation of between the constituent parts. Configural Processing of Faces Eyes Mouth Ears Same or different? Inversion does not prevent you to recognize that specific image features are body parts. Inversion prevents you to recognize the standard facial configuration 10
Configural Processing in core face network Kanwisher N & Yovel G (2005). Neuron Configural Processing in core face network McCarthy et al. (1999). Cerebral Cortex Configural Processing in core face network Rossion B & Gauthier I (2002). Behav. Cogn. Neurosci. Reviews. Kanwisher et al. (2002). Nat Neurosci 11
Configural Processing in core face network Regions parts of the core face network decrease their activity during experimental conditions in which the standard spatial configuration of the face is broken. This suggest that part of the activity of these regions is related to the integration of the face parts into a whole. It is, however, true that other studies focusing on the core face network (in particular FFA) failed at finding configural effects. Thus leaving open the debate about the role played by these regions FFA & Familiarity Faces are the most familiar stimuli EVER. Is FFA sensitive to stimuli of high familiarity, rather than to faces? Greebles Gauthier I et al. (1999). Nat Neurosci FFA & Familiarity Gauthier I et al. (2000). Nat Neurosci Xu Y (2005). Cereb. Cortex 12
FFA & Familiarity Other studies, using either training paradigms or populations with particular expertise, failed to find a familiarity effect in FFA/M170 Xu Y et al. (2005). Neuropsychologia Summary Visual Processing of Bodies Category-specificity in ventral visual system FFA: localization and functional properties EBA and FBA: localization and functional properties Body Parts organization in the human brain IT cortex of macaque Single cells in macaque s IT are sensitive to body stimuli in a category-selective fashion Peelen M & Downing P (2007). Nature Review Neuroscience 13
IT cortex of macaque Single cells in macaque s IT are sensitive to body stimuli in a category-selective fashion Peelen M & Downing P (2007). Nature Review Neuroscience Intracranial Recordings 190-260ms Bodies Faces Tools Animals Hands Faces Objects Peelen M & Downing P (2007). Nature Review Neuroscience Intracranial Recordings 190-260ms Bodies Faces Tools Animals Hands Faces Objects Peelen M & Downing P (2007). Nature Review Neuroscience 14
N190 Component N170 Faces N190 Bodies Thierry et al. (2006). NeuroImage Peelen M & Downing P (2007). Nature Review Neuroscience Extrastriate Body Area A focal region of the lateral occipitotemporal cortex responds strongly and selectively to static images of human bodies and body parts, but weakly to faces, objects and object parts Peelen M & Downing P (2007). Nature Review Neuroscience Extrastriate Body Area Peelen M & Downing P (2007). Nature Review Neuroscience 15
Extrastriate Body Area Corradi-Dell Acqua C et al. (2015). Consciousness & Cognition Extrastriate Body Area Corradi-Dell Acqua C et al. (2015). Consciousness & Cognition LOC MT EBA Malach et al. (1995). PNAS Downing & Peelen (2011). Cog. Neurosci. 16
fmri spatial resolution fmri maps have weak spatial resolution, small cubes (voxels) of about 2/3 mm per side LOC MT EBA It is possible, that same coordinates, might subtend different neuronal populations, each involved in a particular stimulus Body Motion How can we discover whether Body and Motion stimuli are processed by independent (but intermingled) populations? Multivariate Pattern Analysis The signal in each voxel depends on the amount of firing neurons within each voxel. Body Motion 17
Multivariate Pattern Analysis Coord. 1 Coord. 2 Coord. 3 This should yield to distributed patterns of activity reflecting the inhomogeneous distribution of one neuronal population across neighboring coordinates. Multivariate Pattern Analysis Motion Coord 1 Coord 2 Coord 3 Body Coord 1 Coord 2 Coord 3 Multivariate Pattern Analysis Motion Coord 1 Coord 2 Coord 3 Body Coord 1 Coord 2 Coord 3 18
Multivariate Pattern Analysis Motion Body The higher the correlation, the closer the representation in EBA LOC MT EBA Peelen M & Downing P (2007). Nature Review Neuroscience LOC MT EBA Peelen M & Downing P (2007). Nature Review Neuroscience 19
Point-light Displays Participants are not faced with a full human shape, but with a set of point-lights attached to the main joints of a human body. If the body is still, observers are not usually able to identify it through the point-lights; however, if the body starts moving, observers immediately identify through the displacement of the point-light a human motion. Fusiform Body Area An additional category-sensitive region for bodies in located in the fusiform gyrus, near (but separated from) FFA. Peelen M & Downing P (2007). Nature Review Neuroscience FBA FFA Peelen M & Downing P (2007). Nature Review Neuroscience 20
High resolution fmri Special fmri sequence can lead to better spatial resolutions, (voxels of about ~ 1.5 mm per side) < FBA FFA Schwarzlose R (2005). J Neuroscience TMS over EBA Participants performed a task in which they matched two stimuli to a previously-perceived target. Urgesi C et al (2004). Curr. Biol. Peelen M & Downing P (2007). Nature Review Neuroscience 21
TMS over EBA Interference with EBA slowed matching of body parts (but not face parts, or motocycle parts) Urgesi C et al (2004). Curr. Biol. Peelen M & Downing P (2007). Nature Review Neuroscience Brain damaged patients 14 patients with damage to the left hemisphere anterior part 14 patients with damage to the left hemisphere posterior part Moro V et al (2008). Neuron Brain damaged patients 14 patients with damage to the left hemisphere anterior part 14 patients with damage to the left hemisphere posterior part Moro V et al (2008). Neuron 22
Brain damaged patients 14 patients with damage to the left hemisphere anterior part 14 patients with damage to the left hemisphere posterior part Lesions correlating most with body matching deficit Location of EBA FBA OFA & FFA in previous fmri studies Moro V et al (2008). Neuron Summary Visual Processing of Bodies Category-specificity in ventral visual system FFA: localization and functional properties EBA and FBA: localization and functional properties Body Parts organization in the human brain Body Core Network What does this network tell us about how the brain processes the body? 23
Body Parts segregation in the human brain Bracci S et al (2010). J. Neurophysiology Multivariate Pattern Analysis Hand Foot The higher the correlation, the more similar the representation in EBA Body Parts segregation in the human brain Multivariate Pattern Analysis Op de Beeck H et al (2010). NeuroImage 24
Phase Encoding Orlov T et al (2010). Neuron Phase Encoding Orlov T et al (2010). Neuron Phase Encoding Orlov T et al (2010). Neuron 25
Phase Encoding Orlov T et al (2010). Neuron Body Parts segregation in the human brain Orlov T et al (2010). Neuron Configural Processing of Bodies Same or different? Much easier? As for the case of faces, also bodies are held to processed in configural fashion (as already suggested by the neuropsychological study of patients affected by Autotopagnosia) Reed C et al (2003). Psychological Science 26
Body Structural Description Configural Representation of the Body It has been suggested that Autotopagnosia may reflect a trouble in coding the spatial relations among body parts, leaving a spared ability in identifying them (Sirigu et al., 1991; Buxbaum & Coslett, 2001; Schwoebel & Coslett, 2005). Representation of parts Representation of how the parts relate one another in the full body X Configural Processing of Bodies Urgesi C et al (2009). J. Neurosci. Configural Processing of Bodies Urgesi C et al (2009). J. Neurosci. 27
Configural Processing of Bodies Body Parts Buildin g Parts Spatial Judgm. [SJ] BodySJ BuildingsSJ Identificat. [I] BodyI BuildingsI Corradi-Dell Acqua et al. (2008). Cerebral Cortex Configural Processing of Bodies Corradi-Dell Acqua et al. (2008). Cerebral Cortex Configural Processing of Bodies Identification of body parts per se Corradi-Dell Acqua et al. (2008). Cerebral Cortex 28
Configural Processing of Bodies Coding spatial relations between body parts Corradi-Dell Acqua et al. (2008). Cerebral Cortex Configural Processing of Bodies pips/ SPL Representation of the body in terms of spatial relationship between parts Representation of parts regardless of their spatial configurations Other body parts organizations Can the signal elicited by each kind of body part tell something about how the brain codes their relation? Bracci S et al (2015). J. Neurosci. 29
Body Parts organization in the human brain R EBA L EBA R FBA L FBA Can this dissimilarity matrix tell us something about how the body is organized in the brain? Bracci S et al (2015). J. Neurosci. Body Parts organization in the human brain A priori models Bracci S et al (2015). J. Neurosci. Body Parts organization in the human brain R EBA L EBA R FBA L FBA R Occ L Occ Semantic model is the best! Bracci S et al (2015). J. Neurosci. 30
Body Parts organization in the human brain Also in left pips the semantic model is the best! Bracci S et al (2015). J. Neurosci. Body Parts organization in the human brain EFFECTORS (hands, feet, arms, legs) FACE Bracci S et al (2015). J. Neurosci. Summary Visual Processing of Bodies II EBA and FBA: localization and functional properties Configural processing of bodies Body Parts segregation in the human brain Body Parts organization in the human brain 31