THE PHYSIOLOGICAL UNDERPINNINGS OF AFFECTIVE REACTIONS TO PICTURES AND MUSIC Matthew Schafer The College of William and Mary SREBCS, USC
Outline Intro to Core Affect Theory Neuroimaging Evidence Sensory modality processing Current Study Physiological Measures Hypotheses Preliminary Results Discussion Future Directions Acknowledgements
Core Affect Raw feelings in emotions Two dimensions: valence & arousal Different neural circuitry Lead to perception of unified affective state Additional cognitive appraisal leads to state differentiation into emotions For ex: fear vs anger HN, but different cognitive aspect
Affective Circumplex Based off similarity judgments Closeness in space=similarity of state
Neuroimaging Evidence Baucom et al. 2012: Trained classifiers for valence, arousal & unified affective states Results: Predictive power better than chance for both within and between subjects Valence and arousal separable in brain Other studies have shown dissociation for valence and arousal within different modalities Picture, word, face etc.
Sensory Modality Two functional systems process affective stimuli Sensory integration network: modality specific Visceromotor network: modality general Shinkareva et al. (under review): evidence for modality specific processing of valence in pictures vs. sound stimuli No evidence for modality general processing Arousal has yet to be investigated
Current Study Physiological basis of affect Modalities Pictures Music Measures Facial EMG GSR ECG
Physiological Measure: EMG Electromyography (EMG): measures electrical potential of muscles Zygomaticus: smiling muscle positive valence Corrugator: frowning muscle negative valence
EMG studies Valence CLEAR RELATIONSHIP Thoughts (Schwartz, Ahern & Brown, 1979, 1980) Pictures (Lang et al. 1993) Pictures, sounds & words (Larsen, Norris, & Cacioppo, 2003) COMPLICATED RELATIONSHIP NO RELATIONSHIP Arousal
Physiological Measure: GSR Galvanic Skin Response (GSR): Changes in SNS as measured by electrical conductance of skin Increases with arousal
GSR studies Valence CLEAR RELATIONSHIP COMPLICATED RELATIONSHIP NO RELATIONSHIP Arousal Pictures (Bradley et al., 2001; Lang et al., 1993; Greenwald et al., 1989; Cook et al., 1991) Review: Mauss & Robinson, 2009 Correlated in music but not environmental sounds (Gomez & Danuser, 2004)
Physiological Measure: ECG Electrocardiography (ECG): electrical activity of heart; HR controlled by both PNS & SNS Complex relationship to affect
ECG studies CLEAR RELATIONSHIP COMPLICATED RELATIONSHIP NO RELATIONSHIP Valence Deceleration to pleasant & unpleasant film clips, compared to neutral (Codispoti et al., 2008) Arousal Deceleration to arousing pictures (Bradlley et al, 2001) Deceleration to arousing film clips (Carvalho et al., 2012) Accl. to noises, not music (Gomez & Danuser 2004) Music (sad, fear, happy): slowed HR; sad slowed most (Krumhansl, 1997) Music (Zimny & Weidenfeller, 1963)
Method 8 conditions x 2 modalities x 6 trials= 96 total trials HN H HP 20 pictures for each condition N P 2 subsets of 10, presented 3x each LN LP 6 musical clips for each condition L
Method 5s BL 10s Stim Rating Ratings: 1 of 6 Indicate some feeling state: positive, negative, excited, calm, anxious, sad; How positive do you feel? Scale of 1 to 9 1 trial 1 trial baseline Fade in 500ms Fade out 500ms baseline 10 pictures 1s for each + How POSITIVE do you feel? + How POSITIVE do you feel? 5s 10s approx 2s 5s 10s approx 2s 96 trials (8 conditions * 6 ratings * 2 modalities)
Hypotheses EMG: correlation with valence Zygomaticus: increases w positive valence Corrugator: increases w negative valence GSR: increase with arousal HR: exploratory Interactions with Modality: exploratory
Dimension 2 Dimension 2 Preliminary Results: Behavioral Used MDS, 2D solution Valence strong Arousal weaker LN music =HN music 1.5 1.0 0.5 MDS: Picture 1.5 1.0 0.5 MDS: Music Red: positive Blue: negative Black: neutral : high arousal : middle arousal Upside down : low arousal 0.0 0.0-0.5-0.5-1.0-1.0-1.5-1.5-1.0-0.5 0.0 0.5 1.0 1.5 Dimension 1-1.5-1.5-1.0-0.5 0.0 0.5 1.0 1.5 Dimension 1
Preliminary Results: EMG Main effect of valence is significant for both channels Corrugator: F(1,124)=13.683, p<.05 Zygomaticus: F(1,124)=5.948,p<.05 As expected Interaction effect between valence and modality for corrugator F(1,124)=6.428, p<.05 Valence effect occurs for pictures, not music
Change in Corrugator (microvolts) Change in Corrugator (microvolts) Change in Corrugator (microvolts) Change in Corrugator (microvolts) Preliminary Results: Corrugator Pictures Music 0.0000400 0.0000325 HN (Blue), HP (Red) Pictures N (Blue), P (Red) Pictures 0.0000400 0.0000325 HN (Blue), HP (Red) Music N (Blue), P (Red) Music 0.0000250 0.0000250 0.0000175 0.0000175 0.0000100 0.0000100 0.0000025 0.0000025-0.0000050-0.0000050-0.0000125-0.0000125-0.0000200-0.0000200 0.0000400 0.0000325 LN (Blue), LP (Red) Pictures L (Purple), H (Orange) Pictures 0.0000400 0.0000325 LN (Blue), LP (Red) Music L (Purple), H (Orange) Music 0.0000250 0.0000250 0.0000175 0.0000175 0.0000100 0.0000100 0.0000025 0.0000025-0.0000050-0.0000050-0.0000125-0.0000125-0.0000200 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000-0.0000200 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000
Change in Zygomaticus (microvolts) Change in Zygomaticus (microvolts) Change in Zygomaticus (microvolts) Change in Zygomaticus (microvolts) Preliminary Results: Zygomaticus Pictures Music 0.0000400 0.0000325 HN (Blue), HP (Red) Pictures N (Blue), P (Red) Pictures 0.0000400 0.0000325 HN (Blue), HP (Red) Music N (Blue), P (Red) Music 0.0000250 0.0000250 0.0000175 0.0000175 0.0000100 0.0000100 0.0000025 0.0000025-0.0000050-0.0000050-0.0000125-0.0000125-0.0000200-0.0000200 0.0000400 0.0000325 LN (Blue), LP (Red) Pictures L (Purple), H (Orange) Pictures 0.0000400 0.0000325 LN (Blue), LP (Red) Music L (Purple), H (Orange) Music 0.0000250 0.0000250 0.0000175 0.0000175 0.0000100 0.0000100 0.0000025 0.0000025-0.0000050-0.0000050-0.0000125-0.0000125-0.0000200 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000-0.0000200 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Time Interval (ms) 9,000 10,000
Preliminary Results: GSR No significant effects for valence or arousal (ps>.05) Surprising: in literature strong relationship to arousal
Preliminary Results: ECG No significant effects for valence or arousal (ps>.05) Not very surprising: literature all over the place
Discussion EMG: valence relationship stronger for pictures Behav MDS suggests the stimuli were valid something with modality Blood & Zatorre 2001: Ps report lack of Imagery with musical stimuli Imagery may relate to affective processes Pics= episodic &rich in information; music=abstract Gomez & Danuser 2004: Ps try to identify name of musical clips Clips from movies Stimulus presentation? Modality specific processing? Magnitude? HR Literature is messy Attentional processes, stimulus modality, duration of task, intensity of stimuli, task instructions etc. GSR Surprising Methodology changes Sensory modality processing Still not clear
Future Directions Individual difference measures Better stimulus selection LN music: really HN Longer baselines GSR: systematic variation in BL across conditions Stimulus presentation Pictures 10 at a time in rapid succession, one musical clip Episodic v. abstract More participants
Acknowledgements Professor Wedell Professor Shinkareva Professor Vendemia Jongwan Kim Matthew Facciani Laura Baucom SREBCS