The Time Course of Negative Priming

The Time Course of Negative Priming Hendrik Degering Bernstein Center for Computational Neuroscience Göttingen University of Göttingen, Institute for Nonlinear Dynamics 11.12.2009, Disputation

Aging Effects in Selective Attention Behavioral Experiments Theoretical Psychology ERP Analysis Computational Modeling Advanced Averaging Single Trial ERPs Biology of Aging

Outline 1 Introduction of the Negative Priming (NP) Effect 2 A Simple Computational Model for Negative Priming 3 Assessing the Time Course by Behavioral Experiments 4 A Comprehensive Account: The General Model Behavioral Experiments Theoretical Psychology Computational Modeling

The Negative Priming Paradigm DT TT CO trial n 1 trial n trial n+1 trial n+2 time Assessing Selective Attention presentation of targets (T) and distractors (D) important experimental conditions: DT (distractor target), TT (target target), CO (control) Definition Negative Priming (NP) is a slower response in DT trials as compared to CO τ response (CO) τ response (DT) < 0

The Negative Priming Effect Robustness stimuli: pictures, letters, words, shapes... responses: key pressure, voice recording... responses to identity, location, match... BALL match mismatch Sensitivity of Negative Priming to realized priming conditions response stimulus interval presence of probe distractors stimulus onset asynchrony number of distractors age and sex of subjects distractor saliency experiment instructions

Theoretical Accounts of NP Distractor Inhibition active inhibition of distractors inhibitory rebound Houghton and Tipper (1994) activation 1 0 trial onset reaction time target time distractor Response Retrieval similarity triggers retrieval conflicting information Rothermund et al. (2005) Global Threshold (ISAM) semantic feedback loop adaptive threshold mechanism forced activation decay Kabisch (2003) activation "Bank" retrieval 1 "Ball" "Bank" time

Aims concretize global threshold theory test our implementation in behavioral experiments refine the time course of negative priming construct a framework for the comparability of theories

Dynamics of Representation Activations Network Implementation object compound of 1000 integrate and fire neurons all-to-all coupled input [N(µ, σ)] + µ = σ = θ/50 firing rate by spike summation relative average firing rate 1 0.8 0.6 0.4 0.2 0 0 20 40 60 time steps 20 10 80 100 0 change/time step [%] Firing Rate Dynamics 1 dx α dt = I x, I {0, 1} maximum firing rate I = input strength time constant fluctuations caused by relative reset

Representation Dynamics in the ISAM DT TT CO 1 activation 0.5 0 0 1 2 3 4 5 6 7 red ball green ball threshold time [s]

Implementation of the ISAM I T 1 x1 T I2 D x D 2 Setup two variables per object i (e.g. ball): xi T target xi D distractor different time constants: α rise β decay x T 3 x D 1 Object Representation Dynamics adaptation to external input I interference between distractor and target 1 α dx T i dt = I T i x T i ζxi D xi T

The Threshold Mechanism of the ISAM x θ Adaptation of the threshold 1 dθ γ dt = x(t t) θ based on the global activity level ( ) x = 1 n r T + r D + (xi T + xi D ) 2 Decision Making i=1 A response is executed if #(xi T, xi D > θ) = 1.

Performance of the ISAM Simulations of the ISAM provide simple activation dynamics derived from spiking networks reproduction of experimental data sufficiency of a threshold mechanism reaction time predictions for various settings reaction time [ms] 700 600 DT CO TT 500 1 1.05 1.1 1.15 1.2 1.25 relative distractor saliency Reactive Inhibition higher distractor saliency increased reaction times stronger negative priming DT: distractor target TT: target target CO: control

EEG Correlates of NP DT TT CO ERP Experiment simple voicekey paradigm grand average ERP, N=16 10µV - 1s + Fp1 Fpz Fp2 AF7 AF8 AF3 AFz AF4 F7 F8 F5 F3 F1 Fz F2 F4 F6 FT7 FC5 FC1 FCz FC2 FC6 FT8 FC3 FC4 T7 C5 C3 C1 Cz C2 C4 C6 T8 CP5 CP3 CP1 CPz CP2 CP4 TP7 CP6 TP8 P7 P5 P3 P1 Pz P2 P4 P6 PO3 POz PO4 PO7 O1 Oz O2 PO8 P8 Results facilitation during perceptual processing of DT and TT trials higher cognitive control during later stages for DT compared to CO

EEG Correlates of NP µv 2 1 P5 0.2 0.4 0.6 0.8 1 1.2 1.4 s µv 2 1 1 P6 DT TT CO 0.2 0.4 0.6 0.8 1 1.2 1.4 s ERP Experiment simple voicekey paradigm grand average ERP, N=16 1 2 3 4 10µV P300-1s + 2 3 4 T7 P300 FT7 TP7 F7 P7 FC5 C5 CP5 AF7 F5 P5 C3 PO7 F3 FC3 CP3 P3 Fp1 AF3 FC1 CP1 O1 F1 C1 P1 PO3 Fpz AFz Fz FCz Cz CPz Pz POz Oz F2 Fp2 FC2 C2 CP2 P2 AF4 O2 F4 P4 PO4 FC4 C4 F6 CP4 AF8 P6 PO8 F8 FC6 C6 P8 FT8 T8 CP6 TP8 Results facilitation during perceptual processing of DT and TT trials higher cognitive control during later stages for DT compared to CO

EEG Correlates of NP µv 2 1 FPZ DT TT CO ERP Experiment simple voicekey paradigm grand average ERP, N=16 10µV 1 2 3 4-1s + 0.2 0.4 0.6 0.8 1 1.2 1.4 s positive slow wave Fp1 Fpz Fp2 AF7 AF8 AF3 AFz AF4 F7 F8 F5 F3 F1 Fz F2 F4 F6 FT7 FC5 FC1 FCz FC2 FC6 FT8 FC3 FC4 T7 C5 C3 C1 Cz C2 C4 C6 T8 CP5 CP3 CP1 CPz CP2 CP4 TP7 CP6 TP8 P7 P5 P3 P1 Pz P2 P4 P6 PO3 POz PO4 PO7 O1 Oz O2 PO8 P8 Results facilitation during perceptual processing of DT and TT trials higher cognitive control during later stages for DT compared to CO

Perception or Selection Predictions by the ISAM Post-Cue Paradigm perception vs. target selection target color cue after stimuli task activation task 1.2 1 0.8 0.6 0.4 0.2 0 1 0.8 0.6 0.4 0.2 0 10 12 14 16 time reaction Reaction Time time [ms] [ms] CO 0 200 400 600 800 1000 1200 1400 CO DT TT 0 200 400 600 800 1000 CO DT TT reaction Reaction time Time [ms] [ms] DT TT CO 0 500 1000 1500 2000 2500 DT TT 0 500 1000 1500 Results DT and TT are perceived faster NP occurs late ISAM not applicable

Selection or Response Gaze Shift Paradigm target selection vs. response generation word-picture comparison distant stimuli time markers from eye movements cannot be modeled by the ISAM gaze shift Bus retinal focus reaction time [ms] 0 200 400 600 800 1000 CO DT TT CO DT 300 350 400 450 500 550 TT Results response benefits for TT negative priming during selection no explanation by response retrieval

The Time Course of Negative Priming Our Experiments Showed perceptual benefit of DT and TT higher cognitive control for DT negative priming during target selection Consequences neither a perceptual nor a response effect episodic retrieval theory explains NP best Problems of Current Theories not precise enough to reliably generate hypotheses not directly comparable no differentiation of trial processing stages

The General Model for NP color shape word Bus Perceptual Input target: green task: compare CentralExecutive Binding Layer Semantic Layer Episodic Memory Action Layer NO Response

Characteristics of the General Model Advantages flexible model for action selection based on perception biologically realistic structure modeling different stages of trial processing Comparability of Theories the General Model quantifies the incorporated theories definition of theory specific setscrews implementation of single theories as special cases continuous transition from one explanation to another Open Issues parameter reduction dialogue necessary

Summary Quantification of Global Threshold Theory We derived a simple object representation dynamics from firing rate considerations of a spiking neural network. We showed that a threshold mechanism can explain both negative and positive priming. The ISAM provides a generalizable setup for a decision mechanism based on perceptual input and top-down control. Experimental Testing of the Implementation The ISAM generated hypotheses for the post-cue paradigm. We showed the limitations of theories localizing NP in perception or response generation. A comprehensive theory of negative NP should describe the different stages of processing individually.

Summary ctd. The Time Course of Negative Priming EEG correlates show an early similarity of DT and TT trials and a later cognitive intervention in DT trials. The post-cue paradigm and the gaze-shift paradigm show negative priming to be a target selection effect. Computational Model for the Comparability of Theories The simple dynamics of the ISAM is used to construct several specialized and physiologically justified layers. We obtain a flexible neurocomputational model for action selection based on perceptual input. The General Model quantifies different theories of NP. The uniform description of the different theoretical accounts allows for a direct quantitative comparison.

Thanks... C4-Project Theo Geisel Marcus Hasselhorn J. Michael Herrmann Mattias Ihrke Jörg Behrendt NP-Theorists Björn Kabisch Christian Frings Steven Tipper George Houghton Florian Waszak Aging Shu-Chen Li Timo von Oertzen EEG Analysis Torsten Wüstenberg Henning Gibbons Ralph Meier Miguel Valencia Ustárroz Norbert Marwan

... Additional Material

Explaining the Variable Time Constant Inner Structure gap at low potentials due to relative reset gap travels upwards appears as bands Distribution of Potentials average over 10.000 trials values [0.0098, 0.0115]

Dynamics of the ISAM 1 target input activation average target activation threshold sensitivity activation distractor activation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Dynamics of the ISAM 1 target input activation average target activation threshold sensitivity activation distractor activation threshold adaptation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Dynamics of the ISAM activation by input 1 target input activation average target activation threshold sensitivity activation distractor activation threshold adaptation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Dynamics of the ISAM activation by input 1 target input activation average target activation threshold decision making sensitivity activation distractor activation threshold adaptation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Dynamics of the ISAM activation by input 1 target input activation average target activation threshold decision making sensitivity activation distractor activation sensitivity cutoff threshold adaptation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Dynamics of the ISAM negative priming condition positive priming condition 1 activation 0 RT RSI RT 0 500 1000 2000 3000 time [ms]

Advanced Averaging ms 500 400 300 200 100 0 0 200 400 600 800 30 20 10 µv 0 µv 0 10 20 0 200 400 600 800 ms 30 20 10 10 20 0 200 400 600 800 ms ms Accounting for Variability of Processing During Averaging match trial onset and reaction times by time warping optimal time warping function by means of recurrence plots length of the time warping curve defines a metric of trials iterative averaging of pairs of most similar trials obtain an average with more pronounced peaks

Recurrence Plots determining the warping path averaging two artificial trials time (ms) noise strength

Performance of Time-Warped Averaging a) artificial data b) real EEG data

Feature Layers BALL Realization Ball red green grey Background feature decomposition in the visual pathway input only to perceived features number and topology of feature layers paradigm specific distinct feature layers, here Color, Shape and Word every layer holds one variable for each feature instance feature present input = 1, otherwise input = 0.

1.4 1.2 1 0.8 0.6 0.4 0.2 0 1.5 2 2.5 3 x 10 4 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 1 x 10 3 0 1.5 2 2.5 3 x 10 4 1.4 1.2 1 0.8 0.6 0.4 0.2 0 1.5 2 2.5 3 x 10 4 Feature Binding color Binding Layer shape Background the brain has to track object entities features have to be bound together in a flexible way objects are represented by bindings without perception, bindings decay Realization a vector holds indices of feature instance variables the binding has a certain maximum synaptic strength feature instances balance their activation via bindings slow decay of synaptic strength in absence of input

Semantic Layer Realization Background paradigms like object naming and comparison rely on semantic classification language evokes semantic representation action initiation by comparing semantic activations to a threshold feature layers with semantic matter project to the semantic layer words and shapes with the same semantic meaning converge a threshold adapting to a global mean divides the representations into sub- and superthreshold for action decision

Action Layer Background only one action at a time possibility to hold action initiation concurrence of different actions Realization input by superthreshold semantic representations additionally activation for no response contributing to threshold action initiation by crossing an adaptive threshold

Episodic Memory Realization Background every finished episode is memorized the memory decays with time similarities between percept and memory trigger a retrieval the more similar the percepts, the stronger the retrieval After a reaction, the entire actual state of the model is stored. A scalar product of percept and memory determines retrieval. Every variable receives the memorized value weighted with the retrieval strength as additional input.

Computational Modeling in Psychology Benefits links to physiology specification of theories comparability of different accounts proof of plausibility falsifyable straightforward hypothesis generation Difficulties hard to communicate arbitrariness of implementation optimal level of complexity complex data patterns require complex models