Serotonin and Dopamine Interactions. Atanas Stankov Computational Psychiatry Seminar Translational Neuromodeling Unit

Transcription

1 Serotonin and Dopamine Interactions Atanas Stankov Computational Psychiatry Seminar Translational Neuromodeling Unit

2 Outline NT Review Opponent/complement Interactions DA-5HT opponency L-Dopa and Citalopram Polymorphism effects on Reversal Learning Impulsivity Measuring impulsivity Endophenotypes Computational model

3

4 Optimal control problems computationally difficult scarcity of rewards and many dangers constraints from neural processing Methods: model-based (goals) model-free (habits) Pavlovian (active approach and engagement for chance of reward; inhibition and withdrawal for chance of punishment) What influences do DA and 5HT have on these methods? Y-L Boureau and P Dayan 2010

5 Valance vs. Action Control based on: reward vs action invigoration vs punishment Y-L Boureau and P Dayan 2010

6 Other factors Important role of DA in modelfree values: temporal difference (TD) prediction error. Inconsistency in reward anticipation. Shown in monkeys (Schultz et al, 1997) Opportunity cost of time: invested time in reward seeking. Is it better to be a sloth or to be vigorous? Y-L Boureau and P Dayan, 2010

7 Appetitive responses vs Aversive responses GO to avoid punishment GO to get reward NO-GO to avoid punishment NO-GO to get reward Y-L Boureau and P Dayan, 2010

8 Pavlovian vs Instrumental responses Y-L Boureau and P Dayan 2010

9 Y-L Boureau and P Dayan, 2010

10 Effects of L-DOPA and citalopram on action learning with reward and punishment Coupling of valance and action axes Impaired learning during GO to avoid punishment and NO- GO to get reward 90 healthy subjects Placebo L-DOPA Citalopram (SSRI) Raise DA and 5HT Guitart-Masip M et al 2013

11 240 trials (60 trials per condition) Guitart-Masip M et al 2013

12 Experiment performance: Guitart-Masip M et al 2013

13 Experiment performance: Pavlovian responses Guitart-Masip M et al 2013

14 Effects of levodopa and citalopram on choice performance Guitart-Masip M et al 2013

15 Reinforcement learning models Six models (Rescorla-Wagner) with instrumental and Pavlovian reinforcement learning hypotheses Different reward and punishment parameters means different aversive values Stick to account for subject tendency to avoid/chose the same action twice Guitart-Masip M et al 2013

16 Effects of levodopa and citalopram on model parameters Guitart-Masip M et al 2013

17 Results Differential, and not opponent, effects of L-DOPA and citalopram L-DOPA decreased influence of Pavlovian control Citalopram effect independent of valence, higher value for action bias No significance to stickiness from the effects of citalopram (contrary results to Seymour et al in a dietary tryptophan depletion)

18 Dissociable effects of DA and 5HT on reversal learning Polymorphisms of SERT and DAT1 on the effects of dissociable and opponent actions on decision making S (short) and L (long) polymorphisms of SERT promoter region (5HTTLPR) 9R (common) and 10R (rare) alleles of DAT 685 subjects den Ouden et al, 2013

19 Dissociable effects of DA and 5HT on reversal learning Hypothesis: SERT polymorphism will affect lose-shifting DAT1 polymorphism will affect win-staying Compare SERT and DAT1 polymorphism on preservation den Ouden et al, 2013

20 Probabilistic Reversal Learning Paradigm 40 trials for acquisition 40 trials for reversal den Ouden et al, 2013

21 win-stay unaffected by polymorphisms lose-shift affected only by SERT L homozygote preservation error affected by DAT1 polymorphisms (9R homozygote has positive growth with acquisition, 9R/10R no change, 10R homozygote has negative change with acquisition) preservation error vs acquisition history also affected by DAT1 polymorphisms chance decreased with acquisition but was unaffected by DAT1 den Ouden et al, 2013

22 Computational models for preservation experience-weighted attraction (EWA) model n_s,t experience weight, ρ experience decay factor φ decay rate of previous layoffs Rescorla-Wagner model different rates for reward and punishment unchosen stimulus is not updated den Ouden et al, 2013

23 EWA model captures difference in preservation during reversal phase (higher 9R alleles means more preservation) ρ experience decay factor increased with higher 9R alleles Significant preservation error and preservation error vs acquisition history dependence on DAT1 polymorphisms SERT polymorphisms had no significant effect on EWA parameters DAT1 polymorphisms had no significant effect on the Rescorla- Wagner den Ouden et al, 2013

24 Discussion SERT and DAT1 polymorphism consequences are not exactly clear yet L have been shown to have higher 5HT binding (less extrasynaptic) which might explain the higher lose-shift (also confirmed by previous studies with experimental manipulation and tryptophan depletion) No effect on the reversal factor by SERT, contrary to previous studies DAT1 9R allele count has larger past experience weight, and hence more resistance by subjects to update the belief. Therefore resulting in a stronger preservation. Authors conclude to functional dissociation but found no direct oppency den Ouden et al, 2013

25 Measurement of Impulsivity Definition of impulsivity varies across studies Animal and human methods of measurement Dalley and Roiser, 2012

26 A. 5 Choice serial reaction time task B. Delay discounting task C. Stop signal time reaction task Dalley and Roiser, 2012

27 Humans measures Self reporting questionnaires Temporal (Delay) discounting Stop-signal reaction time Go/No-go or Continuous performance tests - premature reaction tendencies Gambling and Risk taking tests Dalley and Roiser, 2012

28 Neurocognitive endophenotypes of impulsivity and compulsivity Quantitative measures in cognitive and neural processes Possibly present in 1st degree relatives of patients with psychiatric disorders May provide early detection of flow blown disorders Robbins et al, 2012

29 Robbins et al, 2012

30 Both impulsivity and compulsivity are failures of response inhibition (top-down cognitive control) Compulsivity - dysfunction of stimulus response habit learning Psychiatric disorders Obsessive-compulsive disorder - treated with SSRI s Motor tics (Tourette s Syndrome) - treated with neuroleptic drugs (antipsychotics) Tests in response inhibition and cognitive rigidity may provide possible endophenotypes Robbins et al, 2012

31 Drug addiction Impulsivity and compulsivity present in substance abuse Rats performing impulsively in the 5CSRTT have higher tendencies for self administration of cocaine higher toleration of foot-shock in drug seeking higher relapse rates Human show analogous tendencies with impulsivity in cocaine users Possibility of accelerated habit formation (dysfunction of habit learning) may transition impulsive individuals to compulsive smoking Impulsivity and compulsivity as an endophenotypes of substance dependence Robbins et al, 2012

32 Meunier et al, 2012

33 Co-morbidity: OCD coupled with drug abuse, or binge eating, or schizophrenia, and or bulimia nervosa Decreased DA D2 receptors in the striatum Future direction: Understanding differences of impaired goal directed behavior over habits and impulsive behaviors for theorybased treatment strategies. Robbins et al, 2012

34 Computational Model of Craving and Obsession Decision making model: 3 parts 1) State recognition Two systems for search: 2) Flexible (similar to model-based) 3) Inflexible (similar to model-free) Redish and Johnson, 2007

35 situation-outcome (S-O) model-based computationally expensive flexible route response VS situation-action (S-A) model-free computationally cheap inflexible cognitive map place Redish and Johnson, 2007

36 Redish and Johnson, 2007

37 Cravings: desired for something, hence the agent is aware of a possible action to the outcome. Associated with planning system (S-O) Obsession: authors argue craving may lead to repeated search of the same (S-O) course Redish and Johnson, 2007

38 Questions? Comments?

39 Appendix A1: Reinforcement learning models Action a_t and stimulus s_t with trial t (go or no-go) with probability: With constant bias parameter: with irreducible noise ξ, and value w. For RW and RW + noise, they use the classic RW update rule: With Pavlovian term: where π is a free parameter. where ε is is the learning rate with r_t {-1,0,1} and ρ is a free parameter for the size of the reinforcer. Guitart-Masip M et al 2013

40 Appendix A2: Reinforcement learning models With stickiness factor: Guitart-Masip M et al 2013