Why we cooperate? The evolution of collective action" " Autonomous Agents and MultiAgent Systems, 2014/15!

Transcription

1 Why we cooperate? The evolution of collective action" " Autonomous Agents and MultiAgent Systems, 2014/15! Francisco C. Santos Instituto Superior Técnico, Universidade de Lisboa & GAIPS / INESC-ID, Portugal The ultimate goal of modelling complex systems Microscopic interactions Emergent collective phenomena / properties local global 1

2 Last class: Rational? Last class: Rational? 2

3 Last class: Rational? synopsis o The costs and benefits of cooperation o Game theory, evolution & social learning o Evolutionary routes to the emergence of cooperation o Kin selection, Multi-level selection, direct & indirect reciprocity, social norms & signals o Networks & peer-influence o The game we cannot afford to lose: An evolutionary game dynamics approach to climate governance 3

4 what is cooperation? Donor Pays a cost c Why should we cooperate? Receiver Receives a benefit b b>c If natural selection is based in competition, how can it lead to cooperation? what is cooperation? Donor Pays a cost c Receiver Receives a benefit b b>c Social insects, etc. Workers do not reproduce and reduce their own profits to help others (the Queen) 4

5 cooperation among humans - collective action to protect, hunt, nourish, etc. - water sharing - tax paying and social welfare - open source projects Cooperation and self-organization in engineering sciences - decentralized control and population-based AI ex: self-organized task allocation, online adaptive systems, swarm intelligence, collective robotics, etc. Rely on - Understanding and control of the complex nature of self-organization of cooperative behavior. 5

6 Interdisplinar?! YES!! Understanding the evolution of cooperation remains a fundamental challenge, for scientists from fields like economics, evolutionary biology, political science, anthropology, mathematics, computer science, etc. Many fields same tools!! Mathematical Framework : (Evolutionary) Game Theory Metaphors:: Prisoner s dilemma, Ultimatum game, etc. Simulation tools : Multi-agent systems & Artificial Intelligence Prisoner s dilemma or the cost-benefit dilemma Donor Pays a cost c If both play as a donor and as a receiver... Receiver Receives a benefit b your opponent RATIONAL GOAL : maximize your own payoff! C D if your opponent plays C : you better play D. if your opponent plays D : you better play D. BUT: CC is better than DD. you C D b - c -c b 0 Dilemma : despite mutual cooperation (CC) being better than mutual defection (DD), individual rational choice leads to DD 6

7 Prisoner s dilemma or the cost-benefit dilemma Donor Pays a cost c Receiver Receives a benefit b If both play as a donor and as a receiver... RATIONAL GOAL : maximize your own payoff! if your opponent plays C : you better play D. if your opponent plays D : you better play D. BUT: CC is better than DD. Rational?! you C D your opponent C D b - c -c b 0 Dilemma : despite mutual cooperation (CC) being better than mutual defection (DD), individual rational choice leads to DD Game Theories Classic Game Theory Few players Evolutionary Game Theory Large populations Rational decisions Natural selection Strategies that do well reproduce faster Static concepts Dynamical concepts 7

8 evolutionary game theory payoff fitness social success genetic evolution Individuals with higher fitness will reproduce more social/cultural evolution or their behavior will be imitated more often (social learning) evolutionary game theory cost-benefit dilemma (prisoner s dilemma) Natural selection & self-organization leads to the tragedy of the commons 8

9 Replicator equation (2 strategies) Assuming$infinite$popula=ons$+$determinis=c$update Gradient$of$selec=on!x = x(1 x) " # f C (x) f D (x) $ % = g(x) For$2%person$games$in"well&mixed"popula/ons,$the$fitness$values$of$Cs$and$Ds$are$ given$by$the$average$over$all$possible$pairwise$interac=on$($x$=$frac=on$of$cs$)$ $ $ $ $ $ $ $ f f C ( x! C (x) = ) = x(b x C R c)+ + x(1 D S x)( c) f f D ( x! D (x) = ) = xb x+ C T (1 + xx).0 D P C D C D " b c c % $ ' # b 0 & The calculus of selfishness g>0 g<0 + % Increase in the number of cooperators Decrease in the number of cooperators g=0 Gradient$of$selec=on x = x(1 x) [ f C (x) f D (x)] = g(x) Replicator equation 9

10 The calculus of 2-player games x (t) g(x) x c Defection dominance (ex: Prisoner s dilemma) C D C D C b - c -c C R S D b 0 D T P and Global Dynamics orks break the symmetry between and global dynamics often found in well mixed populations fector dominance The calculus of 2-player games co-existence x (t) g(x) x c 0.0 coordination 1.0 Defection dominance (ex: Prisoner s dilemma) x c nce nce t t x (t) g(x) x (t) g(x) x (t) x (t) x c x c 1.0 co-existence coordination 0.0 x c 1.0 co-existence defector dominance coordination co-existence defector dominance co-existence 0.0 x x c c x x c x 1.0 c 0.0 c x c 1.0 x c x c 1.0 bistability mixed equilibrium coordination Co-existence (ex: snowdrift, chicken games, x 0.0 c etc.) 1.0 x 0.0 c 1.0 mixed equilibrium From local to global dilemmas in social networks Pinheiro et al, PLoS ONE, 7 (2), e32114 (2012) coordination From local to global dilemmas in social networks Pinheiro et al, PLoS ONE, 7 (2), e32114 (2012) flavio.lpp@gmail.com Bi-stability (ex: coordination games, stag-hunt game, etc.) 10

11 13/04/15 The calculus of 3-strategies 2-player games Scissor Example: Paper Rock General stability concepts C D Game Theory Nash equilibrium If a strategy is a Nash equilibrium, and if both players play that strategy, then neither person can deviate from that strategy and increase her payoff defector dominance C R S D T P Evolutionary Game Theory Evolutionarily Stable Strategy 0 xc 1 If a strategy is an ESS, then an infinitesimally small amount of players of the other strategy will never be able to invade (spread over the entire population) 11

12 Short remark/critique: Infinite populations?! Evolution in finite populations Evolutionary game theory after 2004 Evolution in finite populations in a nutshell Imagine the simplest form of social learning: Imitate a random individual with a probability that increases with the fitness difference. p = " 1+ e β ( f B f A # ) $ % 1 12

13 Natural selection leads to the tragedy of the commons but cooperation surrounds us!! What are we missing here?! The beauty of modelling core principles 13

14 Ways out of the dilemma all in the family... WD"Hamilton,"JTB"(1964)" G."Price,"J"Maynard&Smith,"etc." the more individuals are related, the more cooperation is feasible b 1 > c r genetic relatedness" Ways out of the dilemma pick the right team Wynne&Edwards"(1962)," "DS"Wilson"&"EO"Wilson,"Am"Sci"(2008)" Traulsen"&"Nowak,"PNAS"(2006),"etc." Cooperative groups tend to be more successful than groups of defectors 14

15 Ways out of the dilemma I scratch your back & you scratch mine... R."Trivers,"Q."Rev."Biol."46,"35"(1971)" repeated interactions, memory, complex strategies Ways out of the dilemma I scratch your back & someone else scratches mine... RD"Alexander,"The"Biology"of"Moral"Systems,"1987" Nowak"&"Sigmund,"Nature"1998" Pacheco,"Santos,"Chalub,"PLoS"Comput"Bio"(2006)" reputation based strategies, social control, gossip, social norms and moral systems 15

16 Indirect reciprocity I scratch your back & someone else scratches mine... I help you and somebody else will help me indirect reciprocity reputation building moral judgements... building a reputation either A helps B observers rest of population or A does NOT help B Gossip reputation of A is updated By helping (or not) another individual, a given player may modify its reputation, which may change the predisposition of others to help in the future -> trust, gossip, assessment of actions (social norms) The strength of third parties & social control image time (weeks) eye weeks flower weeks paid per litre of milk consumed [Bateson et al., Biology Letters 2, ] 16

17 Indirect reciprocity and the evolution of moral systems I scratch your back & someone else scratches mine... After each interaction a new reputation is assigned A does NOT help B should the reputation of A decrease? How to define a social norm? A helps B should the reputation of A increase? Famous social norms rep donor action rep receptor SCORING STANDING S-STAND JUDGING SHUNNING GOOD COOPERATE GOOD GOOD GOOD GOOD GOOD GOOD GOOD DEFECT GOOD BAD BAD BAD BAD BAD GOOD COOPERATE BAD GOOD GOOD GOOD BAD BAD GOOD DEFECT BAD BAD GOOD GOOD GOOD BAD BAD COOPERATE GOOD GOOD GOOD GOOD GOOD GOOD BAD DEFECT GOOD BAD BAD BAD BAD BAD BAD COOPERATE BAD GOOD GOOD BAD BAD BAD BAD DEFECT BAD BAD BAD BAD BAD BAD 17

18 Evolution of social norms Indirect reciprocity allows the characterization of the evolution of complex cultural concepts, such as:" " - - Punishment" - - Forgiveness" - - Evolution of laws and moral systems" - - Appealing definitions of GOOD and EVIL." " " Other ways out of the dilemma Tag-based cooperation Signaling for cooperation Commitments Co-evolution of Cognitive/Social Skills & Cooperation B"Skyrms,Signals,"Cambridge"UP"(2010)" F."C."Santos,"JM"Pacheco,"B."Skyrms,"J"Theo"Bio"(2011)$ $ 18

19 Evolution of social-cognition & cooperation Co-evolution of different features i) Social cognition ii) Group size iii) Cooperation and social cohesion humans [ R. Dunbar, Grooming, giossip & the evolution of human language, HUP 02 ] A new evolutionary paradigm Evolution in networked populations Amostra do facebook One of us 19

20 A new evolutionary paradigm Evolution in networked populations Sample from facebook The new network science... as many other networks it shows a very peculiar topology, with several important implications P(k) ~ k γ WWW actores citações sex celular telefónico linguístico γ = 2.1 γ = 2.3 γ = 3 γ = 3.5 γ = 2.1 γ = 2.1 γ = 2.8 Universal features? 20

21 The role of social networks ex: Obesity me d = 1 40% d = 2 20% d = 3 10% d >3 0% Framingham Heart Study, Christakis)&)Fowler,)NEJM,)2007,)2008)! The role of social networks ex: Obesity & others smoking me d = 2 d = 3 d = 1 3 Alcohol Consumption Christakis)&)Fowler,)NEJM,)2007,)2008)! 21

22 The role of peer influence ex: hapiness is contagious On average, the probability of feeling happy increases 15% if I have a happy friend (distance = 1)" Increases 10% if I have a friend of a friend who is happy (distance = 2)" Increases 5% if I have a friend of a friend of a friend who s happy (distance = 3)" Each unhappy friend reduces this probability by 7%. " Fowler"&"Christakis,"BMJ"2009$ What about cooperation? ex: cooperation/sharity 3 me d = 2 d = 3 d = 1 22

23 The power of large-scale simulations of multi-agent systems If we compute the correlations emerging from dynamical processes on complex networks such as Evolutionary Dynamics of cooperation Opinion dynamics (Voter Model) Disease spreading (SIR) We get critical correlations of 2-3, as in empirical results FL)Pinheiro,)MD)Santos,)FC)Santos,)JM)Pacheco,)PRL)2014.! Are we just an outcome of our social network? Is this a route to a discharge of social responsibility? NO!! The impact of our decisions goes well beyond what we had thought!! How to understand the complex dynamics of individuals choices and behaviours in networked populations? 23

24 Agent-based simulations setup: Community = static graph 50% of Ds and 50% of Cs. evolve for 10 6 generations run many simulations for one graph. run many realizations for same class of graphs. F. C. Santos & J. M. Pacheco, PRL (2005). F. C. Santos, J. M. Pacheco & T. Lenaerts, PNAS (2006). Social learning / Evolution on graphs k=2 k=3 k=4 k=1 Imitate a random neighbor with a probability that increases with the payoff difference. k=4 k=2 p = 1+ e β (Π B Π A ) [ ] 1 24

25 What about cooperation? The topology of real social networks seems to maximize the changes of reaching high levels of cooperation [ Santos, Pacheco, Lenaerts, PNAS 2006] cooperation on the star(s) the most advantageous situation for a defector is: defectors are victims of their own success!! 25

26 Groups, kin, reciprocity, memory, reputations, moral systems, signals, gossip, networks, etc etc Are we missing a lot? YES!!! Many of these mechanisms rely on the 2-person nature of cooperative interactions YET Missing pieces Often cooperation dilemmas involve many individuals simultaneously 26

27 Missing pieces Often cooperation dilemmas involve many individuals simultaneously Public goods games Does not contribute Collective investment Pays a cost c (N=4) Pays a cost c Pays a cost c Missing pieces Often cooperation dilemmas involve many individuals simultaneously Public goods games Does not contribute Collective investment Equal sharing of collective profits Pays a cost c (N=4) Pays a cost c Pays a cost c 27

28 Global warming Data: CDIAC/GCP/IPCC/Fuss et al 2014 Saving"the"planet"requires" "coopera/on"!" The problem of discounting tomorrow 28

29 the$nature$of$the$game$we$cannot$afford$to$lose" $failure$of$global$summits$has$been$agributed$$ (among$other$issues)$to:$$ $ $ o $overall$percep=on$of$risk$is$too$small$ $ o $uncertain=es$regarding$targets$to$be$met$ $ o $conflic=ng$policies$between$rich$and$poor$par=es$$ $ o $no$ins=tu=on(s)$to$monitor$or$sanc=on$those$not$ abiding$to$agreements$$ the$nature$of$the$game$we$cannot$afford$to$lose" """""" $$$$$coopera=on$between$individuals$?$ $$$$$coopera=on$between$ci=es$or$regions?$ $$$$$coopera=on$between$countries$?$$$ $$$$$$ " " " """""coopera/on"between"all"countries@once"?"" """"""""&"how"about"" """""mul/ple"agreements"involving"few"countries,"states"or"regions?" " 29

30 International climate agreements? There must be a better way Evolution and a self-organization analysis confirms that a bottom-up approach leads to better cooperative standards Think"globally," act"locally " is this utopia? any feasibility for bottom-up attempts? A bottom-up approach to common-pool resources 30

31 Western$Climate$Inicia=ve$(WCI),$Regional$Greenhouse$Gas$ Inicia=ve$(RGGI),$Midwestern$Governors$Associa=on$(MGA),$...$ WCI$ 03$ $7$North%Eastern$States$ 05$ $MA$&$RI$dropped$out$ 07$ $MA$&$RI$rejoined$ 08$ $MD$&$NH$joined$ 11$ $NJ$dropped$out$$ ReGGIe$ 07$ 08$ 07$ 07$ 08$ 08$ 08$ 08$ 08$ 08$ MGA$ 07$ 07$ 08$ 08$ 08$ 08$ 08$ 08$ these$state/province$ini=a=ves,$regionally$based,$aim$at$aggrega=ng$into$a$ wider$and$stronger$structure,$called$north$america$2050$ Complex GAIPS/INESC-ID - Evolution of cooperation (reciprocity, fairness, reputations, etc) - Network Science - Stochastic Evolutionary Dynamics - Learning in large populations - Computational epidemiology - Preferences and opinion dynamics - Ecology of cancer - Urban growth, planning and local governance - Climate change governance - Historical dynamics - Experimental economics etc. 31

32 today s class in a nutshell 32

33 Natural selection leads to the tragedy of the commons 33

34 34

35 35

36 "Climate"change"is"the"defining"issue"of"our"age."" "It"is"defining"our"present."Our"response"will"define"our"future. $ $Ban$Ki%Moon,$UN$Secretary%General,$23$Sept.$2014$ António Araújo, 2011 franciscocsantos@ist.utl.pt 36