Sympathy and Interaction Frequency in the Prisoner s Dilemma

Transcription

1 Sympathy and Interaction Frequency in the Prisoner s Dilemma Ichiro Takahashi and Isamu Okada Faculty of Economics, Soka University itak@soka.ac.jp Abstract People deal with relationships at home, workplace, and local community. There are various levels of friendships. According to the level of social interaction, different degrees of sympathy and interaction frequency exist. This paper purports to examine how interaction frequency influences the development of sympathy factor, and thereby the emergence of social action patterns in an agent-based simulation model when learning takes place locally. In our artificial society, an agent frequently interacts with other agents who are closely located, and infrequently with those who are remotely located. By endogenizing the sympathy factor based on behavioral patterns of the agents, we examine how the interaction frequency affects patterns of emerging social actions. The results obtained are as follows: () the closer the distance and the relationship between agents, the more cooperative the behavior of agents are; (2) since cooperative behaviors promote the emotion of sympathy, frequent interactions induce more cooperative behaviors through strengthening sympathy than they do without this effect of sympathy. () and (2) together give rise to three patterns of social behaviors: cooperation with tolerance among players who are in a close relationship, reciprocal cooperation among players in a moderately close relationship, and no cooperation among strangers. Introduction People care not only about their own selfinterest, but also about the well-being of others. How much they care about others depends on how generous the others have been to them. This paper shows that such emotion of sympathy can be rationally explained by frequency of social interactions. People have families, colleagues, and acquaintances. There are various levels of friendships. According to the level of social interaction, there exist different degrees of sympathy and interaction frequency. The way they behave in interaction with their families is naturally different from the way they behave toward their friends. On each of these different levels of social interactions, they also seem to learn how to behave socially from their near kin or close friends. This paper purports to examine how interaction frequency influences the development of sympathy factor, and thereby the emergence of social action patterns in an agent-based simulation model when the learning takes place locally. In an iterated game, without considering reputation or altruism, players can still have an incentive to develop cooperation upon reciprocity if they do not know when the game will end. Axelrod (984) uses the notion of shadow of the adaptive future to account for the expected increase in benefit from cooperation when a player can expect an interaction with another player that is likely to last longer. Cohen et al. (999, 2) investigate how the emergence and maintenance of cooperation is influenced by variations in three key dimensions: () strategy space of agents, (2) interaction processes that control how agents interact with each other, (3) adaptive processes that govern the selection of agents strategies over time. They found that the second dimension is most important in inducing cooperation. The key (attribute of condition) is the relative stability of agents neighborhoods over time as in-

2 duced by the underlying interaction process. The preservation of interaction structure across learning periods makes an enormous contribution to the emergence and maintenance of mutual cooperation. They call it context preservation, which creates a shadow of the adaptive future. By having established a systematic framework for studying the iterated prisoner s dilemma played by simulated agents successfully, Cohen et al. (999) reported comprehensive results on the emergence of social action patterns. In order to readily interpret the results of this paper in their general framework, we keep our model similar to theirs in some dimensions while extending the others. Specifically, we maintain the same strategy space and adaptive processes while modifying the social interaction process so that each individual agent has multilayered neighborhoods that classify the other agents according to the level of interaction frequency. In addition, the structure of neighbors allows the agent to imitate better strategies employed by his immediate neighbors, which seems to be more realistic. Put anthropomorphically, we learn how to behave socially from our parents and not from our neighbors. Neighbors with different levels of social structure carry different levels of psychological intimacy among themselves. We assume that agents have a fixed level of sympathy toward others if they live in the neighborhood of the same interaction frequency level. In other words, all intimate players share the same sympathy factor as do the distant players among themselves. Depending on past personal interactions, individuals would be inclined to develop different levels of sympathy factor toward others. However, in order to single out the effect of interaction frequency on sympathy factor, we assume all players living in the same level of vicinity share a common sympathy factor. Another justification for this assumption is that we are interested in the behavior of people who are motivated by social goals with normative rather than personal emotion. Quite frequently or not, people tend to display actions that are confined by social expectations contrary to their will. In our artificial society, except for the most distant neighbors, the number of neighbors is proportional to the distance from a particular individual. A fixed number of players are randomly selected in each period to play a repeated game of prisoners dilemma. Thus, there is oneto-one correspondence between interaction frequency and physical distance: people interact more frequently with those who live closer to him or her. Next, we attempt to endogenize the sympathy factor based on behavioral patterns of the agents. The level of sympathy factor must be consistent with actual behavioral patterns of players. Even intimate emotion among friends would erode over time if they do not help each other consistently. We focus on equilibrium levels of sympathy factor. In equilibrium, the level of neighborhood-specific sympathy reflects the collective kindness in the neighborhood computed from the observed behavior, according to the kindness function developed by Rabin (993). We obtained the following two results: () the more frequently people interact and the more sympathetic they feel toward others, the more likely it is that cooperative behaviors emerge; (2) since cooperative behaviors promote the emotion of sympathy, frequent interactions induce more cooperative behaviors through strengthening sympathy than they do without this effect of sympathy. These two results imply that emerging social action patterns are tolerant cooperation in an immediate neighborhood, reciprocal cooperation in a close neighborhood, and no cooperation in a remote neighborhood. Cohen et. al (999) p. 3. A context-preserving social structure means that strategies resulting from today s interactions will be near each other tomorrow. This fosters the emergence of mutually-compatible strategies. (Cohen et. al (2) p. 3) 2

3 Our finding is consistent with the experimental results by Duffy and Ochs (23). They found that a cooperative norm emerges in the fixed pairings treatment while it does not in random matching pairings. The effect of interaction frequency is closely linked with localization of interaction analyzed by Hoffmann (999) and Kirchkamp (2). Hoffmann concludes that the localisation of interaction has an ambiguous effect on the emergence of cooperation. In contrast, Kirchkamp shows the localisation of interaction has substantial effect, inducing more cooperation. Their findings result from differences in the dimensions of their torus: Hoffman uses one-dimensional torus and Kichkamp uses twodimensional torus. In order for cooperation to prevail, a defector whose strategy is always to defect must die out in the sea of tit-for-tat strategies. The number of neighbors with whom a player interacts should be large enough so that a tit-fortatter can outperform defectors by cooperating among themselves. This requirement is satisfied by a two-dimensional toroidal lattice but not by a one-dimensional lattice. The more localized the interaction, the higher the level of context preservation. Kirchkamp s result is consistent with Cohen et. al (2). To investigate the effect of interaction frequency on cooperation, we adopt a two-dimensional lattice. Why does the context preservation or localization of interaction result in more cooperation? Let us consider Player A. Player A is allowed only one repeated game strategy against n opponent players where n can be one or greater one. Player A has to consider the games played against all of his opponents and to choose the best strategy overall. In turn, each of the opponent players considers Player A s decisionmaking process and chooses his strategy accordingly. The more the opponents there are, the less influence one opponent s strategy will have on Player A s action. In other words, the shadow of adaptive future becomes weaker as the number of opponents, n, increases. This is because Player A revises her strategy in response to all of the opponents behaviors. We find that the smaller the parameter n is, the more often cooperative behavior will emerge. In our model, Player A interacts with a fixed number of opponents, m, chosen randomly from a pool of n players. Therefore, interaction frequency, m/n is crucial in inducing cooperative behaviors 2. Section 2 describes the model, Section 3 presents simulation results for exogenous sympathy factor, Section 4 extends the model and considers the case of endogenous sympathy factor and discusses the results, and Section 5 gives conclusions and future extensions. 2 The Model Following the example of Cohen et al. (999, 2) we chose the Iterated Prisoner s Dilemma (IPD) as the basis of our model. We will deal with models of sympathy a player receives, in addition to pecuniary payoffs, psychological reward or pain depending on how they act. Put simply, if one plays a PD game with one of his friends, he does not want to defect unless the friend stops reciprocating cooperation. One will also feel hurt if his friend does not return his cooperation. Table shows a payoff matrix for a one-shot game in which a player is sympathetic to his opponent. Specifically, players payoffs in every period are the sum of pecuniary payoffs and emotional payoffs represented by α. 3 We assume α [, ] so that for any value of α, the stage game payoffs satisfies the property of PD. Positive value of α means that the agents have sympathy toward his or her partner while 2 Cohen et. al (2) do not use the term of frequency but their 2DK (Two-Dimensional torus, Keep Neighbors) and RWR (Random With Replacement) of social structure can be interpreted as the special cases of interaction frequency. 3 Geankoplos and Pearce (989) analyze this psychological game in which α changes at the end of every period by an amount proportional to the unexpected choices. 3

4 negative α means that he or she harbors malice toward the partner. Table. Payoff Matrix of PD Column Player Row Player Cooperate Defect Cooperate 2, α,3 α Defect 3 α, 3+α 2, 2 Alternatively, we can regard the payoffs as those of a warm glow model in which the players feel satisfied by cooperating and feel sorry by defecting in the PD game. In Table 2, these psychological effects are represented by β. Table 2. Payoff Matrix of warm glow model Column Player Row Player Cooperate Defect Cooperate 2+β, 2+β 3+β, 3 β Defect 3 β, 3+β 2 β, 2 β We can divide the above payoffs by +.5β, while preserving strategic equivalence, to make it comparable with those of Table, yielding the same payoff matrix as Table with the α s being replaced by 5β 2+β s. Since the sum of the payoffs of the strategy pair (C, D) is always zero, the total sum of payoffs across all the players measures the overall degree of cooperation. 2. The Strategy Spaces In our model, an agent plays four rounds of PD with the same partner every period. Throughout the period, the agent sticks to one IPD strategy, which is specified by a vector (i, p, q). The numerical numbers i, p, and q represent the player s probability of cooperation on the initial move of a game, the probability of cooperation on the next move after the partner has cooperated, and the probability of cooperation on the next move if the partner has defected, respectively. In Table 3, the left entries give the 4 move pecuniary scores and the right entries give total pecuniary scores attained by the row strategy when playing with the column strategy. The binary strategy space is error free, so when the agents are restricted to using binary strategies, their outcomes are completely deterministic. We will only examine continuous strategy space, in which a vector (i, p, q) representsa strategy that a player may adopt while playing the IPD game. Sometimes, there are experiences in which the resulting outcome is something different from what was intended. To allow for this kind of mistakes, it is assumed that neither p = nor p =. The same assumption is made for q. We also follow Cohen et. al (999) in determining (p, q) space: p {/32, 3/32,...,3/32} and q {/32, 3/32,...,3/32}. For expositional convenience, we will focus on the strategy space with i = p. Furthermore, to facilitate intuitive interpretation of emerging behavioral patterns, we introduce four quasi IPD strategies. An IPD strategy is called quasi-all-c if p.5 andq.5, quasi-tft if p.5 and q<.5, quasi-atft if p<.5 andq.5, and quasi-all-d if p<.5andq< Structure of Social Interaction At the start of each run, players are placed at random on a N N torus, with N being an odd number 4. For one variant strategy space, agents are restricted to the following four binary strategies:. i = p = q = : Always cooperate (all-c). 2. i = p =, q = : Mimic opponent s last move, i.e., the tit-for-tat strategy (TFT). 3. i = p =, q = : Anti-tit-for-tat strategy (atft). 4. i = p = q = : Always defect (all-d). 4 Thus, N N players are assigned strategies, (i, p, q) vectors, and then placed on this 2-dimensional torus. 4

5 Table 3. Score per move and total score for the pair of binary strategies Column Player Row Player all-c TFT atft all-d all-c TFT atft all-d Each player has M different levels of social interaction, where M (N + )/2. There are 4 m neighbors on m( M )th level of social interaction, and (N 2 ) 2M(M ) neighbors on M th level. A player has 4 immediate neighbors of level social interaction, and 8 neighbors of level 2 who are the immediate neighbors of Level neighbors. Figure illustrates the structure of social interaction for (N, M) =(7, 3) case. Except for the Mth level neighbors, a neighbor of Level m is located at distance of m from the player where the distance is measured as the sum of row and column distances between the two agents Figure. The Levels of Social Interaction with M=3, and N=7 At the beginning of each period, each agent selects 4 partners at random from each level of neighbors to play four rounds of PD. Thus, the player interacts with each of Level m neighbors with probability /m if m<m. The parameter m is negatively related with frequency of interaction. Because the agent is also chosen by his neighbors, he plays 8 games of IPD on average, in each neighborhood. One may interpret Level neighbors as family members, Level 2 neighbors as close friends, and Level M players as complete strangers. Moving is not allowed, so the players keep the same neighbors throughout a simulation run. In every period, each agent has M possibly different IPD strategies, each of which is applied to different level of neighbors. At the end of each period, the player updates his M IPD strategies. 2.3 Adaptive Processes All agents change their M strategies at the end of each period. As stated above, they absorb behavioral patterns from their Level von Neumann neighbors. We use BMGA (Best-Met Genetic Algorithm hybrid) employed in Cohen et al.(999). 2.4 Performance measures An important indicator of the overall performance is the average score of all the agents throughout a simulation, which is denoted by SocPayoff 5. A population average close to 2, indicates the population almost always cooperating whereas an average close to 2 indicates the population almost always defecting. We owe the following measures to Cohen et al. (999):. GenToHigh: This indicates how many periods it takes for the society to reach the threshold performance FracHigh: This is the fraction of the periods the population spends above the.6 5 Since the psychology factor does not appear in the sum of payoffs to any pair of players, in measuring the population average, we do not need to distinguish between the pecuniary score and the score including the sympathy factor. 5

6 threshold after the GenToHigh threshold is first achieved. 3. FracLow: This is the fraction of the periods the population spends below the.6 threshold after the GenToHigh threshold is first achieved. Although there is no objective value to discern whether the population performs well or poorly, we use the threshold score of.6, above which the performance is considered high, and that of.6, below which is low. Table 4. Overall results of the simulation α Level neighborhood SocPayoff GenToHigh FracHigh FracLow all-c (%) TFT (%) atft (%) all-d (%) Level 2 neighborhood SocPayoff GenToHigh FracHigh FracLow all-c (%) TFT (%) atft (%) all-d (%) Level 3 neighborhood SocPayoff GenToHigh FracHigh FracLow all-c (%) TFT (%) atft (%) all-d (%) Note: IPD strategy (%), i.e., all-c (%) means the ratio of agents using quasi-all-c alpha= -.5 alpha=. alpha=.5 Figure 2. Average Social Payoffs for Level 6

7 3 Simulation and Results in the Model with Exogenous Sympathy Factor 3. The Setting of Simulation First, the sympathy factor is fixed at one of equally spaced 2 levels α = k/, k = (,,...,2). If α =., there is no longer a dilemma because cooperation becomes a weakly dominant strategy 6. The smaller the value of α, the more serious will be the conflict of interests between the two players. For each value of α, there are runs of simulation (with different random seeds), each of which has 5 periods. In our simulation, we set (N, M) =(7, 3). The initial neighbor-specific IPD strategies of 49 agents are spread randomly across the (p, q) space. 3.2 The Effect of the Interaction Frequency Table 4 summarizes the average results over runs of the virtual experiments 7. For every value of α, the average social payoffs increase with the intensity of social interaction. This result is consistent with the finding by Cohen et al. (999, 2) and is intuitively obvious 8. The shadow of adaptive future is prevalent when a player expects to play with the same partner more often. Table 4 indicates that, for larger values of α, the fraction of quasi-tft strategies in Level neighborhood is smaller than in Level 2 neighborhood. In these cases, quasi-all-c begins to prevail among Level neighbors because cooperation is more attractive for these sympathetic agents and it is less likely to be exploited by defectors. 3.3 The Effect of the Sympathy Factor Now, let us fix the level of neighborhood. For each level of neighborhood, as the sympathy factor α increases, so does the average social payoffs. The fraction of quasi-tft strategy tends to increase with the values of α, but they do not always increase. The fractions of quasi-all- C as well as those of the sum of quasi-tft and quasi-all-c, however, increase unambiguously. Figure 2, measuring SocPayoff against simulation period, shows that in Level neighborhood, the effect of increase in the psychological factor seems insignificant when the α increases from.5to.. Cooperation emerges much more quickly when α rises from. to.5 In Level 2 neighborhood, social performance drastically improves if α is raised from.5 to.. In other words, cooperative behavior can prevail in Level 2 neighborhood if people are only moderately sympathetic with each other. In Level 3 neighborhood, in contrast, the emergence and maintenance of cooperation requires a high level of sympathy. These results reveal that the sympathy factor and the frequency of social interaction are close substitutes in inducing cooperation among the agents. 4 Endogenous Sympathy Factor We have thus far assumed that α s are fixed exogenously. Since social interaction generates the feeling of intimacy 9, this section endogenizes α: the α in a neighborhood of a specific level of interaction depends on how people behave. One 6 This case is called Avatamsaka Game. 7 The table reports the results for selected values of α only. 8 Regarding the mechanism behind the emergence and maintenance of TFT, please refer to pp of Cohen et al. (999). 9 According to Hamilton (964), the social behavior of a species evolves in such a way that in each distinct behaviorevoking situation the individual will seem to value his neighbors fitness against his own according to the coefficients of relationship appropriate to that situation. (p.8) 7

8 can regard α as a type of social norm which guide people to weigh the importance of other s payoff in making their decisions, e.g. a large value for parents and small value for unknown strangers. The values of α are considered to be fairly constant over time. Nonetheless, when the parameter values become inconsistent with the way people actually behave, they should revise α to reflect the reality. The feeling of intimacy develops toward those who are benevolent. We measure how kind a player is for another player with the kindness function defined by Rabin (993). For example, if player i always cooperates with player j, player j begins to feel a strong sense of intimacy toward player i. On the contrary, if player i always defects against player j, thenplayerj feels animosity toward player i. Nonexistence of interaction results in a neutral value of α, i.e., zero. From simple computation, in a stage game of prisoner s dilemma, the value of kindness of cooperation a la Rabin is.5 and that of defect is.5, irrespective of the opponent s actions. The average kindness, in our multi-period simulation model, needs to be normalized so that their values fall between minus one and plus one. This normalization condition is satisfied if we compute the kindness of player i to player j as follows: subtracting the total number of player i s defection from the total number of i s cooperation, then dividing it by the total number of stages in the simulation periods. To clarify the point, let us consider the following example. Suppose that the stage game outcomes of two players who interact only in period and 3 in 4 simulation periods are listed as follows: i CCDD DCCC j DDDD CCCC Since player i cooperates 5 times and defects 3 times out of 6 stages, his kindness to player j is computed as (5 3)/6 = /8. By computing the aggregate kindness across all is andjs who belong to the same level of neighborhoods, we obtain the collective kindness for each level of neighborhoods. In Figure 3, we measure exogenous parameter values of α on the horizontal axis and computed values of kindness on the vertical axis. Each of three curves plots the aggregate kindness performed among all the players with a particular level of interaction frequency. In equilibrium, this computed value of kindness must be equal to α value. Thus, equilibrium values of αs are derived as intersections of the curves and the 45 degree line equilibria level- level-2 level-3 Figure 3. Computed values of kindness and the equilibrium values of α By inspection, the equilibrium values of α are.74 for Level frequency,.28 for Level 2 frequency, and.46 for Level 3 frequency. Figure 4 displays snapshots of (p, q) distributions at equilibrium values of α for the three levels of interaction frequency. 53 % of agents use quasi-all-c strategies to players who are in Level neighbors whereas 98 % of them use quasi-tft to Level 2 neighbors. In contrast, 69 % of them adopt quasi-all-d to Level 3 neighbors. In short, three patterns of social behavior emerge: tolerant cooperation among closely linked players, reciprocal cooperation among intermediately linked players, and no cooperation among distant agents. Figure 5 summarizes how the interaction frequency, cooperative behaviors, and the emotion of sympathy are interrelated in our model. As seen in the previous section, through the effect.8 α 8

9 q.5 level of the shadow of the adaptive future, frequent interactions induce cooperation among players. The degree of the consequence of one s action toward one s opponent or partner will depend on how often they play with each other. Cooperation, in turn, generates emotions of sympathy through the kindness function, which will promote cooperation. Thus, there is positive feedback between cooperation and sympathy. Interaction frequency can be seen to cause sympathy through cooperative behaviors. This partially explains why there is a kind of social norm dictating that one should care about the well-being of the other with whom he interacts often..5 p level 2 Interaction Frequency (4) Social Norm () Altruism (2) Shadow of the adaptive future Cooperation q Sympathy (3) Kindness function.5 Figure 5. Conceptual Diagram q.5.5 p level 3.5 p Figure 4. (p,q)-phase snapshots at Period 5 for the equilibrium values of α 5 Conclusion We constructed an artificial society where an agent frequently interacts with another who is closely located, while infrequently interacts with a remotely located player. Since psychological factor plays an important role in social interaction, we have introduced sympathy factor in the model. We have examined how these factors affect emerging social behavioral patterns. We have also investigated how interaction frequency affects players behavioral pattern when the latter endogenously generates a sense of intimacy. We obtained the following results: the closer the agents are located and the closer they are attached emotionally to each other, the more likely cooperative behavior prevails. In equilibrium, the sympathy factor is positively correlated with the interaction frequency and these factors reinforce each other, resulting in three patterns of social behavior: cooperation with tolerance among players in immediately close relationship, reciprocal cooperation among players in moderately close relationship, and no cooperation among strangers. 9

10 In this experiment, we have assumed that a player, having no memory of his partner s action in the previous period, believes that the partner played cooperatively. This means our results are significantly biased toward cooperation, which needs to be corrected by incorporating agents with long memory. To enhance the validity of the model, the sympathy factor needs to be individualized and the level of interaction frequency modified to a continuous parameter. References [Axelrod, 984] Axelrod, R.: The Evolution of Cooperation, Basic Books, New York. [Cohen M. D. et al., 999] Cohen, M. D., R. L. Riolo, and R. Axelrod: The Emergence of Social Organization in the Prisoners Dilemma: How Context-Preservation and other Factors Promote Cooperation, SFI Working Paper Series. [Cohen M. D. et al., 2] Cohen, M. D., R. L. Riolo, and R. Axelrod: The Role of Social Structure in the Maintenance of Cooperative Regimes, Rationality and Society, 3, pp [Duffy and Ochs, 23] Duffy J. and J. Ochs: Cooperative Behavior and the Frequency of Social Interaction, mimeo, University of Pittsburgh. [Geanakoplos and Pearce, 989] Geanakoplos. J. and D. Pearce: Psychological Games and Sequential Rationality, Games and Economic Behavior,, pp [Hamilton, 964] Hamilon, W.D.: The general evolution of social behavior. I, II., Journal of Theoretical Biology, 7, pp [Hoffmann, 999] Hoffmann, R.: The Independent Localizations of Interaction and Learning in the Repeated Prisoner s Dilemma, Theory and Decision, 47(), pp [Kirchkamp, 2] Kirchkamp O: Spatial Evolution of AUtomata in the Prisoners Dilemma, Journal of Economic Behavior and Organization, 43, pp [Rabin, 993] Rabin, M.: Incorporating Fairness into Game Theory and Economics, The American Economic Review, 83(5), pp