1. Evolutionary Biology
Download icon

Evolution of empathetic moral evaluation

  1. Arunas L Radzvilavicius  Is a corresponding author
  2. Alexander J Stewart
  3. Joshua B Plotkin  Is a corresponding author
  1. University of Pennsylvania, United States
  2. University of Houston, United States
Research Article
  • Cited 1
  • Views 2,122
  • Annotations
Cite this article as: eLife 2019;8:e44269 doi: 10.7554/eLife.44269

Abstract

Social norms can promote cooperation by assigning reputations to individuals based on their past actions. A good reputation indicates that an individual is likely to reciprocate. A large body of research has established norms of moral assessment that promote cooperation, assuming reputations are objective. But without a centralized institution to provide objective evaluation, opinions about an individual’s reputation may differ across a population. In this setting we study the role of empathy–the capacity to form moral evaluations from another person’s perspective. We show that empathy tends to foster cooperation by reducing the rate of unjustified defection. The norms of moral evaluation previously considered most socially beneficial depend on high levels of empathy, whereas different norms maximize social welfare in populations incapable of empathy. Finally, we show that empathy itself can evolve through social contagion. We conclude that a capacity for empathy is a key component for sustaining cooperation in societies.

https://doi.org/10.7554/eLife.44269.001

eLife digest

When meerkats have pups, they employ an individual to stand guard and warn the others of potential dangers and predators, putting their own life at risk. What seems like a selfless act is actually a common behavior found throughout the animal kingdom. But rather than acting out of concern for another ones wellbeing, it is considered to be an altruistic behavior towards kin, where an individual sacrifices its own reproductive success for the sake of the reproductive fitness of its entire clan.

In human societies, however, people often act altruistically towards unrelated individuals and have developed sophisticated systems of moral evaluation to decide who is worthy of cooperation and likely to reciprocate a favor. In other words, individuals will only help those who have a good reputation for being altruistic themselves. However, for this system to work, reputations need to be public knowledge, and societies need to agree on everyones reputations. But what happens when opinions about an individual's reputation are private and vary across a population?

Now, Radzvilavicius et al. wanted to find out whether altruism can emerge when people have different opinions about each others moral reputations. To do so, they used a so-called evolutionary game theory a mathematical description of how strategies change in a population over time. In their model, each individual could decide if they wanted to pay a personal cost to create a benefit for another individual. Each participant decided whether to act altruistically based on the reputation of the recipient; observers could update the individuals reputation based on their behavior.

The mathematical model revealed that when people are more empathetic and able to put themselves in someone elses shoes, altruism tends to spread over time. When people take into account different opinions and form moral judgements from another person's perspective, the population can sustain a higher level of cooperation. Moreover, the capacity for taking another person's perspective can itself evolve and remain stable in a population meaning that those individuals who evaluate each other empathetically tend to do better, and empathy spreads through social influence.

These findings can help us understand how empathy might have evolved in societies that value reputation as a means of reciprocity. A next step could be to test the theory developed by Radzvilavicius et al. in manipulative experiments, or to compare the theory to field data on reputations and behavior in online interactions.

https://doi.org/10.7554/eLife.44269.002

Introduction

Widespread cooperation among unrelated individuals in human societies is puzzling, given strong incentives for exploitative cheating in well-mixed populations (Ohtsuki et al., 2006). Theories of cooperation based on kin selection, multilevel selection, and reciprocal altruism (Nowak, 2006) provide some insight into the forces driving prosocial behavior, but in human societies cultural forces appear to be of much greater importance (Gintis et al., 2003; Buckholtz and Marois, 2012). One possible explanation rooted in cultural norms is that humans condition their behavior on moral reputations: the decision to cooperate depends on the reputation of the recipient, which itself depends on the recipient’s previous actions (Leimar and Hammerstein, 2001; Nowak and Sigmund, 2005). Altruistic behavior, for instance, may improve an individual’s reputation and confer the image of a valuable member of society, which attracts cooperation from others in future interactions (Nowak and Sigmund, 1998).

Game theory has been used to study how reputations might facilitate cooperation in a population engaged in repeated social interactions, such as the Prisoner’s Dilemma or the Donation Game (Rapoport et al., 1965; Nowak and Sigmund, 2005). In the simplest analysis an individual’s reputation is binary, either ‘good’ or ‘bad’, and the strategy of a potential donor depends on the recipient’s reputation (Ohtsuki and Iwasa, 2004) – for example, cooperate with a good recipient and defect against a bad recipient. A third-party observer then updates the reputation of the donor in response to her action towards a recipient. Reputation updates are governed by a set of rules, known as a social norm, which prescribe how an individual’s reputation depends on her actions during social interactions.

A common simplification in models of moral reputations is that all reputations are both publicly known and fully objective (e.g. Nowak and Sigmund, 1998; Pacheco et al., 2006; Ohtsuki et al., 2009; Sasaki et al., 2017). This means that all individuals know the reputations of all members of the society, and personal opinions about each individual’s reputation do not differ. This is a reasonable assumption if there is a central institution that provides objective moral evaluation, or if opinions regarding reputations homogenize rapidly through gossip (Nowak and Sigmund, 2005). But these conditions are rare in human populations, and opinions about reputations typically differ among individuals – for instance, because observers use different moral evaluation rules, or because of divergent observation histories, or errors. In these cases a single focal individual may have different reputations in the eyes of distinct observers, resulting in much lower rates of sustained cooperation (Okada et al., 2017; Hilbe et al., 2018).

Moral relativity – that is, when an individual’s reputation depends on the observer – introduces an interesting and overlooked ambiguity in how an observer should evaluate a donor interacting with a recipient. One approach is to assume that the observer can refer only to her own opinion of the recipient’s reputation, when evaluating a donor. We call this an ‘egocentric’ judgment, because the observer makes moral evaluations solely from her own perspective (Figure 1a). Alternatively, an observer can perform a moral evaluation that accounts for the recipient’s reputation in the eyes of the donor (Figure 1b). This ‘empathetic’ case requires that the observer take the perspective of another person, which assumes some capacity for recognizing the relativity of moral status.

Empathetic and egocentric modes of moral assessment.

An observer updates the reputation of a donor based on the donor’s action towards a recipient and the recipient’s reputation. (a) An egocentric observer (E=0) forms a moral judgment based on the recipient’s reputation as seen from her own perspective. (b) An empathetic observer makes a judgment based on the recipient’s reputation in the eyes of the donor (E=1). (c) More generally the empathy parameter E corresponds to the probability that observer will assess the donor using the donor’s – not the observer’s – perspective of the recipient’s reputation.

https://doi.org/10.7554/eLife.44269.003

Psychological studies implicate empathy as potent driver of prosocial and cooperative behavior in human societies (Eisenberg and Fabes, 1990; Batson et al., 1997; Batson and Moran, 1999; Decety et al., 2016). The cognitive capacity to intentionally adopt the subjective perspective of another individual is known as a key component of empathetic behavior (Davis, 1983). This so-called ‘perspective-taking’ component of empathy is in turn related to the theory of mind, or the ability to attribute mental states to explain and predict the behavior and emotions of other individuals (Premack and Woodruff, 1978; Hughes and Dunn, 1998). Empathetic perspective-taking generally develops between infancy and pre-school years, with at least some components learned from parents (Krevans and Gibbs, 1996; Knafo et al., 2008; Farrant et al., 2012). And yet empathetic behavior is not universal, as even adults often fail to empathize, especially in interactions with unfamiliar social or different cultural groups (Cikara et al., 2014). In the context of social dilemmas, it has been suggested that empathy might play a role in evaluating the ‘fairness’ of opponents’ actions and predicting their strategies (Singer and Fehr, 2005). However, the role of empathy for moral evaluation of social behavior has not been thoroughly studied. In particular, there is currently no formal way to analyze the role of empathetic perspective-taking in game-theoretic models of human cooperation.

Here we work to resolve the ambiguity of subjective moral judgment by introducing the concept of empathy into game-theoretic analyses of cooperation. We treat empathy E as the probability that an observer will form moral evaluations from the perspective of another person (Figure 1c). First we investigate the effects of empathy on the level of sustained cooperation under simple social norms, while players update their strategies. Next we consider evolution of empathy itself using the formalism of adaptive dynamics; and we determine conditions under which empathy will evolve and remain evolutionarily stable.

Model

A model of moral assessment

We consider a population of individuals who can choose between cooperation or defection in a sequence of pairwise, one-shot donation games. In a given game the donor must choose whether or not to cooperate with the other player. If a donor cooperates she pays the cost of an altruistic act c, while the recipient receives the benefit b>c; if the donor defects she incurs no cost, and the recipient receives no benefit. The donation game is therefore a special case of the prisoner’s dilemma (Rapoport et al., 1965) characterized by the payoff matrix (bccb0).

The decision to cooperate or defect depends on the donor’s strategy S=(p,q), which prescribes an action conditioned on the reputation of the recipient. Here p and q denote the probability that the donor will cooperate with a ‘bad’ or a ‘good’ recipient, respectively. As is common in the game-theoretic literature with reputations (Sasaki et al., 2017), we assume that both p and q are in {0,1}, and so we focus on three strategies: Always Cooperate ALLC, S=(1,1)); Always Defect (ALLD, S=(0,0)); and Discriminate (DISC, S=(0,1)), which cooperates when paired with a recipient with a good reputation and defects against a recipient with a bad reputation. We also relax this assumption and report qualitatively similar results for the continuous strategy space (p,q)[0,1]2. (We neglect anti-discriminators S=(1,0), which can never achieve high frequency.)

Players’ reputations in the eyes of each member of the society are updated according to a social norm. In general, the update rule prescribed by a social norm can depend on the entire history of donor-recipient interactions, including the reputations of all interacting parties (Santos et al., 2018). Complex rules of moral evaluation, however, require high cognitive ability and effort that seem unrealistic in many real-world social interactions. Moreover, relatively simple ‘second-order’ norms of moral assessment, which update a donor’s reputation based solely on the donor’s action and the recipient’s reputation, tend to outperform more complex social norms (Santos et al., 2018).

We consider second-order social norms, which can be encoded by a binary matrix Nij. The row-index i indicates donor’s action, i=1 for defect or i=2 for cooperate; and the column-index j indicates reputation of the recipient, j=1 for bad or j=2 for good. We focus on the four second-order norms that are most prominent in the literature: Stern Judging (GBBG), Simple Standing (GBGG), Scoring (BBGG), and Shunning (BBBG). For example, under Stern Judging (SJ) or Simple Standing (SS) an observer will assign a good reputation to a donor who punishes a recipient with a bad reputation, by defection. Whereas under Shunning (SH) or Scoring (SC) an observer will assign a bad reputation to a donor who defects against any recipient, regardless of the recipient’s reputation. Following (Sasaki et al., 2017) we also allow for errors in strategy execution and in observation: a cooperative act is erroneously executed as defection with probability e1, while an observer erroneously assigns a bad reputation instead of a good reputation, and vice versa, with probability e2.

The broad consensus in the literature is that Stern Judging is the most efficient norm for promoting cooperation, along with widespread adoption of the discriminator strategy. This result is robust to variation in strategy exploration rates (Santos et al., 2016a), population sizes and error rates (Santos et al., 2016b), and it even extends to the realm of more complex norms of third and fourth order (Santos et al., 2018). Pacheco et al. (2006) have additionally shown that Stern Judging is the norm most likely to evolve in a group-structured population, because it maximizes the collective payoff of the group.

Prior studies of cooperation and moral assessment (Nowak and Sigmund, 1998; Pacheco et al., 2006; Ohtsuki et al., 2009; Sasaki et al., 2017; Santos et al., 2016a; Santos et al., 2016b) have assumed that reputations are objective and common knowledge in the population – meaning that opinions about reputations do not differ among individuals. Here we relax this assumption and allow individuals to differ in their opinions about one another. This reveals an under-appreciated subtlety in the application of norms for updating reputations. Namely, when an observer updates the reputation of a donor interacting with a recipient, the ‘reputation of the recipient’ could be considered either from the observer’s own perspective, or from the donor’s perspective. Under a purely egotistical application of a social norm, the ‘recipient reputation’ means the reputation in the eyes of the observer, who is forming an assessment of the donor. In this case the observer either ignores, or is unaware of, the donor’s view of the recipient. This case corresponds to E=0 in our analysis, the no-empathy model of moral assessment. However, we also analyze the possibility of empathetic moral assessment, E>0, whereby the observer may account for donor’s view of the recipient’s reputation when assessing the donor. In the extreme case E=1, for example, the observer always uses the donor’s view of the recipient’s reputation when applying the social norm to update the donor’s reputation. In general, the parameter E[0,1] determines the probability that an observer uses the donor’s view of the recipient’s reputation, as opposed to her own view, when applying the social norm to update the donor’s reputation (see Figure 1 and Equations 5–8 in Materials and methods).

Results

Empathetic moral judgment facilitates cooperation

To analyze how empathy influences cooperation we first examine strategy evolution with a fixed degree of empathy 0E1. We use the classic replicator-dynamic equations (Taylor and Jonker, 1978; Nowak and Sigmund, 2004) that describe how the frequencies of strategies (ALLD, ALLC, and DISC) evolve over time in an infinite population of players’ strategies reproducing according to their payoffs. To simplify analysis, in the infinite-population model we assume that reputation frequencies reach equilibrium before strategies are updated – that is, the timescale of reputation updating is faster than that of strategy evolution (see Materials and methods). For each of the four most common norms we find bi-stable dynamics (Sasaki et al., 2017). That is, depending on the initial conditions the population will evolve to one of two stable equilibria: a monomorphic population of pure defectors, which supports no cooperation, or a population of cooperative (non-ALLD) strategies that supports some degree of cooperation.

How does empathy influence the prospects for cooperation? Under the Scoring norm, strategic evolution does not depend on the degree of empathy, because this norm ignores the recipient’s reputation when updating a donor’s reputation. For the other norms considered, however, empathy tends to increase cooperation. In particular, the basin of attraction towards the stable equilibrium that supports cooperation (green regions in Figure 2) is always larger when players are more empathetic – meaning that when E is larger, there is a larger volume of initial conditions in the strategy space that lead to the stable equilibrium supporting cooperation.

Figure 2 with 1 supplement see all
Empathetic moral evaluation facilitates the evolution of cooperation.

We analyzed strategy evolution in the donation game under different social norms of moral assessment. Triangles describe the frequencies of three alternative strategies: unconditional defectors (ALLD), unconditional cooperators (ALLC), and discriminators (DISC) who cooperate with good recipients and defect against bad recipients. Red circles indicate the stable (filled) and unstable (open) strategic equilibria under replicator dynamics. The basin of attraction towards a stable equilibrium that supports cooperation (green) is larger as empathy, E, increases, for all three social norms shown. Orange curves illustrate sample trajectories towards stable equilibria. Costs and benefits are c=1.0, b=5.0, and error rates are e1=e2=0.02.

https://doi.org/10.7554/eLife.44269.004

In the case of Shunning and Stern judging, the stable equilibrium that supports cooperation consists of a monomorphic population of discriminators (Figure 2). Not only is the basin of attraction towards this equilibrium larger when a population is more empathetic, but so too is the equilibrium frequency of cooperative actions increased by greater degrees of empathy (Figure 2 and Figure 2—figure supplement 1). And so empathy increases the frequency of outcomes that support cooperation, and also increases the frequency of cooperation at these outcomes.

In the case of Simple Standing the stable equilibrium that supports cooperation consists of a mix of ALLC and DISC strategists. The discriminator frequency at this equilibrium increases with empathy as

(1) fZ=1s2(1-e2)-s+1-ε(1-ε+s(2-ε-e2)-1(1-E)(ε-e2))

where ε=(1-e1)(1-e2)+e1e2 and s=b/c, until it reaches fZ=1. The rate of cooperative play at this mixed equilibrium shows only a weak dependence on the degree of empathy (Figure 2—figure supplement 1).

Aside from the stable equilibria discussed above, for all four norms there is also an unstable equilibrium, with some portion of the population playing ALLD and some portion playing DISC. The frequency of discriminators at this unstable equilibrium is

(2) fZ=cb(ε-e2)(1+(1-E)γNorm),

where

γSH=s2(1e2)s(1+e2(εe2))+e2e2(s21)+E(s2(1e2)s+e2),γSJ=(s2+1)(1e2)ss(1e2)(εe2)E((s2+1)(1e2)s)1+e2+s/2,γSS=(s2+1)(1e2)ss(εe2)(1e2)E((s2+1)(1e2)s)+(s21)(1e2).

When E=1 this expression coincides with the expressions found by Sasaki et al. (2017). This result reflects the sense in which previous studies that assumed no variation in personal opinions about reputations are mathematically equivalent to always taking another person’s perspective (E=1).

Social norms that promote cooperation

In a finite population the frequencies of strategies do not evolve towards a fixed stable equilibrium, but rather continue to fluctuate, irrespective of initial conditions, due to demographic stochasticity. To study the impact of empathy on cooperation in this setting we undertook Monte Carlo simulations. In this model, successful strategies spread through social contagion: a strategy is copied with the probability 1/(1+exp(-w[Π1-Π0])), where w is the selection strength, and Π1 and Π0 are payoffs of two randomly selected individuals (Traulsen et al., 2007; Traulsen et al., 2010, see Materials and methods). In addition to these imitation dynamics, player strategies also change via random exploration at a rate μ. For the sake of simplicity, we assumed that the timescale at which games are played and payoffs are acquired is much faster than the timescales of imitation, exploration, and reputation dynamics, so that each individual plays many games selection and mutation take place (see Materials and methods).

Empathy tends to increase mean levels of cooperation in finite populations under stochastic dynamics (Figure 3), similar to our findings in an infinite population. The effects of empathy are pronounced: the stationary mean frequency of cooperation ranges from near zero to near unity, in response to increasing the value of the empathy parameter E.

Figure 3 with 1 supplement see all
Empathetic moral judgment facilitates cooperation.

The degree of empathy, E, determines which social norms of moral assessment produce the most cooperation and thus the greatest social benefit. (a) The Stern Judging (SJ) norm supports the highest rate of cooperation when empathy is high. But Stern Judging performs poorly under egocentric moral judgment, where Scoring (SC) and Simple Standing (SS) produce greater levels cooperation. The scoring norm (SC) does not depend on reputations, and so it shows cooperation levels that are insensitive to the level of empathy. The Shunning norm (SH) always produces the lowest level of cooperation. (b, c) As the strategy exploration rate μ increases, Stern Judging and Simple Standing become less efficient at promoting cooperation under highly empathetic moral evaluation, but they perform better under egocentric evaluation. All panels show ensemble mean cooperation levels in replicate Monte Carlo simulations of N=100 individuals. Similar results hold in a continuous strategy space (see Figure 3—figure supplement 1).

https://doi.org/10.7554/eLife.44269.006

For high values of empathy, Stern Judging is the most efficient social norm at promoting cooperation, followed by Simple Standing and Scoring. This rank ordering of social norms is consistent with the prior literature (Santos et al., 2016a; Santos et al., 2016b; Santos et al., 2018). However we find a striking reversal from the established view of social norms when individuals are less empathetic. As E0 Scoring promotes the most cooperation, while Stern Judging and Simple Standing engender less cooperation. And so the level of empathy strongly influences the amount of cooperation that evolves, and it even changes the ordering of which social norms are best at promoting cooperation. In particular, Stern judging is the most socially beneficial norm (Pacheco et al., 2006; Santos et al., 2016a; Santos et al., 2016b; Santos et al., 2018) only when individuals account for subjectivity in moral assessment, or when individuals are forced to agree with one another through a centralized institution of moral assessment.

Evolution of empathy

We have seen that empathy promotes cooperation in finite populations with reputation-conditional strategies. However, empathy is not inevitable and not universal in humans (Cikara et al., 2014). It remains unclear if empathy itself can evolve to high levels, and whether a population of empathetic individuals can resist invasion from egocentric moral evaluators. In the following analysis we assume that the degree of empathy in moral evaluation can be observed, inferred or learned, and can therefore evolve through social contagion (imitation dynamics) (Cushman et al., 2017), similar to how social norms are learned (Buckholtz and Marois, 2012). Alternatively, an individual’s capacity for empathetic observations may have a genetic component evolving via Darwinian selection.

We analyze the evolution of empathy using the framework of adaptive dynamics (Geritz et al., 1998). Assuming rare mutations to the continuous empathy trait E[0,1], we calculate the invasion fitness of an invader EI in an infinite resident population with empathy ER by comparing their expected payoffs. We report pairwise-invisibility plots and investigate the evolutionary stability of singular points E, where the gradient of invasion fitness W(ER,EI)/EI (evaluated at EI=ER) vanishes. To support our analytic treatment we also perform Monte Carlo simulations in finite populations subject to demographic stochasticity, where empathy evolves through social copying according to individual payoffs, similar to strategy evolution under imitation dynamics (Traulsen et al., 2007).

Evolution can often favor empathy, depending upon the social norm and the initial conditions. To study empathy dynamics, we initially assume that the population is monomorphic for the discriminator strategy. In the case of the Shunning norm, then, there is a single, repulsive singular value of empathy (Figure 4e) at

(3) ESH=e2c/b+e2-1(1-c/bε-e2)+c/bε-e2.

Such a population is bistable. If the initial level of empathy exceeds ESH the population will evolve towards complete empathy (E=1) and the discriminator strategy will remain stable; but if the initial level of empathy is less than ESH the population will evolve towards complete egocentrism (E=0), at which point the discriminator strategy is no longer stable (Figure 2g) and the population will be replaced by pure defectors. The singular value ESH decreases as the benefit of cooperation b/c increases, permitting a larger space of initial conditions that lead to the evolution of complete empathy (Figure 5c). And so, in summary, under the Shunning norm long-term strategy and empathy co-evolution will tend towards a completely empathetic population of discriminators, especially when the benefits to cooperation are high; or, alternatively, evolution will lead to complete population-wide defection.

Evolution of empathy.

The figure summarizes analytical predictions for empathy evolution under three different social norms, using adaptive dynamics in a population of discriminators, compared to Monte Carlo simulations in finite populations. (a, c, e) White areas in the pairwise invasibility plots show values of E for which the invader’s expected payoff exceeds the mean payoff of the resident population. Orange arrows indicate the direction of predicted evolution. (b, d, f) Monte Carlo simulations in small populations of 100 individuals with recurring mutations to E reflect the predictions of adaptive-dynamics analysis. For each norm, sample trajectories showing all E values in three independent populations are show in colors (red, blue, yellow), with the population means shown in gray.

https://doi.org/10.7554/eLife.44269.008
Figure 5 with 1 supplement see all
Evolutionary stability of empathy.

Circles indicate evolutionarily stable (solid) and unstable (open) singular values of empathy, E, in an infinite population of discriminator strategists. (a) Above a critical benefit-cost ratio b/c, increasing the benefit of cooperation promotes the evolution of high levels of empathy under Stern Judging norm. (b) Under Simple Standing there is a single ESS value for empathy. The highest levels of empathy evolve with high benefits and low costs of cooperation. However, in this case the monomorphic discriminator equilibrium is not stable at the ESS value of empathy. (c) In populations governed by the Shunning norm there are no stable internal equilibria for E, and empathy will evolve to either one or zero.

https://doi.org/10.7554/eLife.44269.009

Similar dynamics occur under the Stern Judging norm. In this case, starting from a monomorphic population of discriminators, there are two singular values for E: an evolutionary repeller ESJ<1/2 and attractor ESJ>1/2 (Figure 4a,b) given by

(4) ESJ=12±14+((1c/b)(e2+ε1c/b)(εe2)2).

Provided empathy initially exceeds the repulsive value evolution will favor increasing empathy towards the attractive value, and the population of discriminators will remain stable. Increasing b/c again favors the evolution of empathy, as it increases the value of the locally stable ESJ>1/2 and also the range of initial values that that lead to ESJ>1/2 through fixation of small mutations (Figure 5a). However, if empathy starts below the repulsive value, selection will favor evolution toward the attractive singular point at ESJ=0, which no longer supports DISC as a stable equilibrium in strategy space (Figure 2a). And so, in summary, under Stern Judging co-evolution of strategies and empathy will tend towards a highly empathetic population of discriminators, especially when the benefits to cooperation are large; or, alternatively, evolution will lead to all defectors and empathy will thereafter drift neutrally.

The evolution of empathy is more complicated under Simple Standing. Assuming the population consists of discriminators there is a single evolutionarily stable and globally attractive singular point ESS (Figure 4c). The value of empathy at this singular point is larger when the benefits of cooperation are larger (Figure 5b). However once this value of empathy is reached, the strategic equilibrium at pure discriminators is no longer stable, and the population will instead be replaced by a mix of DISC and ALLC strategists, under replicator dynamics. This new strategic equilibrium will, in turn, lower the singular value of ESS under adaptive dynamics, which again changes the equilibrium balance of DISC and ALLC strategists. Long-term strategy-empathy co-evolution will continue in this fashion, with both ALLC and DISC present in the population, until the singular value of empathy reaches ESS=0 (see Figure 5—figure supplement 1). The strategic equilibrium at this point lies near the boundary of two basins of attraction (Figure 2d) and is vulnerable to invasion by pure defectors in a finite population. And so, in summary, while the exact dynamics will depend on the time scales of empathy and strategy evolution, Simple Standing cannot sustain empathy over the long term as both these components of personality co-evolve, eventually resulting in a population of pure defectors.

Discussion

Empathy has long been associated with prosocial behavior and altruism in humans. Much of the existing literature focuses on the emotional component of empathy – linkage of emotional states between individuals, emotional contagion (Hatfield et al., 1993) and empathy-induced helping (Cialdini et al., 1997; May, 2011). For instance, there is substantial evidence that the effect of ‘self-other merging’ provides moral motivation to cooperate (Batson et al., 1997; Batson et al., 1995; Batson and Moran, 1999) and contributes to the resolution of public-goods dilemmas (Batson, 1994). Empathy is not a unitary construct, however, and besides the purely emotional reaction to the states of other individuals there is the cognitive ability to understand another person’s psychological perspective (Davis, 1983; Smith, 2006).

Very little is known about the origins of empathy in relation to cooperative behavior, although some research suggests that the capacity for emotional empathy evolved in the context of parental care (de Waal, 2008). Even less is known about social evolution and selective forces operating on empathy in modern societies. Even if empathy promotes altruistic behavior, why should empathetic perspective-taking itself evolve and be stable against the invasion of morally egocentric individuals?

Here we have studied the role of empathetic perspective-taking in a game-theoretic context of moral evaluation, where individuals make moral judgments from their own subjective perspectives. Studying the impact of empathy in this context is critical to understanding cooperative behavior in modern, highly-connected societies that generally lack a centralized institution of objective moral assessment.

Social norms specify the rules of moral evaluation. It is well known that moral reputations can sustain high levels of cooperation if individuals discriminate between the ‘good’ and the ‘bad’. Social norms themselves likely emerge from individual beliefs of what reputations should be assigned to defectors and cooperators in distinct social situations. While some studies assume that social reputations are absolute – for instance due to shared information, public monitoring, institutions and gossip – our study draws attention to the potential for disagreements on reputations that arise from errors or different observation histories. The same individual can have different reputations in the eyes of distinct observers; in other words, moral evaluations are not absolute, and social reputation is relative. When monitoring of social interactions is private, cooperation is much harder to evolve and sustain – as reflected by the results of two recent studies by Hilbe et al. (2018) and Okada et al. (2017). Both of those studies analyzed models with private, but egocentric, moral evaluation corresponding to E=0 in our analysis. Our model of private evaluation makes the additional assumption that each observer evaluates a donor based on a different social interaction, along the lines of Okada et al. (2018).

Our key finding is that high levels of cooperation can be sustained, even with private monitoring of reputations, provided individuals recognize moral relativity and are capable of making moral judgments from another person’s perspective (E>0). Egocentric evaluation leads to unjustified or irrational defection, because a person perceived as ‘bad’ by the observer might actually appear ‘good’ in the eyes of the donor who’s action is being evaluated, or vice versa. This point is particularly striking in the case of Stern Judging, the norm that assigns a ‘good’ reputation only to individuals who cooperate with other ‘good’ players and defect against ‘bad’ (Kandori, 1992; Pacheco et al., 2006). Despite being the most efficient norm at promoting cooperation in empathetic societies, Stern Judging performs very poorly in egocentric populations. On the other hand, Scoring – the norm that does not take into account the recipient’s reputation at all – is immune to the effects of empathy and dominates in societies with egocentric moral evaluation rules.

Finally, we have shown that empathetic perspective-taking can evolve through cultural copying, and remain evolutionarily stable if a society is governed by Stern Judging or Shunning. Once these societies evolve empathy, individuals performing egocentric evaluations of observed social behavior will be rewarded less than their empathetic peers, and this remains true even if strategies are allowed to co-evolve with empathy. However, we have also seen that egocentric and uncooperative societies are nevertheless possible evolutionary outcomes. In populations governed by Stern Judging, Shunning and Scoring this outcome represents an alternative locally (though not globally) attractive stable state in the strategy-empathy phase space. In the case of Simple Standing, the egocentric and uncooperative outcome is the only long-term stable outcome as both empathy and strategies are allowed to evolve.

Our study raises a number of questions to be addressed in future work on empathy, norms, and the evolution of cooperation. Whereas we have studied empathy as a fixed trait, an individual’s tendency for empathetic evaluation might instead depend in a non-linear way on the current make-up of strategies in the population. Another question involves the competition of social norms for moral evaluation – a topic that has been studied in some contexts, such as when errors do not occur (Uchida et al., 2018), or in the presence of population structure (Masuda, 2012; Pacheco et al., 2006). Perhaps an even more fundamental question is whether and how population-wide social norms can evolve from individual moral beliefs to begin with. It is unclear whether social contagion or individual-level Darwinian selection is sufficient to establish a hierarchy of norms governing how individuals update each others’ reputations in a population. We have shown that the norms that promote the most cooperation change depending on the capacity for empathetic perspective-taking, but should we also expect different norms to evolve under empathetic and egocentric modes of judgment? For instance, populations characterized by fully empathetic moral judgment might be conducive to the evolution of selfish norms that indiscriminately assign ‘bad’ reputations to evade costly cooperation without being punished, while models with private egocentric evaluation may lead to the evolution of more cooperative norms, such as Scoring or Stern Judging (Yamamoto et al., 2017; Uchida et al., 2018). Such questions about the origin of and competition between moral norms remain outstanding.

Materials and methods

Cooperation under empathetic moral evaluation

Replicator dynamics

Request a detailed protocol

To analyze the evolutionary dynamics of strategies, we consider replicator dynamics in an infinite population with a fixed social norm and fixed value of the empathy parameter E, limiting ourselves to the three strategies: ALLC or X, ALLD or Y, and DISC or Z. Denoting the mean payoff of strategy s as Πs, and the frequencies of the three strategies at fs, the strategy dynamics can be described as dfsdt=fs(ΠssfsΠs). As in Okada et al. (2018), we assume that each observer makes her moral evaluation of a donor’s action based on a different social interaction, that is, each interaction is observed only once.

To describe image dynamics, we let g denote the frequency of ‘good’ individuals within the population, that is g=fXgX+fYgY+fZgZ. Furthermore we let g2 denote the probability that two randomly selected individuals will see the same recipient as ‘good’, that is g2=fXgX2+fYgY2+fZgZ2. The probability that two individuals will see the same recipient as ‘bad’ is then b2=1-2g+g2, and they will disagree on the subjective reputation of the recipient with the probability d2=g-g2.

For the Stern Judging norm, mean frequencies of ‘good’ individuals within the subpopulations of cooperators, defectors and discriminators are (averaged over the perspectives of all players):

(5) gX=gε+(1g)(1ε);gY=ge2+(1g)(1e2);gZ=E(gε+(1g)(1e2))+(1E)(g2ε+d2(e2+1ε)+b2(1e2)).

Here ε=(1-e1)(1-e2)+e1e2 is the probability that an individual with the intention to cooperate is assigned a ‘good’ reputation. The second term of gZ describes egocentric evaluation, in which the donor and the observer may disagree on the reputation of the recipient. If both see the recipient as ‘good’, the donor will cooperate under the DISC strategy, and she will be seen as ‘good’ in the eyes of the observer with the probability ε. If the donor sees the recipient as ‘good’, but the observer disagrees, the donor will intend to cooperate and will be seen as ‘good’ only with probability 1-ε, that is, if cooperation succeeds but an evaluation error occurs ((1-e1)e2) or if cooperation fails and the action is evaluated correctly (e1(1-e2))). In the opposite case, the donor will defect and will be seen as ‘good’ only if the evaluation error e2 happens. Finally, if the donor and the observer both see the recipient as ‘bad’, the donor will defect and will be seen as good only in the absence of observation error 1-e2.

Expected payoffs of the three strategies are then:

(6) ΠX=b(fX+fZgX)(1e1)c(1e1);ΠY=b(fX+fZgY)(1e1);ΠZ=b(fX+fZgZ)(1e1)cg(1e1);Π=fXΠX+fYΠY+fZΠZ.

Likewise for Shunning:

(7) gX=gε+(1g)e2;gY=e2;gZ=E(gε+(1g)e2)+(1E)(g2ε+2d2e2+b2e2).

For Simple Standing:

(8) gX=gε+(1g)(1e2);gY=ge2+(1g)(1e2);gZ=E(gε+(1g)(1e2))+(1E)(g2ε+d2+b2(1e2)).

And finally for Scoring norm, empathy E is irrelevant, because the norm does not take into account the reputation of the recipient:

(9) gX=ε;gY=e2;gZ=gε+(1g)e2.

Stochastic simulations

Request a detailed protocol

In addition to the deterministic replicator-dynamics analysis of strategy evolution, we performed a series of individual-based simulations to measure mean levels of cooperation under continuous influx of mutations in the strategy space (Santos et al., 2016a). We assume that all individuals follow the same social norm and are characterized by the same value of empathy, E. The population consists of N individuals, each with its own strategy and its own subjective list of reputations. Each generation, any given individual interacts with all other members of the society in three different roles: once as a donor, once as a recipient, and once as an observer.

First, each individual plays a single round of the donation game with all other members of the society according to her strategy S=(p,q) and the subjective reputation of the recipient, also taking into account the implementation error e1. Here p and q denote the probabilities that a donor cooperates with a ‘bad’ (B) and ‘good’ (G) recipient, respectively. The act of cooperation fails with the probability e1 (defection always succeeds). The cumulative payoff is then assigned to each individual, with the benefit of cooperation fixed at b and the cost of a cooperative act c.

To update their list of subjective reputations based on the social norm Nij, each player then chooses to observe a single interaction per donor (that is, with a randomly chosen recipient), again taking into account subjective reputation of the recipient either in the eyes of the donor (probability E) or the eyes of the observer (with a probability 1-E). The newly assigned reputation is reversed with the probability e2, representing observation errors. For the sake of simplicity, we assume that all reputations are updated simultaneously after all donor-recipient interactions have taken place.

We model selection and drift of strategies as a process of social contagion implemented as a pairwise comparison process. Following the reputation-updating step, a random pair of individuals is chosen; the first individual adopts the strategy of the second with the probability 1/(1+exp(-w[Π1-Π0])), where w is the selection strength, and Π1 and Π0 are payoffs of the two earned within the last generation. In our simulations of populations with N=100 individuals, we used w=1.0. Finally, each individual is subject to random strategy exploration, in which a new random strategy is adopted with a small probability μ (Santos et al., 2016a).

The simulation is initialized with random strategies and random lists of subjective reputations. We recorded the mean rate of cooperation averaged over 150,000 generations in 50 replicate populations, which is reported in Figure 3.

Evolution of empathy

Request a detailed protocol

Let gij be the frequency of ‘good’ individuals in the sub-population i as seen by individuals belonging to the sub-population j, where i and j correspond either to resident (i,j=0) or invader (i,j=1) sub-population. Working in the limit of negligible invader frequencies, and assuming that the population consists only of DISC strategists, for Stern Judging norm we have:

(10) g00=E0(g00ε+(1g00)(1e2))+(1E0)(g002ε+g00(1g00)(e2+1ε)+(1g00)2(1e2));g01=E1(g00ε+(1g00)(1e2))+(1E1)(g01g00ε+g01(1g00)e2+(1g01)g00(1ε)+(1g01)(1g00)(1e2));g10=E0(g01ε+(1g01)(1e2))+(1E0)(g01g00ε+g01(1g00)e2+(1g01)g00(1ε)+(1g01)(1g00)(1e2)).

Here ε=(1-e1)(1-e2)+e1e2, and E0 and E1 are empathy values of resident and invader sub-population. For Simple Standing norm, the relative frequencies of good individuals are:

(11) g00=E0(g00ε+(1g00)(1e2))+(1E0)(g002ε+g00(1g00)(1e2)+(1g00)g00e2+(1g00)2(1e2));g01=E1(g00ε+(1g00)(1e2))+(1E1)(g01g00ε+g01(1g00)e2+(1g01)g00(1e2)+(1g01)(1g00)(1e2));g10=E0(g01ε+(1g01)(1e2))+(1E0)(g01g00ε+g01(1g00)(1e2)+(1g01)g00e2+(1g01)(1g00)(1e2)).

Likewise, for the Shunning norm:

(12) g00=E0(g00ε+(1g00)(e2))+(1E0)(g002ε+g00(1g00)e2+(1g00)g00e2+(1g00)2e2);g01=E1(g00ε+(1g00)e2)+(1E1)(g01g00ε+g01(1g00)e2+(1g01)g00e2+(1g01)(1g00)e2);g10=E0(g01ε+(1g01)e2)+(1E0)(g01g00ε+g01(1g00)e2+(1g01)g00e2+(1g01)(1g00)e2).

Under Scoring, the frequencies of ‘good’ individuals do not depend on empathy:

(13) g=e21ε+e2.

We then calculate the expected payoffs of individuals in resident and invader sub-populations:

(14) {Π0=b(1-e1)g00-c(1-e1)g00;Π1=b(1-e1)g10-c(1-e1)g01.

These payoffs are used to generate pairwise invasibility plots in Figure 4. Singular points are found by setting (Π1-Π0)E1=0 and setting E0=E1.

Individual-based simulations of empathy evolution

Request a detailed protocol

To verify the ESS results of the adaptive-dynamics calculations we performed a series of Monte-Carlo simulations in finite populations of N=100 individuals. The simulation routine is largely the same as for strategy evolution, except that in this case we fixed the strategy at DISC and allowed E to evolve via constant influx of small mutations. Each generation, empathy of an individual changes via mutation at a rate μE=0.005. Since empathy is a continuous parameter, we draw the mutational deviation δE from a normal distribution centered around δE0=0 with a standard deviation σ=0.01. Selection for empathy is modeled by choosing five random pairs of individuals and assuming that in each pair the first individual copies the empathy value E1 of the second with the probability 1/(1+exp(-w[Π1-Π0])), where Π1 and Π0 are their payoffs.

References

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
    Evolution of cooperation through indirect reciprocity
    1. O Leimar
    2. P Hammerstein
    (2001)
    Proceedings of the Royal Society of London. Series B: Biological Sciences 268:745–753.
    https://doi.org/10.1098/rspb.2000.1573
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
    A theoretical approach to norm ecosystems: two adaptive architectures of indirect reciprocity show different paths to the evolution of cooperation
    1. S Uchida
    2. H Yamamoto
    3. I Okada
    4. T Sasaki
    (2018)
    Frontiers in Physics, 6, 10.3389/fphy.2018.00014.
  47. 47

Decision letter

  1. Michael Doebeli
    Reviewing Editor; University of British Columbia, Canada
  2. Diethard Tautz
    Senior Editor; Max-Planck Institute for Evolutionary Biology, Germany
  3. Naoki Masuda
    Reviewer; University of Bristol, United Kingdom
  4. Martin A Nowak
    Reviewer; Harvard University, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Evolution of empathetic moral evaluation" for consideration by eLife. Your article has been reviewed by three peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Diethard Tautz as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Naoki Masuda (Reviewer #2); Martin A Nowak (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

The paper analyses a mathematical model of reputation-based indirect reciprocity introducing the concept of empathy, which involves assessing the reputation of a recipient from the donor's perspective rather than from the assessors perspective. This is a very interesting and novel addition to the literature on evolutionary game theory in the context of cooperation and social norms. Before the paper can be published, however, we ask you to address a number of issues that the reviewers have pointed out.

Reviewer #1:

A few remarks regarding the adaptive dynamics of empathy:

I think the authors should point out that there is a certain degree of degeneracy in their model, which is due to the assumption that payoffs are linear in the empathy E (a common assumption in game theory, of course). This degeneracy is for example reflected in the vertical lines in the pairwise invasibility plots in Figure 4. It implies that there can for example be no evolutionary branching points in their model.

Generally, speaking, it is easy to imagine that empathy enters in a non-linear way into the model (e.g. when the empathy that an individual displays depends on population level quantities, such as the overall level of cooperation, or the average empathy, or when the empathy of individual is itself a (non-linear) function of some other underlying personal characteristic, etc.). In such cases, it is possible for diversity in empathy to evolve, which would be an interesting and perhaps realistic feature of the model.

Note for example that even in the present model, different levels of empathy can coexist "ecologically". For example, it is evident from Figure 4A that empathy levels 0.6 and 0.9 can invade each other, and hence would coexist. (However, if such coexisting strains were allowed to evolve, the would both evolve to the ESS value, and hence diversity would be lost over evolutionary time.)

Overall, I think it would be good if the paper would discuss these issues. Also, please note that in the second paragraph of the section "Evolution of empathy", the gradient derivative of the function W has to be evaluated at EI = ER to become the selection gradient.

Reviewer #2:

The authors analysed a mathematical model of reputation-based indirect reciprocity analytically (using replicator dynamics and adaptive dynamics) and numerically. Each player is assumed to be either unconditional cooperator, discriminator, or unconditional defector, and also assigns a binary reputation to other players which is updated in the course of game dynamics. This is a common setting (including the analysis methods) since early 2000s. A new component in this paper is the effect of empathy. The authors' model assumes that different players may have different opinions towards a player. This has been done in some prior modelling, but remarkably, the present authors assumed that an observer uses the donor's view (as opposed to the observer's own view) towards the recipient's reputation with probability E, when judging if the donor's action towards the recipient is good or bad. In this way the authors implemented the effect of empathy. This is realistic and presents an implementation of the theory of mind, thus stretching the implication of the present work to much wider domains such as social psychology and neuroscience, which many of similar studies lack. The authors discovered that empathy generally promotes cooperation. The effect of the empathy depended on the social norm (i.e. reputation assignment rule) employed by the population. Furthermore, the authors studied evolution of the empathy (i.e. value of E) both analytically and numerically.

The work presents a solid framework for analysing empathy, which is really a salient issue, in the setting of indirect reciprocity and has led to reasonable results. Furthermore, I feel that the work is very complete. Therefore, I recommend publication of this work.

Reviewer #3:

The paper explores the evolution of cooperation in a model of indirect reciprocity with empathy. It assumes that players are repeatedly matched to interact in a social dilemma game. A player's decision whether to cooperate may depend on the recipient's reputation. There are three possible strategies: ALLC players always cooperate, irrespective of the reputation of the recipient. ALLD players always defect. Discriminators cooperate if the co-player has a good reputation and they defect if the co-player has a bad reputation.

To assign reputations to players, the paper assumes that the entire population applies the same social norm. Four different social norms are considered, Stern Judging, Simple Standing, Scoring, and Shunning. All of these norms assign a good reputation to a donor who cooperates with a good recipient, and they assign a bad reputation to a donor who defects against a good recipient. The norms differ in how they assess interactions with a bad recipient.

As their main contribution, the authors study the effect of empathy. In traditional models of indirect reciprocity, observers judge the donor's action based on their own perspective of the recipient. In contrast, an empathetic observer judges the donor's action based on the donor's perspective of the recipient. Using a mixture of analytical methods and simulations, the main results are as follows:

1) Empathetic judgment facilitates the evolution of cooperation

2) Empathy itself can evolve for three of the four social norms studied.

I think the questions explored in this paper are extremely interesting for at least two reasons. First, a few recent studies have found that indirect reciprocity has problems to evolve when information is private. This paper proposes an elegant solution to this problem.

Second, empathy itself is an important characteristic of human behavior that requires an evolutionary explanation. The present paper can serve as an important first step.

The paper is well motivated, the model is sound, and the figures provide a great illustration of the key results.

However, a few questions remain because the Materials and methods section is somewhat unclear. In particular, I was unable to verify whether the payoff formulas are correct, and whether the qualitative findings are robust with respect to minor changes in the model assumptions. If these remaining questions can be resolved, I strongly support publication.

1) It is unclear how the payoffs for the replicator dynamics model have been derived. In fact, the given formulas might be wrong.

So far it has been difficult to analyze the effects of private information analytically. Most previous models of indirect reciprocity have relied on simulations (such as the two cited papers by Okada et al. and Hilbe et al., [Okada, Sasaki and Nakai, 2017; Hilbe et al., 2018]). Analytical results have only been feasible in rather specific special cases, for example, Okada et al., 2018 "A solution for private assessment in indirect reciprocity using solitary observation".

The difficulties arise because models of private information need to calculate the probability that two randomly chosen discriminators both assign a good reputation to the recipient. This probability is difficult to compute because the opinions of two discriminators will neither be perfectly correlated nor are they completely independent.

Given these previous difficulties, I was surprised to see that the present paper can give simple expressions for the players' payoffs (Equations 5-9 in the Materials and methods section), which also hold for E=0. Unfortunately Equations 5-9 are not derived in sufficient detail, which makes it impossible to check whether they are correct.

In these formulas, g is the frequency of good individuals within the population, whereas the authors seem to use g2 for the probability that two individuals both assign a good reputation to a random recipient. This seems to presume complete independence of individual opinions.

I would like to ask the authors to provide a more detailed derivation of the payoff Equations 5-9. In particular, the authors should make clear how they circumvented the problems described in the above mentioned paper of Okada et al., 2018.

The same comment also applies to the formulas in the Materials and methods section on the evolution of empathy.

2) The paper is based on the assumption that all players in the population employ the same social norm. As the authors note in the last paragraph of their discussion, it is not clear whether cooperation remains stable if mutants were allowed to switch to a self-serving social norm (by assigning a bad reputation to everyone, and then defect against everyone). It appears that such a mutant is a major threat to all social norms studied herein, especially when the empathy probability is high.

I completely agree with the authors that a full analysis of such mutations is beyond the scope of the present paper. However, I would appreciate if the authors could give some basic argument why their results can be expected to be robust.

As one possibility, the authors could repeat the analysis in Figure 2 or Figure 3. But instead of assuming that ALLD and ALLC players apply the same social norm as the discriminators, they could assume that ALLD players always assign a bad reputation to everyone, whereas ALLC players always assign a good reputation to everyone. This would correspond to a case in which the social norms applied by the unconditional types are consistent with their actions.

If such an analysis turns out to be infeasible, I suggest the authors explain under which circumstances we can reasonably assume all members of a population to use the same social norm.

3) The results in Figure 3C are either wrong, or they require more explanation.

In this figure, the authors show that for a rather small mutation rate (μ=0.01), almost all social norms lead to similar cooperation rates and that cooperation rates are largely independent of the empathy parameter. There are two reasons why Figure 3C is somewhat odd:

i) It suggests that empathy can only help for extremely small mutation rates.

ii) It suggests that in the case of no empathy E=0, Stern Judging can yield roughly 40% cooperation. This is somewhat surprising, given that previous papers have found that under private information, a Stern Judging norm typically leads to an ALLD population with almost no cooperation at all (e.g. Okada, Sasaki and Nakai, 2017; Hilbe et al., 2018).

On a similar note, I would appreciate if the authors could be more clear in the main text about the relevant timescales. From reading the Materials and methods section, it seems to me that the authors assume that in the infinite population model, reputations are assumed to reach a steady state before evolutionary updating occurs. In the finite population simulations, evolutionary updating may occur before the players' reputations have reached a steady state, is this correct?

https://doi.org/10.7554/eLife.44269.014

Author response

Reviewer #1:

A few remarks regarding the adaptive dynamics of empathy:

I think the authors should point out that there is a certain degree of degeneracy in their model, which is due to the assumption that payoffs are linear in the empathy E (a common assumption in game theory, of course). This degeneracy is for example reflected in the vertical lines in the pairwise invasibility plots in Figure 4. It implies that there can for example be no evolutionary branching points in their model.

Generally, speaking, it is easy to imagine that empathy enters in a non-linear way into the model (e.g. when the empathy that an individual displays depends on population level quantities, such as the overall level of cooperation, or the average empathy, or when the empathy of individual is itself a (non-linear) function of some other underlying personal characteristic, etc.). In such cases, it is possible for diversity in empathy to evolve, which would be an interesting and perhaps realistic feature of the model.

Note for example that even in the present model, different levels of empathy can coexist "ecologically". For example, it is evident from Figure 4A that empathy levels 0.6 and 0.9 can invade each other, and hence would coexist. (However, if such coexisting strains were allowed to evolve, the would both evolve to the ESS value, and hence diversity would be lost over evolutionary time.)

Thank you for pointing this out. In this initial study, we model empathy in its simplest form as the probability to perform moral evaluations from the perspective of another person, which itself is a fixed trait of the focal individual. We agree that, in general, the propensity to make empathetic evaluations could depend on the state of the population in a non-linear way, leading to qualitatively different results including evolutionary branching. Although we leave this analysis for the future research, we have added some discussion about this in the final paragraph of the manuscript.

Overall, I think it would be good if the paper would discuss these issues. Also, please note that in the second paragraph of the section "Evolution of empathy", the gradient derivative of the function W has to be evaluated at EI = ER to become the selection gradient.

Thank you, we have appended this to the expression of selection gradient. In addition, we discuss the possibility for non-linearity in empathy in our Discussion section.

Reviewer #3:

[…] However, a few questions remain because the Materials and methods section is somewhat unclear. In particular, I was unable to verify whether the payoff formulas are correct, and whether the qualitative findings are robust with respect to minor changes in the model assumptions. If these remaining questions can be resolved, I strongly support publication.

1) It is unclear how the payoffs for the replicator dynamics model have been derived. In fact, the given formulas might be wrong.

So far it has been difficult to analyze the effects of private information analytically. Most previous models of indirect reciprocity have relied on simulations (such as the two cited papers by Okada et al. and Hilbe et al., [Okada, Sasaki and Nakai, 2017; Hilbe et al., 2018]). Analytical results have only been feasible in rather specific special cases, for example, Okada et al., 2018 "A solution for private assessment in indirect reciprocity using solitary observation".

The difficulties arise because models of private information need to calculate the probability that two randomly chosen discriminators both assign a good reputation to the recipient. This probability is difficult to compute because the opinions of two discriminators will neither be perfectly correlated nor are they completely independent.

Given these previous difficulties, I was surprised to see that the present paper can give simple expressions for the players' payoffs (Equations 5-9 in the Materials and methods section), which also hold for E=0. Unfortunately Equations 5-9 are not derived in sufficient detail, which makes it impossible to check whether they are correct.

In these formulas, g is the frequency of good individuals within the population, whereas the authors seem to use g2 for the probability that two individuals both assign a good reputation to a random recipient. This seems to presume complete independence of individual opinions.

I would like to ask the authors to provide a more detailed derivation of the payoff Equations 5-9. In particular, the authors should make clear how they circumvented the problems described in the above mentioned paper of Okada et al., 2018.

The same comment also applies to the formulas in the Materials and methods section on the evolution of empathy.

The reviewer is correct: our initial expressions involving g2 for the probability that two individuals see a focal player as “good” assumed complete independence of evaluations. This assumption was wrong, and we have corrected it in the revision (including changes in the Results and Materials and methods sections, Figure 2, Figure 2—figure supplement 1, and Figure 5—figure supplement 1). In fact, our model assumes that every individual interacts with all recipients, and then an observer makes her moral evaluation of a donor on the basis a single, randomly chosen interaction. In an infinite population each interaction is therefore observed only once, as in “solitary observation” approach of Okada et al., 2018. We have re-written Equations 5-9 in terms of the probability that two individuals assess the same donor as good (g2 = fXgX2 +fY gY2 +fZgZ2), bad (b2 = fX(1−gX)2 +fY (1−gY)2 +fZ(1−gZ)2), or disagree about his reputation d2 = 1/2(1−g2b2). We have updated equations (1) and (2) accordingly. (This issue does not apply to the Equations 10-12, as those assume that the population consists solely of discriminators.)

Our qualitative and even quantitative results remain largely unchanged by these corrections. Nonetheless we are very grateful to the reviewer for noticing the erroneous independence approximation, because correcting this error has clarified our model and its connection to the recent literature.

2) The paper is based on the assumption that all players in the population employ the same social norm. As the authors note in the last paragraph of their discussion, it is not clear whether cooperation remains stable if mutants were allowed to switch to a self-serving social norm (by assigning a bad reputation to everyone, and then defect against everyone). It appears that such a mutant is a major threat to all social norms studied herein, especially when the empathy probability is high.

I completely agree with the authors that a full analysis of such mutations is beyond the scope of the present paper. However, I would appreciate if the authors could give some basic argument why their results can be expected to be robust.

As one possibility, the authors could repeat the analysis in Figure 2 or Figure 3. But instead of assuming that ALLD and ALLC players apply the same social norm as the discriminators, they could assume that ALLD players always assign a bad reputation to everyone, whereas ALLC players always assign a good reputation to everyone. This would correspond to a case in which the social norms applied by the unconditional types are consistent with their actions.

If such an analysis turns out to be infeasible, I suggest the authors explain under which circumstances we can reasonably assume all members of a population to use the same social norm.

The full analysis of strategy-norm co-evolution under high empathy is part of our future work in this area. However, here we would like to point out that a selfish norm that assigns bad reputations to everyone together with ALLD strategy might not necessarily invade, since other norms can still punish defection against ‘bad’ while retaining rewards for cooperation with the ‘good’. In other words, while in the fully empathetic case the observer takes into account the recipient’s reputation from the donor’s perspective, the norm used to make the moral evaluation still belongs to the observer and not the donor. We discuss this point briefly in our conclusion section, but otherwise restrict our analysis to a shared norm (which is the standard assumption in the literature, presumably on the grounds that norm deviation is often punished, in practice).

3) The results in Figure 3C are either wrong, or they require more explanation.

In this figure, the authors show that for a rather small mutation rate (μ=0.01), almost all social norms lead to similar cooperation rates and that cooperation rates are largely independent of the empathy parameter. There are two reasons why Figure 3C is somewhat odd:

i) It suggests that empathy can only help for extremely small mutation rates.

ii) It suggests that in the case of no empathy E=0, Stern Judging can yield roughly 40% cooperation. This is somewhat surprising, given that previous papers have found that under private information, a Stern Judging norm typically leads to an ALLD population with almost no cooperation at all (e.g. Okada, Sasaki and Nakai, 2017; Hilbe et al., 2018).

In Figure 3C the mutation rate is µ = 0.01 with population size N = 100, so that θ = 1. This is the highest mutation rate that we consider, and it corresponds to the case where stochastic effects dominate, with the population moving freely between the cooperative and non-cooperative basins of attraction. All norms therefore lead to the nearly identical mean rates of cooperation. (The same result in seen in Figure 2 from Santos et al., 2016.) Indeed, empathy has no effect when stochastic exploration dominates the strategy dynamics, and this is an intuitive result. For Stern Judging our findings agree well with the previously published results (with E = 0) under lower mutation rates. By comparison, the results described in Hilbe et al., 2018, were derived in the limit of µ → 0, which is a qualitative different regime compared to θ = 1. We are not surprised to find different results under θ = 1 compared to µ → 0, and indeed our results for θ = 1 coincide with previous studies in this regime of mutation.

On a similar note, I would appreciate if the authors could be more clear in the main text about the relevant timescales. From reading the Materials and methods section, it seems to me that the authors assume that in the infinite population model, reputations are assumed to reach a steady state before evolutionary updating occurs. In the finite population simulations, evolutionary updating may occur before the players' reputations have reached a steady state, is this correct?

Thank you for this suggestion. In the infinite-population analysis we indeed assume that reputation frequencies reach their steady state before the payoffs are updated and before selection takes place. In our Monte Carlo simulations each individual plays N games before reputations are updated; the evolutionary updating based on accumulated payoffs follows the moral-evaluation step. It is therefore possible that reputations do not reach steady state before the evolutionary updating, although for weak selection and close to the strategy equilibrium the analysis of the infinite-population model provides a very good approximation of the behavior observed in the finite-population simulations. We have clarified this issue of timescales in the main text (first paragraph of Results section).

https://doi.org/10.7554/eLife.44269.015

Article and author information

Author details

  1. Arunas L Radzvilavicius

    Department of Biology, University of Pennsylvania, Philadelphia, United States
    Contribution
    Conceptualization, Formal analysis, Writing—original draft
    For correspondence
    arunas@sas.upenn.edu
    Competing interests
    No competing interests declared
  2. Alexander J Stewart

    Department of Biology, University of Houston, Houston, United States
    Contribution
    Conceptualization, Writing—review and editing
    Competing interests
    No competing interests declared
  3. Joshua B Plotkin

    Department of Biology, University of Pennsylvania, Philadelphia, United States
    Contribution
    Conceptualization, Methodology, Writing—review and editing
    For correspondence
    jplotkin@sas.upenn.edu
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2349-6304

Funding

David and Lucile Packard Foundation

  • Joshua B Plotkin

Army Research Office (W911NF-12-R-0012-04)

  • Joshua B Plotkin

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Senior Editor

  1. Diethard Tautz, Max-Planck Institute for Evolutionary Biology, Germany

Reviewing Editor

  1. Michael Doebeli, University of British Columbia, Canada

Reviewers

  1. Naoki Masuda, University of Bristol, United Kingdom
  2. Martin A Nowak, Harvard University, United States

Publication history

  1. Received: December 10, 2018
  2. Accepted: March 19, 2019
  3. Accepted Manuscript published: April 9, 2019 (version 1)
  4. Version of Record published: April 29, 2019 (version 2)

Copyright

© 2019, Radzvilavicius et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,122
    Page views
  • 314
    Downloads
  • 1
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

  1. Further reading

Further reading

    1. Evolutionary Biology
    Matthew Alan Benton et al.
    Research Article Updated
    1. Evolutionary Biology
    2. Genetics and Genomics
    Sarah A Signor
    Insight