^{1}

^{*}

^{1}

^{2}

^{1}

^{2}

Edited by: O'Dhaniel A. Mullette-Gillman, National University of Singapore, Singapore

Reviewed by: Philip R. Corlett, Yale University, United States; Tali Sharot, University College London, United Kingdom

*Correspondence: Bojana Kuzmanovic

This article was submitted to Decision Neuroscience, a section of the journal Frontiers in Psychology

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

People tend to update beliefs about their future outcomes in a valence-dependent way: they are likely to incorporate good news and to neglect bad news. However, belief formation is a complex process which depends not only on motivational factors such as the desire for favorable conclusions, but also on multiple cognitive variables such as prior beliefs, knowledge about personal vulnerabilities and resources, and the size of the probabilities and estimation errors. Thus, we applied computational modeling in order to test for valence-induced biases in updating while formally controlling for relevant cognitive factors. We compared biased and unbiased Bayesian models of belief updating, and specified alternative models based on reinforcement learning. The experiment consisted of 80 trials with 80 different adverse future life events. In each trial, participants estimated the base rate of one of these events and estimated their own risk of experiencing the event before and after being confronted with the actual base rate. Belief updates corresponded to the difference between the two self-risk estimates. Valence-dependent updating was assessed by comparing trials with good news (better-than-expected base rates) with trials with bad news (worse-than-expected base rates). After receiving bad relative to good news, participants' updates were smaller and deviated more strongly from rational Bayesian predictions, indicating a valence-induced bias. Model comparison revealed that the biased (i.e., optimistic) Bayesian model of belief updating better accounted for data than the unbiased (i.e., rational) Bayesian model, confirming that the valence of the new information influenced the amount of updating. Moreover, alternative computational modeling based on reinforcement learning demonstrated higher learning rates for good than for bad news, as well as a moderating role of personal knowledge. Finally, in this specific experimental context, the approach based on reinforcement learning was superior to the Bayesian approach. The computational validation of valence-dependent belief updating represents a novel support for a genuine optimism bias in human belief formation. Moreover, the precise control of relevant cognitive variables justifies the conclusion that the motivation to adopt the most favorable self-referential conclusions biases human judgments.

A growing body of research has demonstrated that people update their beliefs about future outcomes in an asymmetric manner: they tend to neglect undesirable information, but take desirable information more readily into account (Eil and Rao, ^{1}

As proposed by the differential scrutiny account, people tend to accept easily information with favorable implications (“

In the context of such complex processing, modeling acts as a computational microscope and provides an efficient method for isolating influential factors with maximal precision. Formalizing competing models with varying components of belief updating allows for the exact specification of hypotheses about possible mechanistic causes of the observed behavior, and for the identification of those components that substantially influence update dynamics. Moreover, the possibility of controlling for relevant cognitive variables on a trial-by-trial basis increases the confidence in conclusions about motivational explanations for asymmetric updating.

Hence, in order to provide a formal proof that belief updates are biased by the valence of new information, the present study combines an established belief update paradigm with computational modeling. We assessed beliefs about average and personal risks of negative future life events (e.g., cancer or car theft) and their updates in response to good news (e.g., a lower base rate of cancer than expected), or bad news (e.g., a higher base rate of cancer than expected). First, we applied previous formalizations of

The Exploratory Software for Confidence Intervals (Cumming,

The experiment was conducted during an acquisition of fMRI scans (neuroimaging data not reported here) using Presentation 18.1 (Neurobehavioral Systems), and consisted of 80 trials with 80 different adverse life events (e.g., cancer or car theft; for a complete list see

The task relied on the belief update methodology used in previous studies (see Figure

The general structure of the experimental design

The difference between E1 and E2 corresponds to the size of the update. The difference between eBR and BR indicates the size of the estimation error (EE), and whether BR was favorable (better than expected), or unfavorable (worse than expected). Moreover, the difference between eBR and E1 allows to infer how similar a participant perceives herself relative to the average person (e.g., E1 < eBR indicates that a person believes that she is less at risk than the average person, and vice versa). This may be important as base rates may become irrelevant if one assumes oneself to be very different from the average person (e.g., somebody without a car will not be concerned about the average risk of car theft, see Shah et al.,

All events (eBR, E1, BR, E2) were incorporated into one trial, and the full history of outcomes was displayed at all times, in order to avoid confounds by memory load and errors. Were these estimates assessed in separate sessions, participants would not be able to remember their exact previous estimates and the presented base rates for all 80 stimulus events. Such memory errors have been reported to be equal for trials with good and bad news (Sharot et al.,

Unbeknownst to participants, we manipulated BR in order to be able to control the number of trials and the size of EE across conditions (see Table

Task and model parameters, their statistics and sources.

Number of trials | 39.37 (2.24) | 37.96 (1.85) | 0.015 | |

Estimated base rate (eBR) | 47.90 (11.57) | 44.81 (10.82) | 0.002 | |

First estimate (E1) | 41.77 (12.38) | 37.85 (10.83) | 0.004 | |

Personal knowledge (P) | 6.12 (6.13) | 6.96 (7.81) | 0.282 | P = eBR - E1 |

Presented base rate (BR) | 34.44 (11.24) | 58.69 (10.85) | 0.000 | ^{†} |

Estimation error (EE) | 13.45 (0.91) | 13.86 (0.65) | 0.006 | EE = |eBR - BR| |

Second estimate (E2) | 34.24 (11.55) | 44.30 (12.21) | 0.000 | |

RT E2 in sec | 3.19 (1.04) | 3.17 (1.06) | 0.960 | |

Actual update (UPD) | 7.53 (2.66) | 6.45 (2.49) | 0.024 | UPD_{GOOD} = E1 - E2, |

UPD_{BAD} = E2 - E1 |
||||

Likelihood ratio (LHR) | 1.56 (1.19) | 1.54 (2.32) | 0.964 | |

% of trials with LHR < 1 | 55.92 (19.29) | 56.00 (19.98) | 0.976 | |

Bayesian E2 (E2b) | 30.74 (10.86) | 49.82 (12.15) | 0.000 | |

Bayesian UPD (UPDb) | 11.04 (1.95) | 11.97 (1.90) | 0.003 | UPDb_{GOOD} = E1 - E2b, |

UPDb_{BAD} = Eb2 - E1 |
||||

‘Optimistic’ UPDb (UPDbo) | 7.60 (2.47) | 6.48 (2.38) | 0.006 | UPDbo_{GOOD} = UPDb * (S + A), |

UPDbo_{BAD} = UPDb * (S – A) |

Participants were free to report a probability anywhere between 1 and 99%. Starting from 50% in eBR, they selected the desired probability within this range by using two buttons to increase or decrease the number displayed on the screen (Figure

Statistics for all task-related parameters are listed in Table

When comparing repeated measures throughout the study, we used paired

We implemented competing computational cognitive models, and assessed which model provided the best account for participants' actual data. More precisely, each model was first fitted to the data using Bayesian variational inference. This procedure yields for each hypothesis (a) a posterior distribution across the parameters, and (b) an approximation to the evidence of the model. The posterior provides sufficient statistics (i.e., mean and variance) of the parameter estimates. The model evidence reflects the goodness of fit of the model, which is penalized for the complexity of the parametrization. Here, we use the Free-energy approximation that has been shown to be superior to other approximations like AIC or BIC (Penny,

We compared models formalizing “rational” (according to Shah et al.,

Furthermore, we formalized an alternative, more simple computational model of belief updating. It relied on the generic form of reinforcement learning (Sutton and Bart,

Learning rate (LR, general tendency of each participant to update her beliefs in response to EE) smaller than 1 indicates updates smaller than EE, and LR greater than 1 indicates updates greater than EE (while EE is also weighted by a function of rP, see below). Furthermore, the learning rate was formalized as a function of the valence of news. That is, it was expected to differ systematically for GOOD and BAD (i.e., lower learning rates and smaller updates for BAD than for GOOD). If Asymmetry is different from zero, then there is an effect of Valence, and the influence of EE is different for GOOD and BAD.

RP stands for

In order to test the respective effects of learning rate, valence, and personal knowledge, we generated all possible variations of the update equation by switching on (by letting the parameter free), or off (by fixing the parameter's prior variance to zero) the parameters Alpha, Asymmetry, and Weight. Note that by setting Asymmetry and Weight to 0 we obtain the classical reinforcement learning rule (UPD = Alpha ^{*} EE), and prior expectations of 0 for Asymmetry and Weight and 1 for Alpha specify the null hypothesis UPD = EE. In the alternative hypothesis, those parameters have an influence on update, and thus need to be estimated for each participant (i.e., included into the model as free parameters).

Eight models with all possible parameter combinations (α+A+W, α+A, α+W, α, A+W, A, W, Ø; α, A and W indicate that the respective parameter was estimated instead of being fixed) were estimated for each subject and were then entered in a random effect Bayesian model comparison procedure. Finally, we compared the winning alternative model to the winning Bayesian model.

With respect to actual behavior, bad news led to smaller updates than good news indicating an optimism bias, _{(26)} = 2.42, _{(26)} = −3.28, _{GOOD}: _{(26)} = 8.48, _{BAD}: _{(26)} = 11.36, _{(26)} = 3.91,

Comparison of actual and Bayesian updates, and of “rational” and “optimistic” Bayesian models of belief updating. ^{*}^{**}^{***}

Linear regression analyses revealed that updates were larger in BAD than in GOOD trials even after controlling for trial-wise EE, _{(26)} = −2.73, _{βvalence} = −0.18, _{(26)} = −2.49, _{βvalence} = −0.18,

Bayesian model comparison of all four Bayesian models revealed that the “optimistic” Bayesian model that included both Scaling and Asymmetry as free parameters best accounted for the actual participants' behavior (_{(26)} = −13.05, _{(26)} = 4.12,

Bayesian model comparison of all eight alternative models revealed that the alternative model that included the effects of learing rate, valence and personal knowledge (_{(26)} = −8.33, _{(26)} = 2.99, _{(26)} = 18.11,

Alternative model of belief updating based on classical reinforcement learning. ^{**}^{**}

However, because W was relatively close to 1, and because α and W may not be completely orthogonal, we tested whether ^{*}

Finally, Bayesian model comparison revealed that

Fitting _{(26)} = −7.85, _{(26)} = 3.00,

The present study provides converging evidence for valence-dependent belief updating. Participants updated their beliefs about hazards more in response to desirable new information than in response to undesirable information. The good news-bad news effect was significant even after controlling for prior beliefs and their violations (by including estimation errors and initial estimates of risks and base rates as covariates in a regression analysis). Moreover, participants updates were compared to simulated belief updates expected to be made by a “dispassionate thinker—one who is not swayed by desires for any particular outcome” (Krizan and Windschitl,

Our findings differ from what has been reported by Shah et al. (^{2}

The conclusion that the differential evaluation of available evidence mediates the influence of desires on predictions is in line with recent neuroscientific research (Sharot and Garrett,

The valence-induced asymmetry in updating was additionally validated by the refinement of existing computational models of belief updating under formal control of trial-wise task parameters (i.e., own risk estimate, estimated base rate, actual base rate and the resulting estimation error). First, we demonstrated that an optimistically biased Bayesian model better accounted for the data than the fully rational Bayesian model (according to Shah et al.,

Moreover, we demonstrated that belief updates can be formalized in a simpler and superior way than suggested by Shah et al. (

Furthermore, model selection revealed that personal knowledge indeed played a significant role during the updating of beliefs about future outcomes. This means that the more participants perceived themselves to be different from average, the less they took the information about the average risk into account. Notably, the winning formalization (“1 – rP” instead of “1 – rP ^{*} W”) implicates that extreme values of personal knowledge will lead either to full consideration of estimation error (when one is equally at risk as the average person; however, there is still an influence of learning rate and valence), or to no consideration at all and thus no updating (when one is maximally different from the average person, independent of learning rate and valence). While this appears trivial at first glance, it does not necessarily represent a reasonable updating in the context of the present task. Even if there is an a priori perceived difference between the estimated base rate (e.g., 20%) and the own risk (e.g., 10%), learning about the actual base rate (e.g., 25%) may be expected to shift the estimate of the own risk by the size of the estimation error (from 10 to 15% due to the difference between the estimated and the actual base rate of 5%). Personal knowledge has long been considered to play an important role in the research area of unrealistic optimism (Shepperd et al.,

The third aspect of belief updating—the updates lower than predicted by the error—may relate to varying degrees of precision, or uncertainty, of beliefs and new information. For instance, generally reduced updating could result from low trust of presented base rates relative to high certainty of initial beliefs about base rates and own risks. Indeed, while none of the included participants doubted the general creditability of the presented (manipulated) base rates, 60% reported that some of them appeared odd. Thus, assessment of participants' certainty about their probability estimates, and their trust in the presented base rates could help to optimize further computational models. Moreover, using expressions of certainty as dependent variables may improve the understanding of motivational influences on expectations (Krizan and Windschitl,

Related to this, the superiority of the alternative model does not suggest that reinforcement learning is a better mechanistic explanation for belief updating than the Bayesian theorem. The two frameworks share central assumptions and differ mainly with respect to the consideration of precision (included in Bayesian models, but not in reinforcement learning ones). Therefore, the model based on reinforcement learning might be justified merely by limited data obtained with the current design, which does not include measures of subjective belief precision. Another reason that might explain the failure of the Bayesian model (beyond the valence effect) relates to the way belief precision is formally implemented. Building on the work of Shah et al. (^{3}

While the psychological and neurobiological evidence for differential processing of desirable and undesirable new information and resulting belief updates (Yacubian et al.,

BK developed the study concept and the study design, and performed the testing and data collection. BK and LR performed the data analysis, the interpretation and wrote the manuscript. Both authors agree to be accountable for the content of the work.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

We thank Marc Tittgemeyer, Anneli Jefferson, and Alexandra DiFeliceantonio for valuable comments on an earlier draft of the manuscript, and Thorben Huelsduenker for assistance with data collection.

The Supplementary Material for this article can be found online at:

^{1}In addition to the valence-dependent bias, i.e., the tendency to draw the conclusion that a choice option is rewarding, Palminteri and colleagues also demonstrated a confirmatory bias, i.e., the tendency to draw a conclusion that one made the correct choice. Thus, confirming a preexisting belief seems to be similarly desirable as obtaining an external reward.

^{2}Difference measure (predicted belief change—observed belief change, values closer to zero represent more normative belief updating) was valence-dependent only in two of four experiments: experiment 2 [good news: _{(16)} = 4.37, _{(111)} = 1.95,

^{3}