Expert Political Judgment by Philip E. Tetlock

Author:Philip E. Tetlock
Language: eng
Format: epub
Publisher: Princeton University Press
Published: 2013-09-01T16:00:00+00:00

QUANTITATIVE ANALYSIS OF BELIEF SYSTEM DEFENSES

We now shift from listening to arguments to counting them. Our goal is to document how often various groups of forecasters advanced various categories of arguments in various contexts. The closer we look, the stronger the grounds become for characterizing the arguments in aggregate as belief system defenses that experts deployed in largely self-serving ways to justify refusing to change their minds when they lost reputational bets that they themselves once endorsed.

The case for a self-serving bias in argumentation rests on several lines of evidence:

1. The suspicious selectivity with which forecasters advanced arguments that trivialized earlier reputational bets. Experts were far more likely to endorse such arguments when something unexpected occurred. In thirty-nine of sixty comparisons, t-tests revealed that experts who had just lost reputational bets (their most likely future failed to materialize) endorsed arguments from the list of seven more enthusiastically (p < .05) than experts who had just won such bets. By contrast, experts who had won their bets never showed more enthusiasm for “defensive” cognitions than experts who had just lost them.

2. The linkage between the size of mistakes and the activation of defenses. The psychologic is straightforward: the more confident experts were in the original forecast, the more threatening the disconfirmation and the more motivated experts will be to neutralize the troublesome evidence. All else equal, an expert who in 1988 was 90 percent confident that Soviet hardliners would reassert control between 1988 and 1993 should be more unsettled by intervening events than an expert who attached only slightly more than “guessing” confidence to the same forecast. To test this prediction, we created a composite defensiveness index by summing the six measured belief system defenses. The predicted pattern emerged. Among less accurate forecasters, the correlations between ex ante confidence and the composite defensiveness index are always positive, ranging from 0.26 to 0.42 across domains; among more accurate forecasters, the same correlations hover near zero, between – .05 and + 0.08.

3. The linkage between reliance on defenses and retention of confidence in prior opinions. If belief system defenses cushion the blow of unexpected events, then experts whose most likely scenarios do not materialize but who endorse “defensive” cognitions should retain more confidence in their original forecasts after they learn what happened (ex post confidence). But there should be no such correlation among experts whose forecasts were borne out and who should therefore not have experienced any threat to the core tenets of their belief systems. As predicted, among inaccurate forecasters, the defensiveness index is correlated with ex post confidence across domains (correlations ranging from .29 to .59). By contrast, among accurate forecasters, there is almost no relationship between defensiveness and ex post confidence (correlations again hovering near zero, between – .01 and .06).

4. The linkages among reliance on defenses, failures of belief updating, and cognitive style. We find that (a) the more “losers” resisted revising their prior opinions, the more defenses they embraced (r = .49); (b) hedgehog losers (who resisted changing

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.