Thinking in Bets by Annie Duke

Author:Annie Duke
Language: eng
Format: epub
Publisher: Penguin Publishing Group
Published: 2018-02-06T05:00:00+00:00

Wanna bet (on science)?

If thinking in bets helps us de-bias, couldn’t we apply it to help solve the Heterodox Academy problem? One might guess that scientists would be more accurate if they had to bet on the likelihood that results would replicate as compared to traditional peer review, which can be vulnerable to viewpoint bias. Especially in an anonymous betting market, confirming the strength of your pre-existing ideology or betting solely on the basis that replication of a study confirms your own work or beliefs counts for nothing. The way a scientist would be “right” in such a betting market is by using their skill in a superior way to make the most objective bets on whether results would or would not replicate. Researchers who knew in advance their work would be subject to a market test would also face an additional form of accountability that would likely modulate their reporting of results.

At least one study has found that, yes, a betting market where scientists wager on the likelihood of experimental results replicating was more accurate than expert opinion alone. In psychology, there has been a controversy over the last decade about a potentially large number of published studies with results subsequent researchers could not replicate. The Reproducibility Project: Psychology has been working on replicating studies from top psychology journals. Anna Dreber, a behavioral economist at the Stockholm School of Economics, with several colleagues set up a betting market based on these replication attempts. They recruited a bunch of experts in the relevant fields and asked their opinions on the likelihood the Reproducibility Project would replicate the results of forty-four studies. They then gave those experts money to bet on each study’s replication in a prediction market.

Experts engaging in traditional peer review, providing their opinion on whether an experimental result would replicate, were right 58% of the time. A betting market in which the traders were the exact same experts and those experts had money on the line predicted correctly 71% of the time.

A lot of people were surprised to learn that the expert opinion expressed as a bet was more accurate than expert opinion expressed through peer review, since peer review is considered a rock-solid foundation of the scientific method. Of course, this result shouldn’t be surprising to readers of this book. We know that scientists are dedicated to truthseeking and take peer review seriously. Arguably, there is already an implied betting element in the scientific process, in that researchers and peer reviewers have a reputational stake in the quality of their review. But we know that scientists, like judges—and like us—are human and subject to these patterns of confirmatory thought. Making the risk explicit rather than implicit refocuses us all to be more objective.

A growing number of businesses are, in fact, implementing betting markets to solve for the difficulties in getting and encouraging contrary opinions. Companies implementing prediction markets to test decisions include Google, Microsoft, General Electric, Eli Lilly, Pfizer, and Siemens. People are more willing to offer their opinion when the goal is to win a bet rather than get along with people in a room.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.