Rigor Mortis by Richard Harris

Rigor Mortis by Richard Harris

Author:Richard Harris
Language: eng
Format: epub
Publisher: Basic Books
Published: 2017-04-04T04:00:00+00:00


Chapter Seven

SHOW YOUR WORK

BRIAN NOSEK KNOWS all about HARKing. The psychology professor at the University of Virginia realized to his chagrin that he was guilty of that bad practice himself. “When you talk to graduate students in my lab they will describe this for you. We sit down at the beginning of a semester and talk about experimental design. They go and do the study. When they come back, and we’re looking at the data, the first question that I have is, why did we do the study?” He can’t typically remember what hypothesis they were trying to test, so he can’t determine whether the results confirm a hypothesis or explore a new one. “We do both [confirmation and exploration] all the time, but it’s hard to distinguish it because we’re busy. We’re distracted. We’re just doing lots of stuff.”

After thinking about his own research practices, Nosek had an epiphany. Simply increasing transparency could go a long way toward reducing the reproducibility problems that plague biomedical research. For starters, scientists would avoid the pitfall of HARKing if they did a better job of keeping track of their ideas—especially if they documented what they were planning to do before they actually sat down to do it. Though utterly basic, this idea is not baked into the routines in Nosek’s field of psychology or in biomedical research. So Nosek decided to do something about that. He started a nonprofit called the Center for Open Science, housed incongruously in the business center of the Omni Hotel in downtown Charlottesville, Virginia. His staff, mostly software developers, sit at MacBook computers hooked up to gleaming white displays. Everyone works in one big, open room and can wander over to cupboards stocked with free food. The main project at the center is a data repository called the Open Science Framework.

Nosek put this new system of organization and transparency to the test in 2012 by trying to reproduce some of the studies in his field. After he floated this idea on a listserv, more than two hundred other scientists from around the world said they wanted to get in on the action. Over the next few years, this loose affiliation of psychologists selected one hundred research papers and set about redoing them. The results made news around the world. “Psychology’s Fears Confirmed: Rechecked Studies Don’t Hold Up,” read the page-one New York Times headline on August 28, 2015. Two-thirds of the reproduced results were so weak that they didn’t reach statistical significance. Many of those at least leaned in the same direction as the original study but could not on their own be considered evidence of an effect. About a third of the studies actually suggested there was either no effect whatsoever or even an effect opposite to what the original paper reported. There has been some pushback against these findings, but the broad conclusions still stand—and Nosek happily pointed out that his critics had made their case by accessing his readily available working material, which in itself was a triumph for transparency.



Download



Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.