Gaming the Metrics: Misconduct and Manipulation in Academic Research by Mario Biagioli & Alexandra Lippman

Author:Mario Biagioli & Alexandra Lippman [Biagioli, Mario & Lippman, Alexandra]
Language: eng
Format: epub, pdf
Tags: gaming; metric; citation; misconduct; manipulation; academic research; gaming metrics; peer review; impact factor; fraud; fake; predatory publishing; impact assessment; citation index; open access; journal impact factor; JIF; scholarly communication; research evaluation; publication pressure; publish or perish; education policy; spam; computer-generated articles; altmetric; data quality; performance indicators;, Higher, Evaluation & Assessment, General, education, Library & Information Science, Language Arts & Disciplines
ISBN: 9780262537933
Google: kADJDwAAQBAJ
Publisher: MIT Press
Published: 2020-01-28T00:25:43.975551+00:00

Overall, two-thirds of retractions are due to misconduct, as Fang, Steen, and Casadevall showed in 2012. Sometimes, however, it is hard to tell. Here is a notice that was typical in a particular journal until quite recently: “This article has been withdrawn by the authors.” That is not very helpful, but the Journal of Biological Chemistry, where the “article has been withdrawn by the authors” retractions appeared, must have become tired of having us beat them up after five years, so they changed their policy and now include details (Guengerich, 2015).

Speaking of metrics, here is how they might be related to how long it takes to retract. The average is about three years (Steen, Casadevall, and Fang, 2013). To be a conspiracy theorist for a moment, it is worth noting that three years is a year longer than the amount of time citations count toward a journal’s impact factor. That means if journals can drag out the process, they would not take an impact factor hit. Similarly, authors and universities can drag out the process to make it less likely that a retraction will affect promotion or funding decisions.

And retracted papers keep being cited, often as if they had never been retracted (Budd, Coble, and Anderson 2011). As an illustration, we have a leaderboard of the top ten cited retracted papers (Retraction Watch, 2016b). Number two on the list at the time of this writing was Andrew Wakefield’s infamous Lancet paper claiming a link between autism and vaccines (Wakefield et al., 1998). (Look, I just cited it.) And number one has had far more citations after it was retracted than before. Whether those are positive or negative citations, we do not know.

Which journals retract most? It turns out that journals with the highest impact factor—there are those metrics again–also have the highest rate of retraction. Again, we think that is mostly due to the fact that there are more eyeballs on those journals, although it is certainly possible that scientists are pushing the envelope to publish in those journals in ways that would constitute misconduct. “Meta-scientists” continue to debate the data on whether “publish or perish” plays a significant role in misconduct (Fanelli et al., 2017). And which researchers retract most? We like our leaderboards and our rankings at Retraction Watch. The top retraction holder has 183 (Retraction Watch, 2016a).

Let me leave you with an interesting finding that relies on metrics. A study—whose findings have been replicated by another group (McCook, 2015)—found that when people retract papers for fraud, you see what you would expect: their citations drop (Lu et al., 2013). In fact, citations in their whole subspecialty drop by about ten percent to fifteen percent. That is bad news, which, again, you would expect. But if you retract for honest error, however, and it is clear in the notice that the mistake is the result of honest error, you do not see that drop. So there is an example of using citations—a metric—to figure out what happens after a retraction.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.