A new article has just been published in the journal Science, titled: “Who’s afraid of peer review?”

The article can be found here.

The author of the study, John Bohannon, created a fake PI at a fake university in a developing country and submitted a fictitious article to over 300 peer-reviewed journals (changing the author and university affiliation each time). The paper described a molecule that can be extracted from a species of lichen that will slow the growth of cancer cells. According to Bohannon, there were very obvious errors intentionally written into the paper:

 “There are numerous red flags in the papers, with the most obvious in the first data plot. The graph’s caption claims that it shows a “dose-dependent” effect on cell growth—the paper’s linchpin result—but the data clearly show the opposite. The molecule is tested across a staggering five orders of magnitude of concentrations, all the way down to picomolar levels. And yet, the effect on the cells is modest and identical at every concentration.”

“One glance at the paper’s Materials & Methods section reveals the obvious explanation for this outlandish result. The molecule was dissolved in a buffer containing an unusually large amount of ethanol. The control group of cells should have been treated with the same buffer, but they were not. Thus, the molecule’s observed “effect” on cell growth is nothing more than the well-known cytotoxic effect of alcohol.”

“The second experiment is more outrageous. The control cells were not exposed to any radiation at all. So the observed “interactive effect” is nothing more than the standard inhibition of cell growth by radiation. Indeed, it would be impossible to conclude anything from this experiment.”

Despite the obvious errors and issues in the paper, 157 journals accepted the paper, 98 rejected it, and 49 had not responded (of the 49, 29 seemed to no longer exist, and the other 20 were still reviewing the paper).

Bohannon reports that the paper made it through the editing process 255 times and that 60% of those times, there was no evidence of peer review. If the paper was outright rejected, it would not have any peer-review, so that is great. but the paper was only rejected 98 times. That leaves a percentage of journals that must have accepted without any peer-review. This is one of many startling details revealed by the study. Another is this:

“Of the 106 journals that discernibly performed any review, 70% ultimately accepted the paper. Most reviews focused exclusively on the paper’s layout, formatting, and language. This sting did not waste the time of many legitimate peer reviewers. Only 36 of the 304 submissions generated review comments recognizing any of the paper’s scientific problems. And 16 of those papers were accepted by the editors despite the damning reviews.”

 A fake paper written by a fake author from a fake author, written about a study that never occured (and had glaring issues) was accepted for publication more often than it was rejected. It was submitted to “peer-reviewed” scientific journals. The point of the peer-review is to identify the merit of a study. This study reveals some issues inherent to peer-review, and open-access publishing specifically.

Open-access publication is great. As a scientist and advocate, it is the ideal situation. Everyone can view the paper and have access to the supplemental material. I have never understood the point of restricted access to scientific publication. In order for people to become more knowledgeable about something, they need to have access to the subject. We can’t expect the public, legislature, or even managers to help make informed decisions unless they have access to the newest scientific research on the subject. This is the point of open-access publication.

However, publication is a business, and there are many our there who will sacrifice quality for profit.  Such publishers are called “predatory publishers” by this study. When an author submits to an open-access journal and is accepted, there is an author fee (ranging from less than $100 to over $200). As a PI at a major university in a western country, this is no big deal. But to a lesser known author, grad student, or researcher from a developing country, this fee is a barrier. It is also a source of profit for a publisher. As the study points out, there are many journals out there with names that are similar to top journals (for the sake of credibility). These journals or publishers mask their geographic locations and often do not have real addresses. It would seem that these are essentially shells for companies that wish to prey on authors. They advertise that they are accepting author submissions, accept papers with minor review (or with no review), and then charge the author. This essentially leads to a journal that is not vetted and has no credibility, but the publisher is making a profit. Bohannon focused on open-access publications, and it would appear that many of these predatory publishers are using the appeal of “open-access” as a way to draw in more submissions. They are taking a great idea, that is designed to make valid and important scientific research available to all, undermining it, and making a profit in the process. This is not something we need in the scientific community. Hopefully this new study will help make people more aware of the situation. Peer-review and open-access are both important parts of the scientific process. Clearly some sort of reform is necessary.