Scientists are reviewing the peer-review process, and early returns are in: Scientists loathe rejection as much as everyone else.
Through last week, submitters to peer-reviewed journals delivered more than 400 journal ratings to SciRev — a site founded by two economists — grading the speed, feedback and overall experience of having their work evaluated for publication by peers. For every broader criticism of peer review that scientists have leveled before — for example, that there aren’t enough competent reviewers to handle the growing number of papers — scientists have several corresponding, more personal criticisms of the process.
The two-thirds of raters whose papers were accepted were, overall, quite pleased, providing an average score of 3.78 out of a maximum of five points. About one-third of review reviewers had their papers rejected. The average score they gave was 1.85. Eight researchers who withdrew their papers — basically, unsubmitted them, one after waiting more than three years — gave an average rating of 0.25.
These results are statistically significant overall but not ready to publish in any peer-reviewed journal worth its salt. Fewer than three dozen journals have been rated more than once, and only one — PLoS One, which employs what Nature described as a “publish first, judge later” ethos — has more than four ratings. SciRev also doesn’t provide any evidence the reviews are based on genuine experiences. “We do not yet have any data available for analysis,” Janine Huisman, executive director of SciRev and an economist, said in an email response to a request for information on reviews thus far.
It’s nonetheless interesting to examine the rare group of journals that elicited excellent ratings from authors whose papers they rejected. Five journals each got a rating of five from a rejected author. If these are genuine ratings, they demonstrate that the low ratings for other rejecting journals may be less about sour grapes than how journals reject: too slowly. The fabulous five rejecters made authors wait an average of almost eight weeks to get the bad news, compared with almost 19 weeks for rejecting journals with lower ratings. And reviewers used words like “efficient” and “professional” to describe the process.
Some of the happy rejectees may also simply have mistakenly entered a positive rating, since it didn’t match their words. It’s hard to believe the author who called a referee report “unnecessarily offensive” found the overall experience excellent.
Herein lie lessons for rejecters everywhere, from book publishers to online daters: 1) Don’t dawdle — spit out the bad news; and 2) If the person you’re rejecting rates your rejection highly, it might be a typo.