Dynamic Ecology has had a couple of recent posts relating to peer review roles (reviewer, associate editor) that seem to have struck a nerve. I want to provide some thoughts on the two fundamental roles of peer-review: gatekeeping and editing.
I think the two notions are fairly clear, but briefly:
- Gatekeeping – the focus is on deciding (or advising others who are deciding) what is “good enough” to go forward (e.g. to go to the next round of review or be published in a given journal, or to be considered good enough for the awarding of a PhD degree).
- Editing – the focus is on giving advice to improve the manuscript. Shorten the discussion. Reorient the introduction around X which is really your main point. Add this additional analysis. It is probably important to note that to me editing is mostly about big picture advice like the examples just given. A list of 50 typos, grammar mistakes, missed citations, and ways to improve word choice is fine, but is not much of a job of editing without the big picture advice in my opinion.
It is my belief these roles are fundamentally orthogonal or independent of each other. One can do zero, just one or both of these roles at the same time. I’m not sure what one is doing if one is neither gatekeeping nor editing but I’ve certainly seen a few reviews that have achieved this in my time. And of course gatekeeping may be done in a selfish fashion (this paper contradicts my work) or altruistic (honest advice to a journal editor about whether the readers of that journal will find the paper interesting). I suppose editing can also be selfish (cite my paper, pay homage to my idea even though it is not really central to your paper) or altruistic (this is my honest best assessment of what would take your paper up another notch).
The relative proportions of gatekeeping vs editing probably naturally vary depending on one’s role. As a PhD adviser reading a student’s first draft editing is foremost. As an external examiner on a PhD committee or as a reviewer for Science or Nature, gatekeeping is probably foremost (indeed in my experience a lot of people get so into the gatekeeping role when invited to review for Science or Nature that no paper ever would be good enough to make it through the gate).
But mostly in my experience a lot of people are innately gatekeepers or editors and don’t do a great job of wearing both hats at the same time, even though wearing both hats simultaneously is probably what is called for in most circumstances (PhD committee member reading a thesis, reviewer or associate editor peer reviewing a manuscript). I can tell you as an Editor-in-Chief (or previously as an associate editor) a reviewer who could wear both hats won my eternal gratitude. Tell me whether the paper is fundamentally sound or unsound and how exciting and novel it is (with some rationale and detail behind those arguments). And even if you think the paper doesn’t belong in the current journal, tell me and the authors the 3-5 most important things they can do to improve their paper (in part because it is kind to the authors and in part because your opinion might get overriden and the paper could go ahead at the current journal).
Unfortunately, I think the world is shifting more and more to gatekeeping. The symptoms from a reviewer are very short reviewers mostly arguing why the paper doesn’t belong in the journal or an associate editor saying “because I have a negative reviewer I cannot recommend the paper goes forward.” But every once in a while you will see a reviewer or an AE who has that rare gift to recognize the diamond in the rough. That there is a nugget of something really cool, so buried that probably even the authors don’t see it. And perhaps recognize that the author is a student. And then be willing to work through three rounds of revision with the authors to completely turn the paper inside out and upside down and then polish and tighten it in its new version to become a great paper. The times I have done that as an AE are among my most satisfying experiences as a peer reviewer (and something I value highly in associate editors as an Editor-In-Chief). It would be inappropriate to name them, but I still remember very clearly some examples and taken almost as much pride in those papers as if I wrote them myself.
No journal with 50 associate editors is going to be completely uniform in the balance of gatekeeping vs editing. But there are journals out there that have a history and reputation of focusing on editing as well as gatekeeping. I hope you think about that as a trait every bit as important as turnaround times and impact factors. Editing often actually goes in direct conflict with turnaround times (aside from the fact one cannot always think deep thoughts on a schedule, journals focused on turnaround times often strongly limit the number of revisions they will go through as this counts against the key statistic of time from first submission to publication).
Do you think gatekeeping and editing are the main roles of peer review or do you see others? Which do you focus on more? As an author or peer reviewer do you remember an example where good patient editing really turned a paper into something much better than the original? Any journals you think are unusually good at editing and not just gatekeeping? Do you think the world is swinging towards gatekeeping at the cost of editing?