Journal referees often disagree. Referee disagreements can be challenging for editors to handle. How should editors deal with them?
One common approach, especially among editors at selective journals, is to just reject the paper. That is, anything other than unanimous approval or near-approval of the referees is fatal. This is the path of least resistance for editors. It’s usually justified on the grounds that there are lots of good, or potentially-good, papers to choose from and so decisions have to be made somehow. Plus, editors often aren’t specialists on the topics of the papers that they handle, so when faced with a disagreement between specialists they may feel it’s safest to reject the ms, figuring that if that’s the wrong call the ms can just be published somewhere else.
Personally, while I appreciate the motivations for that approach, it’s one I tried to avoid taking during my days as an editor at Oikos. I felt like my job as an editor went beyond simply choosing the referees. I took the view that my job was to make decisions, informed but not dictated by the reviews. I also felt that it was my job to give authors clear direction as to how the ms could be improved. Just saying “address the concerns of the referees” isn’t a clear direction if the referees disagree with one another. So if the referees disagreed on an important point, or on their overall view of the ms, I saw it as my job to decide who was right, and then explain my reasoning to the author.
These two approaches are really two ends of a continuum. For instance, probably few editors literally just compile the “votes” of the referees and reject any paper that doesn’t get a unanimous “yes”. But I do think editors vary a fair bit in how willing they are to dig into the details of explicit or implicit disagreements among referees*, and how willing they are to take sides when referees disagree (particularly when doing so would involve overruling a negative review).
Of course, another approach is for editors to hedge their bets by rejecting the paper if referees disagree, but with the possibility of resubmission as a “new” ms. We’ve talked about that approach before.
Another approach, if two referees disagree, is for the editor to get a third “tiebreaker” review. I don’t know how common this practice is, although anecdotally I think it’s rare in ecology. There are various reasons why an editor might want a “tiebreaker” review. An editor might be genuinely unsure how to resolve a substantive disagreement between two referees, and so want additional input before deciding who’s right. Personally, I think that’s the best reason to want a “tiebreaker” review. At the other end of the continuum, the editor might simply want a tiebreaking vote on whether to accept the ms or not. Personally, I think that’s an abdication of editorial responsibility; I hope nobody ever does that.
But while I certainly have my own preferences and my reasons for them, there are principled and pragmatic arguments for various approaches. So as an author, what approach do you prefer editors to take when referees disagree? What about when you’re one of the referees? What about about when you’re the editor? What about when you’re a reader of the journal? Why? Looking forward to your comments.
*An implicit disagreement being a case where referee A sees some problem with the ms that isn’t mentioned by referee B. This is common. Sometimes this occurs because referees have different interests and so read the same ms with different eyes (e.g., a specialist in system X might care a lot about how the results relate to previous work in system X, while another referee might not care). Sometimes this occurs because one referee spotted something another missed. Neither of those are really disagreements. But sometimes it occurs because one referee thinks something is a problem but another referee doesn’t and so doesn’t mention it. It’s for this reason that, when I’m reviewing a paper, I try to be explicit about where I think there aren’t problems, as well as where I think there are problems. Indeed, if there’s some aspect of an ms that I think is fine, but that I anticipate that other referees might object to, I sometimes go out of my way to explain why I disagree with those objections, so as to give the author ammunition against any reviewers who might raise them.