Just an FYI: if your ms is rejected after review by one journal, and you resubmit to a different journal, it’s fairly common for it to go to some or all of the same referees that reviewed it for the first journal. I don’t know exactly how common (how could you ever get data on that?) But it’s not rare. For instance, just in the last few months I’ve twice reviewed an ms I’d previously reviewed for another journal.
Why do journal editors acting independently nevertheless often end up choosing some of the same referees for a given paper? In part because a minority of academics do a majority of the reviewing, so the pool of potential referees isn’t as big as you might think. At least not the in the eyes of editors who like to get reviews from referees whom they know from experience will agree to review and do a good job. In part because many editors use similar criteria for choosing referees. For instance, seeking reviews from leading experts on a topic–who often are few in number, even for topics that you might not think of as narrow or specialized. Many editors also like to seek reviews from people who’ve published on the topic recently, which often isn’t that many people. And in part because, if an ms heavily cites or discusses someone’s work, that someone likely will be asked to review it.*
This means that, as an author, you need to take the reviews of your rejected mss seriously and revise as needed before resubmitting to a different journal (even if only to clarify the ms and prevent misunderstandings you think the first referees had). Do not just resubmit a previously-reviewed ms to a different journal without revising, on the assumption that you got “unlucky” with your referees and that the “new” referees will like your ms. Because the “new” referees could well be the “old” referees–and nothing annoys referees more than authors who ignore their comments!
The flip side of that is that nothing pleases referees more than authors who take their comments seriously and revise appropriately. So do it!
*In case you’re wondering: no, a referee who’s reviewed your ms before for another journal is not “biased” and does not have a “conflict of interest”. Not even if the previous review was negative. See this old post for discussion. Although journals that have double-blind review may avoid referees who’ve learned the authors’ identities by reviewing the ms previously for a journal lacking double-blind review (I’m not sure on this).
Dynamic Ecology 
I have a question on a related note, Jeremy.
I was wondering what you and your readers think of asking a specialist to comment on a draft version of a manuscript prior to submission, and then later proposing that same specialist as a reviewer when you eventually submit to a journal. Assuming, of course, that this person is thanked in the acknowledgements section and is not a regular collaborator.
For what its worth, I actually tried this once before. My manuscript was closely related to a study published a few months earlier, so I asked one of the authors for feedback. He was kind enough to offer detailed feedback and we even discussed the best journals to aim for. Even though I proposed this person to the handling editor, I don’t think he ever reviewed the manuscript (although one of his co-authors did and even signed his review).
Although the manuscript was eventually accepted, I still wonder how the editor perceived the whole situation initially. (FYI: I explained the whole situation in the cover letter, so the editor was fully informed.)
Personally, I wouldn’t do this, and I’ve never heard of it being done. If you’re going to do it, you’d want to do it as you did it. But even then, I wouldn’t bother. I’d be very surprised if any editor would use the referee you suggested.
I agree with you. It does seem a bit strange reading it again… even to me.
But perhaps there is an opportunity here! By asking your harshest critic for feedback prior to submission, you get the most critical feedback on your work while simultaneously discounting a potentially negative reviewer.
(I’m kidding, of course, but it is still something to think about…)
I’ve been trained that if somebody reads a mss and gives feedback they belong in the acknowledgements. And anybody in the acknowledgements should not be a reviewer. This makes good sense to me.
However, I am aware of a significant minority that do what you describe intentionally (pass a mss around for pre-review comment, see who has the best reaction, and then suggest them as reviewers). NB this is clearly not what you did, especially because you were forthright with the editor (and in conflict of interest revealing all and letting a higher independent person decide is close to absolution from conflict of interest). However, others are definitely gaming the system this way and you know it is gaming because they never mention the pre-reviews in the acknowledgements or to the editor.
One time I was in this situation (I pre-reviewed and then asked to review) and I refused to review because of a conflict of interest (explained to the editor) and the editor sounded pretty annoyed at the authors. So don’t include me in these games!
“I’ve been trained that if somebody reads a mss and gives feedback they belong in the acknowledgements. And anybody in the acknowledgements should not be a reviewer. This makes good sense to me.”
That’s perfectly clear – it makes good sense to me now too.
The scenario you describe is pretty disgraceful. It’s also a real pity because I think pre-review is underutilised (in my experience, at least) and stories like yours certainly won’t encourage other scientist to start using it…
Falko – I 100% agree pre-review is an excellent and underutilized tool. To me the proper way to use pre-review is to place them in the acknowledgements and also briefly mention all pre-review readers you have in the cover letter to the editor*. But if you do it this way it is an excellent approach that is honest and makes editors and reviewers pre-disposed to think that you are valuing their time (which you are). This is not so different from what you describe having done yourself.
*(of course people have to actually have read and given at least some feedback – you can’t just email-bomb famous ecologists and then list them when they didn’t even open your email – I’ve seen that too …)
“I’ve been trained that if somebody reads a mss and gives feedback they belong in the acknowledgements. And anybody in the acknowledgements should not be a reviewer. This makes good sense to me.”
Yes!
I’ve also seen the opposite scenario to what you describe, where people also try to game the system by acknowledging someone who did not provide any substantive feedback on the study or manuscript. I don’t know if the authors are hoping that it will seem like Big Name X approved the manuscript, or if they are trying to avoid Big Name X as a reviewer, but, either way, this is clearly not an appropriate use of acknowledgments. Hope Jahren tweeted about this recently, and I think someone storified it, but can’t recall who.
“I’ve also seen the opposite scenario to what you describe, where people also try to game the system by acknowledging someone who did not provide any substantive feedback on the study or manuscript.”
Seriously?! I’ve never heard of that. That’s really asinine. And here’s the thing: *it doesn’t even work*. Speaking as a former editor who’s spoken to many other editors, no editor gives a rat’s ass who you acknowledge. Nobody thinks that your ms must be awesome because Dr. Famous is acknowledged.
As for acknowledging someone who you think will hate the ms in hopes of keeping it from being sent to them, that is not only unethical, it’s a terrible idea because word will get out that you did it, at which point you’re screwed professionally.
I may post on this kind of thing at some point. People who try to “game the system” in this sort of way are making mistakes even by their own ruthlessly Machiavellian standards. They’re like students who find ridiculous ways to cheat–ways that not only are likely to be detected, but that *don’t even help you even if they go undetected*.
Thanks to Jeramia Ory for the Storify! https://storify.com/DrLabRatOry/acknowledgements-and-anonymity
If I had pre-reviewed your manuscript as a favor to you and then was asked by the editor to review the manuscript, I would refuse because it would represent a conflct of interest for me.
I’ll just add as a frequent reviewer I often get a mss a 2nd time from a different journal like Jeremy mentioned. If the mss hasn’t changed at all I will often just say in the review “I reviewed this previously for a different journal and none of my comments have been incorporated so my opinions remain the same. Below is my previous review” and attach my previous review.
I don’t think you have to respond to everything a reviewer says (if it was rereview at the same journal you could reasonably argue with a point or two and it would be fine). But the bottom line is as a reviewer I am donating my time to help you get published. Don’t take me for granted!
Exactly!
I’ve done this a lot too! I really wonder what proportion of the times I see the same manuscript more than once, is because I’ve been suggested by the authors as a reviewer.
So far, my record is four. I saw the same paper four times, four different journals, the authors never did the relatively easy changes that I was recommending, all the way down the tiers.
Eventually saw it in PLOS One, with the changes I recommended.
I think my record is 3 journals. SOme people are slow learners …
@Brian, Terry:
My record is 3 as well. Although in my personal record-setting case there were some (in my view inadequate) revisions with each iteration, so it wasn’t as if I was being repeatedly ignored.
Yes, I’ve done this before. I also agree with Meghan that it’s extremely irritating to receive a verbatim manuscript that I’ve just reviewed for another journal. It’s remarkable to me that an author would ignore the well-meaning and constructive comments that I’ve provided at great effort. It suggests to me that the author has never reviewed any manuscripts and therefore has no idea of the effort involved (or, the author him or herself does cheap, shoddy, snarky reviews and expects that others do as well). In any case, it’s highly unprofessional behavior.
@Ellen:
“It suggests to me that the author has never reviewed any manuscripts and therefore has no idea of the effort involved”
From my own anecdotal experience, this is rarely if ever the reason for authors to ignore reviewer comments when resubmitting to another journal. In most of the cases when I’ve reviewed an ms a second time for another journal and the authors failed to take my previous comments seriously, the authors were experienced faculty who often serve as reviewers (and even editors!) themselves.
I long ago stopped trying to guess why authors would ever ignore reviewer comments when resubmitting elsewhere. I’m sure they must have their reasons, and probably different reasons in different cases. But I can’t fathom those reasons.
We recently resubmitted a paper that had been rejected at one journal after receiving four thoughtful, helpful reviews. We spent a long time working on the revisions, mostly because the reviewers had good ideas, but also knowing that it may well go back to one of them (and knowing how annoying it is to spend time on a review, only to receive the exact same manuscript for review at a different journal). I hope it goes back to one of the earlier reviewers! This manuscript is the one that inspired this post:
https://dynamicecology.wordpress.com/2015/03/09/is-it-worse-to-admit-a-paper-was-rejected-than-to-not-acknowledge-helpful-anonymous-reviews/
In the end, we decided not to acknowledge the anonymous reviewers from the previous submission on our first submission to a new journal, but plan on adding them in later.
But. What if the reviewer is actually, 100%, wrong? I would ignore their comments resubmitting in that instance.
I can believe a reviewer is 100% wrong on one issue, in which case (as I mentioned above) I am fine with not doign what the reviewer wanted. But:
a) most reviewers bring of 3 or 10 things. Its not too likely they’re wrong on all of them. There should be plenty you can incorporate.
b) If the reviewer is 100% wrong, there is a good chance you can improve how you communicated around this issue in your mss. Even if they’re wrong it is a form of feedback on what you need to do better.
Sorry, no buts. Even if a reviewer says something 100% wrong–and sometimes they do!–what are the odds that they’re the only person who would make that mistake? If a reviewer says something 100% wrong, in most cases that’s a sign you need to revise the ms so as to disabuse readers of that mistake.
Yes, sometimes an incompetent or sloppy referee says something so obviously and bizarrely incorrect that your best move when resubmitting elsewhere is to just ignore it. But this is really rare. Much rarer than the temptation all authors experience to ignore or too-hastily-dismiss referees with whom they disagree. Sometimes when you *think* the referee is 100% wrong, it’s actually *you* who’s wrong!
I definitely agree there. If the reviewers misunderstand something in your ms, then I think it is on you as an author to make sure that you have communicated your point clearly. Assuming the reviewer is also a member of the target audience of their paper, there is a good chance that if they made the error, a number of your potential readers will as well.
Exactly. I just had a reviewer who was 100% wrong on what could be inferred from a pattern of PCA loadings. The manuscript was sent to a 3rd reviewer who had the same misinterpretation (agreeing with the other reviewer). I was forced to do heavy revision to acknowledge that this misinterpretation must be common, at least in my field.
I believe this current string eludes to something I’ve often contemplated: Is it necessary, or even a necessary evil, to restrict reviews to the handful of folks considered experts for a given topic? I put this out there, because I think there could be much gained in expanding the reviewer pool beyond the “Gurus” in the room. To me, anyway, a great deal of the review process is about the Xs & Os. Were hypotheses sound and testable? Did the protocols and related data sufficiently assess those hypotheses? Were the statistics appropriate for a) the hypotheses and b) the data? Were all critical assumptions adhered to for each statistic? Were statistical outcomes appropriately interpreted?
These X and O questions could be reviewed by any competent scientist- not just one in your specific area. Obviously, the broader philosophical aspects of the work should be left to someone intimately familiar with the topic, but that could be assigned to a single reviewer. I feel an approach such as this could increase objectivity and reduce bias in the long run, because I think we all recognize that as experts of anything, there is a tendency to see the trees but not the forest.
Editors choose reviewers for various reasons. Often different reviewers are chosen for different reasons. For instance, if an ms includes some highly technical aspects (very sophisticated statistics, technical mathematical derivations, etc.), an editor might ask a referee with the relevant specialized technical expertise, and another referee who can comment more broadly on the mss interest to a wide readership. For instance, I was once asked to review a Science paper from Rich Lenski’s group on the evolution of evolvability (this is no secret; I signed my review, and the paper was published). I am definitely *not* an expert on the evolution of evolvability! So I assume I was asked to review to provide the prospective of a broadly-curious non-specialist, which I did.
The situation I was put in a while ago that really pissed me off – I was sent a MS for a pre-review, as was another colleague. I gave it a lot of attention, and a great deal of feedback, as did the other colleague – we shared notes (there was no secret about it, didn’t break any confidentiality) and pretty much agreed. Then I was sent the MS to review for a journal, and in the cover letter the author stated that the colleague and I had reviewed the MS for him earlier. But as a reviewer I could see that NONE of our concerns had been addressed, the MS was exactly the same one we’d seen the first time. So my review to the editor was, in essence – yes, I saw this MS before, and here is what I had told the authors in private, and now I see they have not followed any of it, so I would recommend they revise it as I originally suggested.
Wow! That’s another data point in my growing collection of anecdotes about people trying to game the scientific publication system in ways that not only are likely to fail (and to cost them if they do), but that wouldn’t be of any benefit even if they worked. Seriously, WTF could be the point of getting pre-reviews and then ignoring them? Especially when there’s some chance that the ms might go back to the people who did the pre-reviews?
I think the author in question thought our suggestions were too much work, wanted a quick turnaround, and assumed that using our names were some magic spell that would mean neither of us were sent it to review, or that we’d refuse to review it. I not only told the editor, I also warned one of the co-authors who I was fairly certain had no idea such shenanigans were happening.
“Because the “new” referees could well be the “old” referees–and nothing annoys referees more than authors who ignore their comments!”
You know where this won’t happen? If you resubmit your manuscript to a journal that’s attempting double-blind peer review. I recently reviewed a manuscript for a top journal and was one of the three reviewers who liked it. The editor and another reviewer didn’t. I then got a request to review a manuscript with an identical abstract at another good journal that was attempting double-blind peer review. I had to tell them I knew who the authors were but would be happy to review the paper again. The editors at this second journal said they’d never encountered this situation (which surprised me–I’m guessing some reviewers don’t disclose that they’ve seen the paper before), and they decided I shouldn’t review. I felt bad for the authors, since I was very qualified to review the manuscript and really liked it. Presumably, several other good reviewers were disqualified too.
A similar thing just happened to me. I reviewed an ms for a journal without double-blind review, and later was asked to review the same ms (I could tell it was the same from the abstract) by a journal with double blind review. I told the second journal that I’d seen the ms before and so knew the names of the authors (I always disclose to any journal editor when I’ve previously reviewed an ms for another journal). But in my case the journal told me they still wanted a review from me, due to some unique circumstances they didn’t specify (which shouldn’t be taken to indicate anything ontoward was going on–in general, there can be lots of good reasons for wanting to make an exception to any general policy). It was this experience that led to my remark about journals with double-blind review at the end of the post.
Pingback: The dumbest thing I ever wrote to an editor | Scientist Sees Squirrel
Pingback: Reviewing with imposter syndrome | Scientist Sees Squirrel
Pingback: How to write, and read, a (manuscript) rejection letter | Scientist Sees Squirrel
Pingback: Reconciling the two functions of peer review | Scientist Sees Squirrel