Poll: What does the “Are you willing to review a revised manuscript” question mean to you?

A while back, there was a twitter discussion related to Associate Editors (AEs) sending manuscripts back out for review when the changes are pretty minor. One part of the discussion indicated that there’s some variation in interpretation of the “Would you be able to review a revised version of this manuscript?” question. This topic recently came up again in some emails between Brian, Jeremy, and me (and then again on twitter after I mentioned writing a post on it), so I figured it’s worth a quick poll:

(Please answer the poll before reading the rest of the post!)

As an example of the sort of thing that made me think about this, last year I reviewed a manuscript where my review had a total of three comments:

  1. Asking the authors to better explain how they identified certain individuals
  2. A wording suggestion, and
  3. Asking the authors to move something to earlier in the paper

(The other reviewer’s comments were similarly minor.) I submitted my review and, as I always do, checked the “yes” button when asked if I’d be willing to review a revised version of the manuscript. (I’m not really sure what would make me click “no” on that, to be honest.)

When I got a request to review the revision, I agreed. I had forgotten how minor my concerns were, though, and, since I was in a particularly busy stretch right then, didn’t open the file right away, waiting until I had enough time to do a full review. When I got a chance to work on the review, I opened the file and wondered why it had come back to me at all. In my opinion, an associate editor could and should evaluate whether the authors had addressed that level of concern. As an AE, I go through each revised manuscript carefully anyway, and don’t see why we should ask two other people to do that, too. (I should note this is just for minor edits, recognizing that there will be some variation in what is considered minor.)

In cases like the one I just described, I think the AE should be able to evaluate whether the changes the authors made addressed the concerns raised by the reviewer. For manuscripts I handle as an AE, I want to save reviewers’ time for cases where the changes made were more complex and where I feel like it would help to have another opinion (or two).

In the twitter discussion of this, though, some people indicated that people answering “yes” to the “Are you willing to review a revised manuscript?” question (which is a standard part of the review submission process) could be taken as the person saying they really want to see a revised manuscript. That conversation made it clear that some AEs send manuscripts with minor edits back out for review because they don’t want to upset reviewers. And here I thought that sending something with minor revisions back out for review would annoy reviewers by wasting their time!

After realizing there is confusion on this topic, I have now started adding something like “While I could review a revised version of this manuscript if you think that would be useful, I do not need to see a revised version” in my confidential comments to the AE. (Except typing this just made me realized I forgot to do that on the last review I submitted!) Perhaps we could use a second question: “Do you want to review the revised version of this manuscript?” I’m not sure if that would really help (and, as an AE, I certainly wouldn’t view that as binding), but it would at least avoid ambiguity about whether the reviewer really wants to see the manuscript again.

Have you received requests to review papers again that you felt were unnecessary? Or have you not been asked to review a revision and been annoyed about that? If you are an AE, have you had pushback for either sending something back to original reviewers or for not sending it back to them (or, lucky you, both)?

42 thoughts on “Poll: What does the “Are you willing to review a revised manuscript” question mean to you?

  1. Whether or not I agree to look at a revision usually depends on how major the revision is: if it’s a big revision I say “yes”, if not I say “no”. That’s probably not what the question means but it’s how I interpret it.

    “have you not been asked to review a revision and been annoyed about that?”

    Oh yes! One case in particular – I’ll not go into details – but it was a major journal that published a paper some of us believe is flawed and which ought to have been re-reviewed by the more critical reviewers who had clearly read it more carefully than the non-critical reviewers. But that didn’t happen and when I raised it with the EiC and AE I was ignored, i.e. no reply at all. I was not impressed….

    • “Whether or not I agree to look at a revision usually depends on how major the revision is: if it’s a big revision I say “yes”, if not I say “no”. That’s probably not what the question means but it’s how I interpret it.”
      That’s probably a reasonable way to go about answering it, but it never occurred to me to answer it any way other than literally!

  2. I’m surprised that so far 42% of respondents say they want to see the ms again to be sure their concerns have been addressed. I thought that would be a much smaller minority.

    I think the followup poll should ask about the role of reviewers. How many people think that mss should only be accepted when/if all reviewers are completely satisfied? Vs people who think that reviewers advise but editors decide (meaning that sometimes editors will overrule reviewers, or stop the review process before all reviewers are satisfied)?

    • I have to confess that I was part of that 42%, but after clicking, I realized that I misunderstood the question. Wondering if others did the same. I thought the question was referring to when you are actually asked to review the manuscript again, which I would likely have said yes to. Not blaming the poll, but blaming reading a blog early in the morning. 🙂

    • I’m also surprised it’s as high as it is. We could have a follow up poll asking about the role of reviewers — and include an option for the Caseys of the world where they can say it’s too early/their coffee hasn’t kicked in yet. 🙂

      I do think the general question of whether people think all reviewers need to be satisfied would be interesting. I sometimes overrule reviewers as an AE and feel like I wouldn’t be doing my job if I always just went with whatever the reviewers said. I certainly value their opinions (otherwise I wouldn’t have asked them!), but I don’t think that means the authors always need to do what they say.

  3. I must be weird or something, because I mean exactly and only what a “yes” response to the bare question would appear to mean: “I’m willing to review the paper again, if requested”. It has nothing to do with whether I want to see it again, or whether I think I need to see it again.

    I’ve answered “no” to this question twice, and meant exactly and only “No, I never want to see this again”. In both cases the manuscripts needed only minor revisions, but it was the third time I’d seen them, and the authors were clearly never going to take the advice of the reviewers and make those minor revisions, so I had no interest in re-reviewing 4th revisions only to see the same lack of attention to detail.

    I guess I’ve never been much for nuance!

    • I mean the same thing as you when I answer it! It never occurred to me to answer otherwise, until the twitter conversation that I was part of.

      I think that sounds like a completely reasonable reason to answer “no”! I’ve been asked to re-review a manuscript in a case where the AE overruled my concern about the manuscript. That re-review was a little strange to do!

  4. I’ve been told that the general difference between “major revision” and “minor revision” (in journals that have those choices) is that major revision needs to go through the reviewers again, while minor don’t need a new round of reviews. I’ve used that logic when asked to suggest a decision, so I’ve interpreted the question if I can actually review revised version as very much just meaning that.

    • In the twitter discussion on this (well, in one of them — there have been multiple at this point!), Allen Moore indicated he thinks you should say “reject” if you plan to send it back out:

      (At least, that’s how I read that tweet, though I’m not exactly sure of what the “but maybe” means.)
      So, I guess there’s uncertainty about this seemingly straightforward question plus the uncertainty about what reject, major revisions, and minor revisions should mean! 🙂

  5. I agree with W Ray. When I tick that box I mean that if the AE decides that the paper needs to be sent out for review again I am happy to be one of the reviewers. It is not a linked to the review I have given or a need to check in on how my comments were addressed.

  6. I always check “yes”, feeling that, having agreed to the review the manuscript, I should agree to another revision if the editors ask me to. It feels a bit like an obligation and certainly does not mean to me that I necessarily want to see a revised version.

    However, recently, an established professor made the comment that she always checks “no”, feeling that having already made a detailed review of the manuscript and provided her opinion, it is entirely up to the editors to decide whether the revised version adequately addresses the comments and the standards for the journal. I thought that was a really interesting view and I wondered whether it was shared by many and whether I should adopt it. It seems from this post that it is not shared by many, and it seems that there is not much of a common understanding of what is expected of reviewers and editors. Some reviewers seem to feel it’s disrespectful not to ask them to see the revised version, while other reviewers seem to feel its disrespectful to ask them to review the manuscript again since it should be the editor’s job to assess the manuscript at that point…

    • I always say “no” but most AEs ignore that and still send me a revised ms – if my comments / edits were substantial. I find revisions much more time consuming because I have to address the science and scholarship again AND I have to see if the authors heeded my advice. So, I always say “no” to the question of whether I would be willing to review a revision. But, even when I say “no” to the original question, I will always say yes if the AE comes back to me despite my “no”. Weird, but that has been my mantra. Speaking of which, as an AE, I don’t recall ever seeing what the reviewer indicated to the question of whether they would be willing to look at a revision. Where is that indicated on manuscriptcentral and other software interfaces?

      • These comments are really interesting! I find it so interesting that there’s this much variation in how people think about this question. It never would have occurred to me without the twitter discussions!

        As for Eric’s point: I confess that, as an AE, I don’t think I’ve ever checked whether the reviewer said “yes” or “no” to that question. It’s not intentional — it’s just something that I forget to do.

  7. So I had an interesting situation recently in which I was asked to review the response to *another* reviewers comments. Apparently, the other reviewer never responded to the request to re-review, but the editors weren’t able to sign off on the revision until they had the okay from someone (and they may have gone to yet another reviewer to check on it if I didn’t sign off). In the end, the comments were relatively minor, and I was happy to help the authors out by preventing them from getting another set of reviewer comments to deal with, but I think it was pretty crummy for that other reviewer to flake out. One could say that the editor should “do their job and make the call”, which I agree with in principle, but in practice, not re-reviewing only punishes the authors, not the system, flawed as it may or may not be. I don’t know how often this happens, though.

    • Does that mean that the journal has a policy of requiring reviewers to look at responses to reviews? That is, to not allow the AE to make the call? Don’t they trust their AEs?

      In terms of not reviewing the revision: I was once asked to review a revision and wrote back to say I could do it but would need an extra week due to travel. They replied and said they would find another reviewer instead. That surprised me!

      • Not sure, but I think this journal has a policy that requires sign off by reviewers on all revision (editors are professional, not academic). Which is sort of nuts. If the editor can’t make a call, then the review cycle can become endless. If a journal is fancy enough, then I guess they can get away with it, which is an unfortunate reality.

        So weird that they went to someone else because of a week’s extra time… I would imagine it would still be faster than getting a fresh reviewer, no? (In the case I was mentioning, the re-reviewer was like 2-3 weeks overdue.) I don’t think I’ll ever understand this publishing business.

  8. If any reviewer would be offended at not being offered to review a revision, I would argue that the reviewers do not decide whether a manuscript is accepted or not. Nor should they. This is the responsibility of the editor (though I hope that she/he would weigh the reviewer’s comments heavily in their decision). I’ve certainly seen papers with multiple pretty good reviews rejected by the editor. I’ve also seen the opposite situation unfortunately. Still, that’s how this is supposed to work.

    • Whilst I agree that editors have final responsibility to decide whether or not to accept a paper, that judgement has got to be backed up by appropriate evidence, in this case from the subject-specialist reviewers. Otherwise what’s the point of sending it out for review? A situation in which “papers with multiple pretty good reviews rejected by the editor…[and]…the opposite situation” is symptomatic of an editor not doing their job properly, particularly in the latter case. If an editor is really unsure about which call to make then the manuscript ought to go out to another reviewer.

      • I agree with Casey. I recently overruled a reviewer on something they viewed as a fatal flaw because it didn’t seem like a fatal flaw to me. In that case, I let the authors edit and respond to the concerns and then sent it to a different reviewer after revision.

      • Nobody’s talking about blithely ignoring reviewers.

        When I was an editor at oikos and made decisions contrary to a reviewers recommendation, it was usually because I disagreed with the reviewer on matters on which I was in a better position to judge. Like whether the ms was of broad interest. I also sometimes overruled reviewers on technical issues on which I knew the reviewer was wrong, or which weren’t as important to fix as the reviewer thought. For instance, if a reviewer insisted on swutching to a generalized linear model when it was obvious that the substantive conclusion would remain unchanged, I would suggest the change to the authors while making clear the change wasn’t required.

      • “Nobody’s talking about blithely ignoring reviewers.”

        But isn’t that what Casey was alluding to when he said “I’ve certainly seen papers with multiple pretty good reviews rejected by the editor. I’ve also seen the opposite situation unfortunately.”?

        As I said, the editor should and does have the last call, and I agree with you that when the editor has a better perspective on technical issues, etc., they can over rule reviews. I’ve done it myself, but informed the reviewers why it’s been done.

        But all of this is missing the original point; what we are talking about here is reviewers not being given an opportunity to re-review manuscripts that they have rejected on good grounds and not being told reasons why.

      • Re: not being told reasons why, most journals routinely share the other reviews and the decision letters with the reviewers as well as the authors. I take it you want more information than that?

      • No, in the specific case I was referring to (which is what we are discussing here) I wanted just that information, but didn’t get it.

      • I wonder if part of the issue here comes down to the way the recommendation/decision letter is written. In the cases where I’ve overruled a reviewer, I try to explain my logic very clearly in the decision letter, for both the authors and the reviewer.

  9. Many editors send revised mss out for 2nd review as a matter of course. I figure that my job as an editor is to edit: most of the time I can look at the revision and see whether the issues are resolved. Sometimes I can’t, and then I ask for a 2nd look by reviewers, but I think much time and effort are wasted with most of these requests.

  10. 1) I’ve always assumed the question means, “If we send you the revised version, would you review it?” It never occurred to me that some people might interpret it to mean, “Do you feel that it is necessary that we send you the revised version for your approval?”

    2) If the journal rejects the original paper you reviewed but allows a revision, which you also review, does that count as two papers reviewed on your c.v. or one? (Not a rhetorical question; I really want to know the answer!)

      • Agreed, I’d also count it as one. But then I don’t tally up the number of reviews I’ve done on my CV, I just say how many journals I’ve reviewed for and include some examples.

  11. The fact that nearly 40% of people view this phrase to mean that they want to see whether the revised manuscript addresses their review satisfactorally, suprises me. Such a simple sentence and yet there are two incredibly different interpretations (possibly more). Now imagine sending an email about a conflict to a collaborator or student; if we can’t seem to agree on what this simple sentence means, I think it is almost certain that any email regarding a conflict is nearly gauranteed to be interpereted by the reader in a way we don’t mean it to. A bit of a random tangent, I know, but when reading this post, the first thing that came to mind was your post on ambiguous emails.

  12. As an Editor-in-Chief I hope (expect) my AEs to make the call about whether to send out a revised paper yet again. If it was a relatively minor revision then I would expect them to accept it without sending out again. When acting as an AE and dealing with a minor I always attach a note (electronically) to the manuscript saying no need to send out for review again. This is because the electronic systems we use these days, have a default that adds the reviewers who clicked the “happy to see revision again” to the manuscript and I suspect that many AEs and Editors who are all busy people often dealing with editorial stuff at home, mostly just tell the system to invite the Ro reviewers without thinking too hard about it, which is why I add the note 🙂

  13. I initially thought Meg’s revised question “do you want to review the revised version of this manuscript?” could replace the ambiguous “are you willing” question. If not willing, they won’t want to. But then there would the reviewer with hurt pride because they wan’t to see it again, and the AE didn’t want to be held hostage and have to hold it up for another month or more.

    These electronic management systems and all their rigid questions, and canned auto-letters are definitely a blessing and a curse.
    Thanks for kicking off a good discussion. As Matthew Holden noted, email language and meanings can be so misunderstood.

    • Yes, good point on the second question potentially making AEs feel obliged to send it back out again if a reviewer says “yes” to that one. That would not be good.

  14. From the AE side, I don’t think I’ve ever paid attention to that box. I don’t interpret it as a requirement to send a paper out for re-review and agree with Meg that I try to avoid a second review unless obviously warranted to avoid annoying reviewers. I also tend to get pretty peeved as an author when editors don’t want to make a decision and keeping sending things to re-review that seem minor to me. But, eye of the beholder and all that.

    • Yes to all of this, including not paying attention to that box as an AE. In my case, it’s not intentional — it’s just something that has never occurred to me to look at when I’m thinking about sending a manuscript back out.

  15. Pingback: Poll: should all reviewers be satisfied before a paper is accepted for publication? | Dynamic Ecology

  16. I just received a review request from a journal that said, “You have previously provided a report on the above paper and noted that you would like to see any new version of the paper.” When I reviewed the original paper, I checked the “Willing to review a revision” box. But I certainly did not say that I would like to see a new version of the paper! So it looks like the journal was interpreting this comment differently from my interpretation.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s