Ask Us Anything: how to get invited to review manuscripts, and who writes the best reviews

A while back we invited you to ask us anything. Here are our answers to our next two questions, from Pavel Dodonov and Dave Fryxell.

Pavel asked: I like reviewing; how can I get more invitations to review manuscripts?

Dave asked: Who writes the best or most positive reviews in ecology? Grad students? postdocs? Senior faculty? Nonacademics?

Jeremy’s answers: If you want to be invited to review papers, submit papers. Editors often ask for reviews from authors who’ve recently published on the topic, whether in the same journal or in another journal. More broadly, be active in your field in other ways, for instance by presenting at conferences and being active on social media. The more people who’ve heard of you, the more likely you are to be asked to do a review, all else being equal. If you’re a grad student or postdoc, ask your supervisor if you can assist on a review your supervisor has been asked to do, or do the review for your supervisor. Your supervisor will need to get the editor’s permission, but most editors will happily agree to this.

I don’t think you can generalize about who does the best reviews. Purely anecdotally, I think senior grad students and postdocs tend to be particularly conscientious reviewers. But there’s not a ton of variance in my experience (most reviewers are pretty conscientious), and what variance there is isn’t all that correlated with career stage or gender or other reviewer attributes. As to who writes the most positive reviews, I have no idea and I doubt you can generalize. I certainly don’t think you can game the system as an author by requesting certain sorts of people (grad students or whoever) as reviewers in order to try to ensure your ms gets positive reviews! There are some data indicating that reviewers who are blinded to author identity tend to be a bit more negative than unblinded reviewers (sorry, can’t find the link just now). And Charles Fox and his colleagues at Functional Ecology have looked hard for gender biases in various aspects of the peer review system at that journal. They found that outcomes were gender neutral: men and women reviewers scored papers identically on average, and papers with women vs. men as authors (whether first, last, or corresponding author) were equally likely to be sent out for review, scored identically by reviewers, and equally likely to be accepted for publication.

Brian’s answers: I agree with Jeremy. The best way to get invited is to write papers. Authors in your field will then recommend you. AEs looking for names will scan the reference list of the paper they need reviewers for and see your name. And in general, that intangible reputation will increase. One thing not to do – don’t send a letter to the editor in chief. We don’t actually choose reviewers and are not going to forward your email to 50 AEs. If you really want to do something, let people you know who are AEs know that you are interested. Or if you still have a PhD or Postdoc adviser, let them know you would be interested in co-reviewing (some journals allow this, some don’t). But mostly write papers. Soon you will get more invites than you can perform.

I don’t know about who writes the best reviews. More junior people tend to spend more time and find more details (often with 3 pages of grammar and word choice suggestions). More senior people tend to have a knack of cutting to the heart of the matter in a few words (often only writing a few paragraphs or bullet points of really big picture stuff). Both are valuable. But for sure late PhD and on are more than prepared to write high quality reviews, and I regularly encourage people having a hard time finding reviewers to go more junior.

That said, there is no debating who writes the hardest reviews: postdocs! I’m having a hard time putting my finger on the paper but I recently read a peer-reviewed paper that confirmed a long time suspicion of mine. Postdocs and early tenure track researchers are by far the toughest and most likely to recommend reject. I suspect that earlier career students aren’t confident enough to recommend reject. And later career researchers have learned that all research is flawed but are better at identifying the advances a paper contains despite its flaws. My PhD adviser used to require everybody who took a seminar course to answer 3 questions for each paper read – one of which was to identify the thing we liked best about the paper. We’re so good at teaching people to tear down papers, but in fact identifying what a paper contributes and adds is just as important.

Beyond that I don’t see general patterns as far as basic, applied, academic vs non-academic etc. I would say the perfect set up as an AE is to have one reviewer who is highly expert in the field and misses nothing, and one reviewer who is in an allied field and is able to bring a bit of an outside (and hence big picture) perspective.

12 thoughts on “Ask Us Anything: how to get invited to review manuscripts, and who writes the best reviews

  1. For being asked to review more: “publish more” isn’t a very helpful answer (sorry)! But I agree entirely, offer to co-review with supervisor, or let an AE with expertise in your area know. Most journals list their AEs on their web page.

    As to who does the best reviews: here I differ from both Jeremy and Brian a bit. In my analysis-free anecdata experience, I find early-career reviewers are almost always superb. Wrote about it here, jointly with Tim Poisot:

    • “publish more” isn’t a very helpful answer (sorry)!”

      Well, it’s an answer to a related question: Why am I not being asked to review? The answer, for grad students and postdocs, is usually “Because you haven’t published much/anything yet, so editors don’t know who you are or what you work on.”

      Re: ECRs as reviewers, they’re increasingly being asked to do more reviews, as journals broaden their reviewer bases to compensate for faculty turning down more review requests. The BES journals published some data on this a while back (sorry, can’t find the link just now).

  2. I think I posted on Facebook once, back when I first started my postdoc, something like “I enjoy reviewing, always decide on whether to review a paper immediately when receiving the invitation, I usually say yes, and complete the review within 3 work days (except if traveling, which I disclose), and when I say no I suggest one to five alternative reviewer names with email addresses.” Let’s just say I apparently have enough AE friends on facebook that I had a very full inbox the next few weeks. The percentage of review requests I accepted dropped from then on ;), but proud to say the rest in that quote still holds true.

    So perhaps even if you haven’t published much, but you have AEs in your network (at your uni or part on your online social network), letting them know (1) your specialization (2) that you are a conscientious reviewer and (3) enjoy reviewing couldn’t hurt. It worked for me and I only had two first author publications at the time.

    Jeremy, Meg, and Brian, when someone provides a review, is the quality of the review marked in the database? In other words, do reviewers get reputations for useful reviews and hence get asked more?

    • “do reviewers get reputations for useful reviews and hence get asked more?”

      Yes, informally. And also reputations for agreeing to do reviews and reputations for timeliness.

      My experience is that these informal reputations mostly get built up with editors for whom you’ve reviewed, but spread more widely only very slowly. There are a few editors who repeatedly ask me for reviews, because they often handle mss on topics I work on and because they know from experience that I’ll probably say yes and do a good job. (A couple of these editors are also my friends.)

      Whether reviewers also build up formal reputations (i.e. whether journal reviewer databases record info on review quality, and whether editors use that info to choose reviewers), I don’t know. Brian and Meghan could speak to that.

      You can also get a reputation for being a bad reviewer. Back when I was an editor at Oikos, I learned through experience that there were a few people who never agreed to do reviews, so I stopped asking them.

      • At least at the American Naturalist, AEs score reviews for quality, and average scores are available in the reviewer database. I do pay some attention to them in selecting reviewers – not sure if other AEs do.

      • Yes also at GEB we have a reviewer scoring system. Some AEs use it, some don’t. We’ve also implemented an annual prize for the top reviewers. That’s not necessarily going to give an overall increase in the flow of information. But hopefully it brings the community to value those attributes (promptness, thoughtfulness and willingness to review).

    • I don’t know anyone who even looks at Publon. I’m sure some people do. But it hasn’t taken off in a significant way. Being on Publon won’t increase your chances of being invited to review.

      I’m not approving or disapproving of this state of affairs, just describing it.

  3. Thanks for the response! I had also been wondering whether doing good reviews increases your chances of being invited to review more often, which was also answered in the comments. And it does make a lot of sense that, in order to be invited to review, one has to publish, as it would be weird for someone with little publishing experience to review other people’s manuscripts…

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.