A while back we invited you to ask us anything. Here are our answers to our next two questions, from Pavel Dodonov and Dave Fryxell.
Jeremy’s answers: If you want to be invited to review papers, submit papers. Editors often ask for reviews from authors who’ve recently published on the topic, whether in the same journal or in another journal. More broadly, be active in your field in other ways, for instance by presenting at conferences and being active on social media. The more people who’ve heard of you, the more likely you are to be asked to do a review, all else being equal. If you’re a grad student or postdoc, ask your supervisor if you can assist on a review your supervisor has been asked to do, or do the review for your supervisor. Your supervisor will need to get the editor’s permission, but most editors will happily agree to this.
I don’t think you can generalize about who does the best reviews. Purely anecdotally, I think senior grad students and postdocs tend to be particularly conscientious reviewers. But there’s not a ton of variance in my experience (most reviewers are pretty conscientious), and what variance there is isn’t all that correlated with career stage or gender or other reviewer attributes. As to who writes the most positive reviews, I have no idea and I doubt you can generalize. I certainly don’t think you can game the system as an author by requesting certain sorts of people (grad students or whoever) as reviewers in order to try to ensure your ms gets positive reviews! There are some data indicating that reviewers who are blinded to author identity tend to be a bit more negative than unblinded reviewers (sorry, can’t find the link just now). And Charles Fox and his colleagues at Functional Ecology have looked hard for gender biases in various aspects of the peer review system at that journal. They found that outcomes were gender neutral: men and women reviewers scored papers identically on average, and papers with women vs. men as authors (whether first, last, or corresponding author) were equally likely to be sent out for review, scored identically by reviewers, and equally likely to be accepted for publication.
Brian’s answers: I agree with Jeremy. The best way to get invited is to write papers. Authors in your field will then recommend you. AEs looking for names will scan the reference list of the paper they need reviewers for and see your name. And in general, that intangible reputation will increase. One thing not to do – don’t send a letter to the editor in chief. We don’t actually choose reviewers and are not going to forward your email to 50 AEs. If you really want to do something, let people you know who are AEs know that you are interested. Or if you still have a PhD or Postdoc adviser, let them know you would be interested in co-reviewing (some journals allow this, some don’t). But mostly write papers. Soon you will get more invites than you can perform.
I don’t know about who writes the best reviews. More junior people tend to spend more time and find more details (often with 3 pages of grammar and word choice suggestions). More senior people tend to have a knack of cutting to the heart of the matter in a few words (often only writing a few paragraphs or bullet points of really big picture stuff). Both are valuable. But for sure late PhD and on are more than prepared to write high quality reviews, and I regularly encourage people having a hard time finding reviewers to go more junior.
That said, there is no debating who writes the hardest reviews: postdocs! I’m having a hard time putting my finger on the paper but I recently read a peer-reviewed paper that confirmed a long time suspicion of mine. Postdocs and early tenure track researchers are by far the toughest and most likely to recommend reject. I suspect that earlier career students aren’t confident enough to recommend reject. And later career researchers have learned that all research is flawed but are better at identifying the advances a paper contains despite its flaws. My PhD adviser used to require everybody who took a seminar course to answer 3 questions for each paper read – one of which was to identify the thing we liked best about the paper. We’re so good at teaching people to tear down papers, but in fact identifying what a paper contributes and adds is just as important.
Beyond that I don’t see general patterns as far as basic, applied, academic vs non-academic etc. I would say the perfect set up as an AE is to have one reviewer who is highly expert in the field and misses nothing, and one reviewer who is in an allied field and is able to bring a bit of an outside (and hence big picture) perspective.