I recently received an automated email asking me to review an NSF grant. And immediately after I received that email, I got a surprise: a personal email from Doug Levey, a Program Director responsible for population and community ecology at the NSF Division of Environmental Biology. Yes, a personal email–addressed only to me, clearly not following any template, explaining specifically why NSF really wanted me, Jeremy Fox, to review this grant. I’d have agreed to do the review even without the personal note (the grant really was right up my alley, as the personal note indicated). But I was surprised and intrigued to get a personal email from a Program Director. I mean, surely Program Directors have lots of better things to do with their time than send personal emails to grant referees! I had a lot of questions: Why is NSF using personal emails to solicit for grant referees? Is NSF DEB sending such emails to everyone? Or only to certain people, and if so, why those people? So I asked Doug if he could satisfy my curiosity, and he was kind enough to reply. I thought his reply would be of interest to many readers, and so with Doug’s permission I’ve paraphrased it below.
Doug sends personal notes to about 40% of the people from whom he requests reviews. It’s prompted by one of two situations: lots of people have already declined to review the grant or failed to respond, or because he really wants a review from a particular person who has specialist expertise making them uniquely qualified to review the grant.
Doug reports that he typically needs to send 3-4 review requests for every review he receives. These are known as “ad hoc reviews” and are in addition to the 3 reviews that are always provided by the folks who serve on review panels (“panel reviews”). Population and Community Ecology shoots for 3 ad hoc and 3 panel reviews for each of ~150 proposals/panel. With several panels/year and that response rate, finding enough reviewers isn’t easy, given the size of the pool of potential reviewers.
Personal notes work–they increase the positive response rate by ~30%. Whether they work well enough to justify the extra time it takes to write them is an open question.
I want to thank Doug for taking the time to satisfy my curiosity. I think the fact that Doug is doing this is a dramatic illustration of how much pressure the peer review system is under, and how that pressure has grown over time (NSF Program Directors didn’t used to see any need to send personal notes!) And the fact that the personal notes work is a nice illustration of how it is simply not true that people are “too busy” to review. Yes, everyone has lots of demands on their time–and we choose which demands to agree to. That personal notes can affect our choices is a reminder that we do have choices. Doug Levey is choosing to spend a lot of extra time writing people personal notes in an attempt to make sure that every NSF grant gets the full and fair evaluation it deserves. Something to keep in mind next time you’re asked to review for NSF and feel the urge to decline because you’re “too busy”.