From Meg:
Joan Strassmann has a thoughtful post on how to review an NSF grant proposal. Her key point is that, while there are four things that reviewers should take into account, many reviewers get hung up on just one of these (the feasibility of the methods and the project). This made me feel better about what I tend to focus on in reviews. In particular, I definitely take into account the past productivity of the PIs, since, to me, spending $600K (or whatever amount) on research that will never see the light of day (in the form of publications) is really wasteful. This relates to NSF’s “Big Pitch” where some proposals were short and anonymous to reviewers. The panel with anonymous proposals reached a very different conclusion than the one without anonymous reviews. Part of this, as ProfLikeSubstance pointed out, might be the difference in proposal format. But, as both Joan and ProfLikeSubstance point out, previous productivity of a potential PI is important, too.
(Jeremy adds: The Canadian NSERC system already uses short (5 page) proposals as proposals, not preproposals. And these cover your entire research program for the next 5 years, not a single project. For this to work, proposal authors and reviewers obviously can’t be worried solely, or even mainly, about nitpicky methodological details. NSERC referees score proposals on three equally-weighted criteria: track record of the PI over the past 6 years (publications, etc.), the quality of the proposal, and training of “highly qualified personnel”, which includes undergrad assistants, grad students, technicians, postdocs, etc. Methods get evaluated as one aspect of “quality of the proposal”, but not in nitpicky detail. As long as it looks like the PI has a reasonable idea of what she’s doing, that’s fine. (And if it turns out she didn’t know what she was doing, she won’t produce any good papers and she’ll get a lousy score on “track record” next time around…) If it’s of interest to enough readers, I may post one or two of my old NSERC grants, just so people can see what a reasonably good short proposal looks like.)
Gerty-Z and Joe Simonis have had what I think is a great idea, which is to solicit questions from faculty who want to make the academic environment better for LGBT students, but who aren’t sure what to do. If you have questions about how to be a faculty ally, or about how to make your class/university environment more welcoming, check out Gerty’s blog, where you can submit questions. (There is a way to submit questions anonymously if you prefer.) You can also submit questions via twitter using the hashtag #AlliesFTW.
From Jeremy:
Head on over to the Phylonetworks blog for Carnival of Evolution #52. I liked this fun little post on visual metaphors for evolutionary relationships (evolutionary trellises, anyone?)
From the archives:
Here’s my very first substantive blog post. Still one of my favorites. Think community ecology is just a stamp collection of special cases? Well, as I explain in this post, that’s our fault, not nature’s–if we want general answers, we need to ask different questions.
On the other hand, there’s such a thing as too much generality, as I explain in this old post. My prime example of overgeneralization comes from the most cited Oikos paper of all time. Curious? Click through…
I’m delighted every time I hear about proposals being evaluated for big ideas and not nit-picky techniques. I wish a sociologist would do a study evaluating scientific productivity by big ideas people vs. those that over-critique methods of others. Of course, when we do our work, we have to get the details right, and I’m a real stickler for that in my group, for example counting things blind whenever possible. That sociologist could take the reviews of a few panel groups, all successful scientists, evaluate them for attention to big ideas vs. methods details, then look at the panelist’s own career…
As a proposal writer, and a reviewer, I am opposed to proposals being presented anonymously to peer-reviewers. The very same research proposal may be eminently feasible when proposed by one researcher and not at all feasible when proposed by another, depending on their respective experiences, their knowledge and expertise, their abilities to attract students and RAs, and the differential support provided by their respective institutions. It it thus not possible to adequately peer-review proposals where the proposers’ identities are anonymous to the reviewers.