Last month the 2015 NSERC Discovery Grant results were released. If you don’t know, NSERC is the Canadian equivalent of NSF in the US or NERC in the UK–the main national government agency funding basic research in ecology, evolution, and other non-biomedical fields. But the similarities end there, because NSERC has a unique funding strategy. The purpose of Discovery Grants is to support excellent research programs, not individual projects. NSERC also prefers to support many strong research programs rather than trying to identify and fund only the very few best ones. That’s a policy for which there are good “bet hedging” arguments, given the unpredictability of advances in basic research. See here for more on this.*
What does this mean in practice? Here’s some data from the linked report. Read on if you’re a Canadian and so have a personal interest in these numbers, or if you’re a non-Canadian and are curious how the other half lives. You non-Canadians may want to sit down so you don’t faint from shock after reading the first bullet. 🙂
- Success rate for established researchers (those renewing an existing grant) was 82%. For “early career researchers” (basically, junior PIs who’ve never held a Discovery Grant) it was 65%. For “established researchers not holding a grant” (mostly PIs who’ve previously held a Discovery Grant but don’t currently do so because their most recent application(s) was/were declined), it was 38%.
- Average award size was $32,132/year ($35K for those renewing an existing grant; $26-27K for others). If that seems small to you, well, that’s the price of funding many research programs. Aside: the vast majority of these were 5 year awards; that’s the standard and maximum length.
- The distribution of award sizes was highly concentrated–the majority of the awards were for about $25K-35K/year. Only a very few “high fliers” got more than $60K/year, and nobody got more than $130K/year. Hardly anybody except for mathematicians (whose award sizes run very low compared to other fields) got less than 20K/year.**
- A slight majority of successfully-renewing PIs got an increase in nominal terms. The modal change for successfully-renewing PIs was small (±10% or less), but changes anywhere in the range of ±30% or so weren’t all that rare.
- The numbers for the Evolution and Ecology panel were broadly similar to the overall numbers: success rates of 90%, 50%, and 45% for renewers, early career folks, and non-renewing experienced folks, respectively. Average award size of $35.7K for renewers and about $25.5K for others.
- Male and female applicants at the same career stage had essentially identical success rates and average award sizes.
- All of the above is quite similar to last year. Click through to the report if you want some data on longer-term trends (since 2009).
On a personal note, I was up for renewal this year. Like most people, I put my grants on my cv, so I’m not revealing any secrets when I say that I got renewed for 5 years, for $26K/year. I have slightly mixed feelings about that. On the one hand, I got renewed, which is great! And I’m fortunate to be in Canada, where people doing good work can be confident in getting renewed. These data help me keep my mixed feelings in perspective. Given that I’m in Canada and I got renewed, any grant-related regrets I might have are definitely “First World problems”.
On the other hand, I’m a little disappointed to have gotten cut a bit relative to my previous award ($29K). And I dropped a bit relative to the E&E average, compared to where I was 5 years ago. But really, you can’t read much into that. My award places me within the range where most Canadian researchers fall, which is where I belong. Given the inevitable stochasticity associated with even a very thorough and fair evaluation process like NSERC’s, it would be silly to get upset about getting 26K rather than the 35K I was hoping for.
More substantively, all three external reviewers wanted more detail about the specific projects I proposed. Which has me kicking myself, because lack of detail about specific projects was the main issue I was worried about when I submitted. I had reasons for proposing numerous specific projects, and for feeling like the resulting lack of methodological detail wouldn’t be too much of an issue. But it turned out to be more of an issue than I’d hoped. The reviewers’ comments were very fair, and I take them seriously. So next time around I’ll probably try erring on the side of fewer projects developed in more detail. I’ll perhaps also try focusing more on my core lines of research. There’s no rule (or even unwritten expectation) that says that everything my lab is planning has to be in my NSERC Discovery Grant proposal. And then maybe I’ll just end up kicking myself for something else, but that’s the way it goes. All you can do is develop and explain your ideas as best you can, and then take the feedback seriously and try to do better next time. Performance, feedback, revision–if it’s good enough for evolution, it’s good enough for me.*** 🙂
p.s. I highlighted my blogging in my application, as a scholarly contribution. As I expected, it doesn’t appear to have made any appreciable difference one way or the other to the evaluation of my application. Only one of the three external reviewers mentioned it, and only in passing. Anecdotal, of course, but it’s consistent with my view that the rewards and costs of blogging are mostly intangible. Note that you can’t necessarily follow my example here. Depending on what sort of blogging you do, and your funding agency’s rules, you might have to list blogging in a different part of your application, or not list it at all.
*There are also other differences between NSERC’s policies and those of similar agencies in other countries, some of which reflect the structure of Canadian academia more broadly. For instance, there’s no overhead in NSERC Discovery grants (overhead is handled separately), no summer salary for PIs (Canadian academic appointments are all 12 month appointments), and Canadian grad students are mostly supported by TAships or scholarships during the fall and winter. So no, funding agencies in other countries probably could not adopt NSERC’s policies whole hog. But agencies in other countries could consider elements of the NSERC Discovery Grant model, for instance by reducing average award sizes so as to hopefully increase success rates. Note that Canada does quite well compared to other countries, including the US, on measures of research productivity per-dollar and per-investigator, so the NSERC model isn’t some egalitarian thing designed to make individual scientists happy at a cost to their collective productivity. The Discovery Grant model works, though of course so do other funding models.
**Note that the distribution of award sizes has almost nothing to do with PIs’ proposed budgets or with the cost of research (there is a cost of research component to award sizes, but it only affects a small fraction of awards). Applications are scored on a 6-point scale in each of three equally-weighted categories (quality of your research over the past 6 years, quality of your proposal, and training of students/postdocs/technicians/etc.). Within each of the 12 evaluation groups (i.e. research fields, of which “Evolution and Ecology” is one), everybody with the same total score gets the same amount of money. You’re not under any obligation to do anything you proposed to do. The assumption is that PIs will be able to spend whatever money they get and spend it well, no matter how much they asked for.
***I of course do get feedback from colleagues on draft proposals before submitting. That’s not just for newbies, everyone I know does this.
Dynamic Ecology 
I always enjoy learning how other countries allocate public grant funds. In the US, not everything is investigator-based, and we have equivalents of what you describe. The NSF & NIH have program-project grants and center grants to support broad collaborative efforts. In some cases the approach is highly efficient but I think to some extent suppresses a diversity of ideas. For example, I was a research faculty member at an NIH funded center for electron tomography. Our group was only one of two or three such facilities in the nation, and we received the lion’s share of the funding pie. Anyone wanting to integrate electron tomography into their research had to submit proposals to our group, and we decided who got in, and who did not. Our decisions largely reflected the research bias of our director, so many ideas went unexplored. I was curious, Jeremy, if Canada’s approach had any adverse effects similar to what I observed in the states. Thanks!
NSERC of course has various programs to fund different sorts of work–a postdoc program, student scholarship programs, a program for purchasing major equipment, a program for supporting big collaborative efforts, programs for academia-industry partnerships, etc. And there are of course various other funding agencies in Canada–we have an NIH equivalent (CIHR), we have Genome Canada, etc. So I’m sure it’s possible that something like what you describe could happen in Canada, but I can’t point to any examples. I’m not the sort of person who’d be likely to know of any such examples.
Canada’s overall approach to funding scientific research isn’t all that different from that of any other developed Western country, I don’t think. As far as I know, it’s really one specific program–the NSERC Discovery Grant program–that’s unique.
The NSERC model of funding sounds just brilliant to me – here in Australia we have a far lower success rate with Australian Research Council discovery grants (overall success rate 19.9% last year), but get much more money when we succeed. This leads to a “rich get richer, poor get poorer” cycle, because the funded researchers can buy help and hire postdocs (and often have fellowships that reduce/remove their teaching load) while non funded researchers battle along on fumes and teach a lot. All of this leaves the non-funded researchers even less competitive than the funded researchers in the next round.
I have heard plenty of people argue that spreading funding too thinly is a bad thing though – has there been any research on which funding distribution model gives the best research outcome? I am betting the Canadian system is better overall [and yes, I take your point that there are other funding schemes in Canada for big equipment, postdoc fellowships, industry collaborations etc, but there are here too].
“This leads to a “rich get richer, poor get poorer” cycle, because the funded researchers can buy help and hire postdocs (and often have fellowships that reduce/remove their teaching load) while non funded researchers battle along on fumes and teach a lot. All of this leaves the non-funded researchers even less competitive than the funded researchers in the next round. ”
This is a very interesting issue. NSERC’s instructions to reviewers are silent on whether a PI’s track record should be evaluated relative to how much funding the PI has. Well, the rules specify that you can’t use previous grant funding itself as evidence for excellence–but all the usual indicators of excellence correlate with your previous funding. So since any researcher presumably would be way more productive if they had enough to money to hire postdocs and technicians, hire an army of summer assistants, etc., reviewers should evaluate PIs’ track records on a per-dollar basis. I’d be very curious to hear what others think of this idea, might even do a separate post on it at some point.
“I have heard plenty of people argue that spreading funding too thinly is a bad thing though”
There are tradeoffs. Hardly anyone in Canada can hire a postdoc or technician for instance. There is a separate NSERC postdoctoral fellowship program, but it’s extraordinarily competitive. And while DNA sequencing is getting cheaper, you have to do more of it to impress anyone these days, so the amount you have to spend on sequencing to do leading-edge stuff remains constant and high. So beyond the reach of many people on NSERC Discovery Grants.
” has there been any research on which funding distribution model gives the best research outcome?”
Yes. It’s not a literature I know well, and the data I’ve seen are about 10-15 years old and so might be outdated. Keeping those big caveats in mind, on a per-dollar or per-PI basis my recollection is that the UK comes out on top in scientific productivity in many non-biomedical fields. Canada is right up there, in particular in the same ballpark as (or sometimes above) the US depending on what measure of scientific productivity you look at. (Sorry, don’t recall where Australia comes out…) But here’s the thing: those different countries have funding (and more broadly, academic) systems that differ in all sorts of ways. So while I don’t know the literature well, I’d think it would be very hard to make a cast-iron argument that the NSERC Discovery Grant system is just flat-out better (or flat-out worse) than the alternatives. I have various old posts touching on this issue, here’s one: https://dynamicecology.wordpress.com/2013/07/11/in-praise-of-shopkeeper-science/
I definitely agree that this is an issue where one ought to look at data before opining strongly. It really bugs me that lots of scientists are *sure* they know that, say, an ARC-type system is “best”. Even though they can’t back it up with anything more than gut feelings, anecdotes, and “logic” that doesn’t stand up to scrutiny. We’re so data-driven when it comes to doing science, but when it comes to talking about how science ought to be funded a lot of us (including me) have pretty strong opinions relative to the evidence and logic backing them up. Years ago I got really upset with someone who just kept insisting, with no evidence whatsoever, that the NSERC DG system couldn’t possibly be a good idea. It was just “obviously” some sort of uncompetitive communist scheme wasting tons of money on terrible science and terrible scientists. I eventually snapped and resorted to mockery: https://dynamicecology.wordpress.com/2012/01/29/the-last-word-on-us-vs-canadian-funding-systems/
As it happens I have a post nearly ready to go up, strongly in favour of spreading the funding thinly (look for it early next week). I would not take quite the same optimistic view of the stats that you do – NSERC spreads the money less thinly than it used to, and I’m not a fan of its movements toward a more unequal system.
One minor point of disagreement, Jeremy: you say “Canadian grad students are mostly supported by TAships or scholarships during the fall and winter”. This will come as a big surprise to researchers in provinces that lack a provincial scholarship system (i.e., most of them) and at universities where compensation for TAing is a very small fraction of the cost of supporting a grad student (at UNB, TAing defrays about $5K from a total cost around $20K – numbers very rough). This is a big problem when grants are evaluated based on HQP training, but the grant itself will only pay for 1 HQP per year! (Having said that, there are other ways to get HQP, and my grant (like yours) was just renewed – phew!)
“at UNB, TAing defrays about $5K from a total cost around $20K – numbers very rough”
Thanks Stephen. I knew that TA support varied among Canadian universities, but wasn’t aware it varied so much.
Are you really able to support students primarily through TAships? At Calgary, the most a TA can make is ~$8600/year. Support doesn’t vary that much among Canadian universities, at least not to the point where at any school it is sufficient to fund one’s degree.
I only make the point in case some take your comment as implying that scholarships and external funds are not critical to funding graduate education in Canada. (And both funds are usually from NSERC).
“At Calgary, the most a TA can make is ~$8600/year.”
With respect, you’re incorrect. You seem to have misread or misunderstood information from the UCalgary website. Full TAs at Calgary pay about $8600 *per term*, not per year (EDIT: some courses have partial TAships that pay less). Grad students in my dept. (Biological Sciences) ordinarily receive 2 terms of TAship per year. (Their offer letters only guarantee them one, but in practice everybody who wants a second term gets one).
EDIT: To finish the thought: Yes, other sources of funding besides TAships are important to supporting grad students at Calgary. Indeed, students in my department are guaranteed a minimum of $21,000/year (and in practice ordinarily make more than that). Their supervisors are obliged to make up the difference between the guaranteed minimum and departmental support (students who get scholarships also make up part or all of the difference that way). In practice, this usually means that supervisors at Calgary pay their grad students’ summer salaries out of research funds.
I am not certain how the Canadian system operates. In the US, TAships usually involve a tuition and fee waiver, so neither the TA nor the PI incur any curriculum expenses for the TA’s education. The money the TA earns is intended to keep the bloke fed, clothed & housed. Research expenses, at least in the US, are almost entirely grant supported. So if a TA’s PI does not have a grant, you best be creative if you intend to write a dissertation. For my doctoral research, my PI did not have grant funds to support my research, so I acquired my own grant from The Nature Conservancy. It was a similar situation for my MS, where I acquired a grant from the State of Wisconsin.
I don’t know to what extent the world is aware of a really significant shift happening in the US concerning public funding. When I met with Obama cabinet members and US senators in 2013, they described the outlook for public science grants as “bleak over the long haul”. Sequestration in the wake of our financial crisis caused a lot of publicly-funded research programs to go belly-up. Universities were left in the lurch, as nowadays, most big US universities balance their budgets on the backs of professors’ grant funds. So now, many academic institutions are moving away from the public funding system and instead forming public-private partnerships. Thus, for example, the University of Colorado Boulder, a hot bed of science evidenced by several Nobel laureates, announced about 2 years ago they are going full steam ahead with the public-private model to support academic research.
The organization I work for now entirely funds its academic and applied research programs via applied agricultural work, thus we are freed of the boom and bust cycle of public funding in the US. I am curious if Canada is experiencing any of the shifts we observe happening in the US.
“I am curious if Canada is experiencing any of the shifts we observe happening in the US.”
The current Canadian government isn’t much interested in basic scientific research (or indeed, in any independent information whatsoever…), and is gradually moving towards favoring applied research (for which read “subsidized commercial R&D”). I think you’ll find the same broad trend elsewhere.
And the post I mentioned is now up, at http://wp.me/p5x2kS-3J: “Why grant funding should be spread thinly”.
Thin spread sounds good! situation in Sweden is a bit similar to Australia, except that more academics depend on external funding for their own salary. At the end of the day, in environmental sciences the two major funding agencies (similar to NSERC) had in the past few years a success rate of about 8% – and successful projects usually get over 200 k$ a year. Hard for ECR, and the rest. This year they have tried to change that by introducing an “average project size” recommendation, meaning that large projects need justification to be funded…we’ll see how it goes!
I second Stephen’s comments. The TA situation is similar in my department at the U of S.
Another interesting component to the DG awards is the offering of 1 year awards in cases where a researcher’s proposal ranks competitively on the researchers track record and HQP training plan, but the proposed research reviewed badly. This happened to me last year on my first renewal attempt; the committee essentially said “we think you are a better scientist than this proposal”, gave me one year of funding and an opportunity to rewrite. This year I got a 5 year renewal.
It is only anecdotal, but I personally know several people who recently have gotten 1-year awards (sometimes more than one in a row) and then gone on to renew for the full 5 year term. I wonder if this is a deliberate strategy by NSERC, and how common it is.
“It is only anecdotal, but I personally know several people who recently have gotten 1-year awards ”
Yes, NSERC does sometimes award 1 year Discovery Grants in exactly the circumstances you describe. It’s not that common–most awards are 5 year awards–but I think it’s the most common reason for 1 year awards.
Pingback: Why grant funding should be spread thinly | Scientist Sees Squirrel
Pingback: Ask us anything: what are the most common mistakes in grant proposals? | Dynamic Ecology