One way to answer the question “What makes instances of X good?” is to look at instances of X that everyone agrees are good, and reverse-engineer the reasons why those instances are good. Or, you could identify bad instances of X, reverse-engineer what makes them bad, then avoid doing that.
For instance, last year Dey et al. (great paper) reviewed one particular kind of opinion/perspectives paper in ecology: “research prioritization” papers. That is, papers with titles like “100 key questions for the next century of ecological research” or “Emerging issues in global change research” (I made those titles up, but I’m sure you recognize the type). Dey et al. couldn’t find any detectable effect of such papers on the direction of ecological research.* So insofar as the goal of those papers is to influence future ecological research (and if that’s not the goal, what is the goal?), those papers apparently are ineffective. So perhaps “research prioritization” papers provide an example of what not to do when it comes to writing opinion/perspectives papers in ecology.
But what about positive examples? What are the most important or influential opinion/perspectives papers in the history of ecology?
The Dept. of Biological Sciences at the University of Calgary is hiring a tenure-track asst. professor in eukaryotic genetics. Ad here. Here are the key details, and a bit of context for non-Canadians (who are welcome to apply!):
Application deadline is Mar. 29, 2021, for a start date as early as July 1, 2021 (you can start later)
It’s a broad ad; candidates who self-identify as evolutionary biologists are among those from whom we’d welcome applications. Candidates using live-cell imaging techniques and/or genome editing tools are particularly encouraged to apply.
The University of Calgary is one of Canada’s leading research universities. We have about 30,000ish undergrads and 8,000ish graduate students. The Dept. of Biological Sciences is the biggest department on campus, we have 50ish faculty and 160ish graduate students. Between our department and other units on campus, we have a bunch of outstanding evolutionary biologists working with genomic data (Sean Rogers, Sam Yeaman, James Wasmuth, others). We have a bunch of shared core facilities at our med school, including sequencing and live cell imaging. Calgary is a great place to be a geneticist.
By law, applications from Canadian citizens and permanent residents have to be given priority, but do NOT let that stop you from applying if you don’t fit those categories. I was a US citizen when Calgary hired me. We just hired Kelsey Lucas, who is a US citizen. Every application will be given full consideration, so if you think you might want the job, don’t take yourself out of the running by not applying.
No. From my fairly comprehensive database of 476 ecological meta-analyses, here’s a graph of weighted mean effect size vs. meta-analysis publication year:
This graph is very crude. It includes meta-analyses based on different effect size measures. And it’s not showing you error bars around those means (as you’d expect, the means that are farther from zero tend to have bigger error bars). But better graphs wouldn’t change the basic picture: there’s no trend over time. It’s not the case that ecologists decades ago discovered all the big effects, so that now we’ve all moved on to studying smaller effects.
This result doesn’t surprise me, and I’m guessing it doesn’t surprise most of you either. There’s an infinity of effects that ecologists could study. The supply of big effects is not some exhaustible finite resource. And ecologists’ choices about what effects to study don’t have much to do with how big those effects are on average.
Via my fairly comprehensive database of 476 ecological meta-analyses: here’s the year range covered by each meta-analysis (publication year of the oldest paper included in the meta-analysis up to the publication year of the most recent paper), as a function of the meta-analysis publication year. Note that some data points are actually several identical data points, because I was too lazy to jitter the plot:
As you can see, there’s no trend. More recent meta-analyses do not cover longer time spans than older meta-analyses.* Ecological meta-analyses typically include papers published over a 10-30 year period, and always have.
Which I think is kind of interesting. After all, more recent meta-analyses have a longer time span of ecology they could draw from–but they don’t. Why not?
Recently, I showed poll results and other data tentatively making three points:
The typical ecological meta-analysis includes 50ish effect sizes
Many ecologists think that an ecological meta-analysis only needs 50ish effect sizes in order to provide a stable estimate of the mean effect size
In fact, ecological meta-analyses typically needs 250-500 effect sizes, or maybe even more, to provide a stable estimate of the mean effect size
In other words, many ecologists overestimate how informative most meta-analyses are about the mean effect size. Why?
I don’t think it’s because most ecologists haven’t ever thought about this issue (is it?). I don’t think most ecologists just subconsciously assume that however big a typical ecological meta-analysis is, that’s surely big enough (is it?). And I don’t think it’s because most ecologists are unaware of the assumptions and statistical properties of the hierarchical random effects models used for most ecological meta-analyses (is it?). So what’s the thinking? Here are the lines of thinking I’ve heard, in our comment threads and in correspondence with colleagues, and my responses to them.
tl;dr: I don’t come to bury Caesar meta-analyses. Meta-analyses can be useful! They’re a good tool for ecologists to have in our toolbox! I’m not one of those rare-but-annoying ecologists who think that meta-analyses and the ecologists who do them are somehow Bad. I just think that, for certain purposes, meta-analyses in ecology aren’t as useful as many ecologists think they are. I think it would be healthy to have a clear-eyed discussion about that. Maybe there are ways to do meta-analyses better. Or better ways to accomplish some of the goals we’re trying to accomplish with meta-analyses.