Via my fairly comprehensive database of 476 ecological meta-analyses: here’s the year range covered by each meta-analysis (publication year of the oldest paper included in the meta-analysis up to the publication year of the most recent paper), as a function of the meta-analysis publication year. Note that some data points are actually several identical data points, because I was too lazy to jitter the plot:
As you can see, there’s no trend. More recent meta-analyses do not cover longer time spans than older meta-analyses.* Ecological meta-analyses typically include papers published over a 10-30 year period, and always have.
Which I think is kind of interesting. After all, more recent meta-analyses have a longer time span of ecology they could draw from–but they don’t. Why not?
I suspect it’s because relevant papers typically only go back 10-30 years. That is, I suspect that it typically takes 10-30 years for ecologists to publish “enough” papers to make a meta-analysis worth conducting (or what meta-analysts see as “enough” papers). Here’s a graph of the number of papers (“studies”) included in ecological meta-analyses, as a function of meta-analysis publication year:
As you can see, the typical ecological meta-analysis includes 10-30 papers or so, and always has. Once there are “enough” papers, somebody’s going to publish a meta-analysis of them.**
If that speculation is right, then we can infer the typical rate at which ecologists collectively publish papers on “meta-analysis-sized” topics. It’s typically 1-2 papers per year on average, over the first 1-3 decades of research on the topic. Which sounds like a pretty low publication rate when you put it like that!***
An alternative possibility is that meta-analysts tend to overlook older papers. But I find that possibility hard to square with the data. Why would all meta-analysts, regardless of when they were working, tend to overlook papers that were more than 10-30 years old at the time? I mean, it’s not as if Web of Science’s coverage only goes back to 30 years before whatever the current year is!
Does any of this surprise you? Interest you? Bore the socks off you? Or what? Looking forward to your comments, as always.
*And in case you were wondering: the most recent paper in a meta-analysis generally was published 0-5 years before the meta-analysis. That gap hasn’t grown, or shrunk, over time.
**Note that many papers include multiple effect sizes, so most meta-analyses include more effect sizes than papers. The median ecological meta-analysis includes about 60 effect sizes, and that median hasn’t really moved much over time.
***It would be fun to keep pursuing this line of thought. Can you infer something about whether the breadth of “meta-analysis-sized” topics is increasing or decreasing over time? And the annual rate at which ecologists publish papers keeps increasing. Can we tell from the available data if that increase is due to ecologists publishing on a growing range of topics, as opposed to publishing more papers/year on any given topic?