As regular readers will know, my side project over the last 18 months has been a meta-analysis of ecological meta-analyses. With the assistance of a very capable undergraduate assistant, I’ve compiled a fairly comprehensive database of 466 ecological meta-analyses.* That’s a lot of meta-analyses!**
Specifically, it’s a lot of meta-analyses compared to how many there are in evolutionary biology. Last winter, an undergraduate independent study student of mine decided to compile meta-analyses from evolutionary biology. She only found 33. Now, she had only limited time to complete her project, so she may well not have found all meta-analyses that would’ve met the inclusion criteria. But still, I don’t think she overlooked several hundred meta-analyses that would’ve met her inclusion criteria. And FWIW (not much…), her data support my offhand impression that there are many fewer meta-analyses in evolutionary biology than in ecology. It’s definitely possible I’m wrong about this, and you think I’m wrong you should definitely say so in the comments.
But assuming for the sake of argument that I’m not wrong, why are there so many more ecological than evolutionary meta-analyses? Because it’s a bit strange, when you think about how central the comparative method is to evolutionary biology. Shouldn’t that emphasis on the value of comparative studies translate into doing lots of meta-analyses like this one? Here are some speculative hypotheses as to why there are so many more ecological than evolutionary meta-analyses. They’re not mutually exclusive hypotheses.
- Evolutionary biology is a smaller field than ecology, so there are just fewer evolution papers, period. There’s some evidence consistent with this possibility. For instance, the annual Evolution meeting is much smaller than the annual ESA meeting.
- Perhaps many of the questions evolutionary biologists ask are not amenable to meta-analysis. For instance, if you’re working on reconstructing the evolutionary history of phylogenetic group X, or the history of the evolution of trait Y within phylogenetic group X, you don’t have any reason to do a meta-analysis (do you?). What would a meta-analysis of papers on those topics even look like? But I dunno, there are also ecological questions that can’t be addressed with meta-analysis.
- Perhaps evolutionary biology is more narrowly focused on a smaller number of questions than ecology is. This might be my evolution envy talking. To my outsider’s eyes, evolutionary biology looks like a field with a unifying conceptual framework. Whereas to me, and to many of the non-random sample of ecologists who read this blog, the field of ecology looks like a disunified mess. If that unifying conceptual framework focuses evolutionary research on a relatively small number of questions (I mean, small relative to the total number of researchers), then you’re going to end up with fewer meta-analyses. In contrast, lots of ecological research has to do with associations between variables. Say, between some measure of biodiversity, and some other variable(s) that are thought to either affect, or be affected by, biodiversity. There are a lot of measures of biodiversity one could look at, and lots of other variables that might be associated with biodiversity! And that’s without even getting into how you could associate biodiversity with variable X at different spatio-temporal scales, in different taxonomic groups, etc. (Aside: yes, sometimes a single meta-analysis will address several of those questions using moderator variables. But other times, you’ll end up with multiple meta-analyses, each addressing a different question to do with how biodiversity relates to some other variable). Another big chunk of ecological research asks about interactions between variables–how the effect of A on B depends on C, D, E… If your field is really into picking apart interaction effects and multicausality, you are going to have reason to do lots of meta-analyses. Because there are lots of variables in the world whose effects might interact with those of other variables.
- Perhaps evolutionary biology is less applied than ecology. Many ecological meta-analyses (though far from all) concern applied issues, and are intended to offer guidance to land managers, government officials, and others trying to address those applied issues.
But I’m sure there are hypotheses I haven’t thought of.
It would probably make sense to consider more fields than just ecology vs. evolutionary biology, if only I knew anything about the prevalence of meta-analyses in other fields. I feel like researchers on human biology and behavior are really into meta-analysis? (many social science fields; medicine)
Looking forward to your comments, as always.
*It was 476, but a few were mistakenly entered twice so it’s down to 466.
**And of course, it’s an undercount of all ecological meta-analyses, for various reasons. We only looked for ecological meta-analyses in English, from journals indexed in Web of Science. We didn’t include meta-analyses that didn’t provide all the effect sizes, the sampling variances of the effect sizes, a citation or identifier for the study that originally published each effect size, and the year each effect size was originally published (or else data from which those numbers could be calculated). We didn’t include any meta-analysis for which more than 10% of the effect sizes had zero sampling variance. And we didn’t include any meta-analysis for which more than 10% of the effect sizes had no date of publication.