As regular readers will know, my side project over the last 18 months has been a meta-analysis of ecological meta-analyses. With the assistance of a very capable undergraduate assistant, I’ve compiled a fairly comprehensive database of 466 ecological meta-analyses.* That’s a lot of meta-analyses!**
Specifically, it’s a lot of meta-analyses compared to how many there are in evolutionary biology. Last winter, an undergraduate independent study student of mine decided to compile meta-analyses from evolutionary biology. She only found 33. Now, she had only limited time to complete her project, so she may well not have found all meta-analyses that would’ve met the inclusion criteria. But still, I don’t think she overlooked several hundred meta-analyses that would’ve met her inclusion criteria. And FWIW (not much…), her data support my offhand impression that there are many fewer meta-analyses in evolutionary biology than in ecology. It’s definitely possible I’m wrong about this, and you think I’m wrong you should definitely say so in the comments.
But assuming for the sake of argument that I’m not wrong, why are there so many more ecological than evolutionary meta-analyses? Because it’s a bit strange, when you think about how central the comparative method is to evolutionary biology. Shouldn’t that emphasis on the value of comparative studies translate into doing lots of meta-analyses like this one? Here are some speculative hypotheses as to why there are so many more ecological than evolutionary meta-analyses. They’re not mutually exclusive hypotheses.
- Evolutionary biology is a smaller field than ecology, so there are just fewer evolution papers, period. There’s some evidence consistent with this possibility. For instance, the annual Evolution meeting is much smaller than the annual ESA meeting.
- Perhaps many of the questions evolutionary biologists ask are not amenable to meta-analysis. For instance, if you’re working on reconstructing the evolutionary history of phylogenetic group X, or the history of the evolution of trait Y within phylogenetic group X, you don’t have any reason to do a meta-analysis (do you?). What would a meta-analysis of papers on those topics even look like? But I dunno, there are also ecological questions that can’t be addressed with meta-analysis.
- Perhaps evolutionary biology is more narrowly focused on a smaller number of questions than ecology is. This might be my evolution envy talking. To my outsider’s eyes, evolutionary biology looks like a field with a unifying conceptual framework. Whereas to me, and to many of the non-random sample of ecologists who read this blog, the field of ecology looks like a disunified mess. If that unifying conceptual framework focuses evolutionary research on a relatively small number of questions (I mean, small relative to the total number of researchers), then you’re going to end up with fewer meta-analyses. In contrast, lots of ecological research has to do with associations between variables. Say, between some measure of biodiversity, and some other variable(s) that are thought to either affect, or be affected by, biodiversity. There are a lot of measures of biodiversity one could look at, and lots of other variables that might be associated with biodiversity! And that’s without even getting into how you could associate biodiversity with variable X at different spatio-temporal scales, in different taxonomic groups, etc. (Aside: yes, sometimes a single meta-analysis will address several of those questions using moderator variables. But other times, you’ll end up with multiple meta-analyses, each addressing a different question to do with how biodiversity relates to some other variable). Another big chunk of ecological research asks about interactions between variables–how the effect of A on B depends on C, D, E… If your field is really into picking apart interaction effects and multicausality, you are going to have reason to do lots of meta-analyses. Because there are lots of variables in the world whose effects might interact with those of other variables.
- Perhaps evolutionary biology is less applied than ecology. Many ecological meta-analyses (though far from all) concern applied issues, and are intended to offer guidance to land managers, government officials, and others trying to address those applied issues.
But I’m sure there are hypotheses I haven’t thought of.
It would probably make sense to consider more fields than just ecology vs. evolutionary biology, if only I knew anything about the prevalence of meta-analyses in other fields. I feel like researchers on human biology and behavior are really into meta-analysis? (many social science fields; medicine)
Looking forward to your comments, as always.
*It was 476, but a few were mistakenly entered twice so it’s down to 466.
**And of course, it’s an undercount of all ecological meta-analyses, for various reasons. We only looked for ecological meta-analyses in English, from journals indexed in Web of Science. We didn’t include meta-analyses that didn’t provide all the effect sizes, the sampling variances of the effect sizes, a citation or identifier for the study that originally published each effect size, and the year each effect size was originally published (or else data from which those numbers could be calculated). We didn’t include any meta-analysis for which more than 10% of the effect sizes had zero sampling variance. And we didn’t include any meta-analysis for which more than 10% of the effect sizes had no date of publication.
Dynamic Ecology 
A very interesting question Jeremy, but I do wonder about how it’s framed. I’m sure that some (many?) of the meta-analyses in your ‘ecology’ database are just as much ‘evolutionary’ as they are ‘ecological’, using phylogenetic comparative methods in their analyses, and asking questions about natural and sexual selection. I know that there’s at least one* in your ecology database that I would consider as much a study of evolution as of ecology. Of course, it’s about evolution that’s happening _now_ (i.e. patterns of natural selection that we can perceive in the field) as opposed to evolution that happened in the past. Nonetheless I would see it as falling in the area between these two categories, and indeed linking them.
*Munguía-Rosas, M.A. Ollerton, J. Parra-Tabla, V. & Arturo De-Nova, J. (2011) Meta-analysis of phenotypic selection on flowering phenology suggests that early flowering plants are favoured. Ecology Letters 14: 511-521
Yep – came here to say something similar. I’d imagine a few behavioural ecology studies/meta-analyses could easily be re-labelled “evolutionary ecology”.
How did you/the students distinguish/define the two options, Jeremy?
See my reply to Jeff. Meta-analyses of topics on the interface between ecology and evolution (or ecology and other fields) aren’t nearly common enough to explain the gap here. It’s not that there are hundreds of meta-analyses on topics at the interface of ecology and evolution that all got classified as ecological.
Yes, you’re right that there are a few meta-analyses in the ecological database that could just as well be put in the evolutionary database. But only a few.
Part of what prompted this post was a convo I saw on Twitter a while back, by some neuroscientists puzzling over why there aren’t many meta-analyses in neuroscience. One hypothesis was that neuroscientists know that slight methodological differences among studies can have big effects on the results. So they’re reluctant to lump together different studies into one meta-analysis. Which, if so, is very interesting, because that definitely does not stop ecologists from doing meta-analyses!
Not saying that ecologists are right (or wrong) to do meta-analyses in the face of substantial methodological heterogeneity. I just find it interesting how in different fields there can be different attitudes about the same statistical/methodological issues.
I think typical meta-analyses ask “what’s the effect of X on Y” type of questions. In ecology, an inherent high degree of context-dependancy gives us many Xs and many Ys. In evolution, even if you have many Ys, there’re only a handful of Xs.
At least, most of the Xs that one can think of are ultimately derived from a parental_X (genotype, phylogeny, and a few more things).
So, there are many more Y|X problems to look at in ecology, than there are in evolution.
One can split an evolutionary meta-question into several ecology questions. e.g., Jeff mentions a “parent” question — phenotypic plasticity in phenology. This can be broken down into multiple “offspring” questions: what is the effect of X on plasticity~phenology. Here X can be any biotic or abiotic variable, such as temperature, precipitation, something in the soil, something about pollinators or about pests, etc.
I agree with this, except that I think that the comparative lack of Xs in evolution is because evolutionary biology has a unifying theoretical framework. That is, I think ecologists could cut down on their number of Xs, if they chose to do so, by developing a unifying theoretical framework. Well, probably more than one theoretical framework; I doubt there’s one framework that would cover all of ecology. Think for instance of Mark Vellend’s community ecology book, which attempts to cut down the number of Xs in community ecology by adopting the same theoretical framework as evolutionary biology. (book review here: https://dynamicecology.wordpress.com/2016/12/19/book-review-the-theory-of-ecological-communities-by-mark-vellend/)
Yes, agree.
There’s too many Xs, and there’s no signs of it slowing down. For instance, see this recent paper from earlier in the week. In my view, it can help readers to “find their own X” and frame a new question.
“A practical guide to question formation, systematic searching and study screening for literature reviews in ecology and evolution”
https://doi.org/10.1111/2041-210X.13654
Thanks for passing that on, hadn’t seen that paper.
I wonder if a few of the options mentioned are not, in a sense, one and the same. I’m thinking about the following: 1) evolutionary biology is smaller field. 2) evolutionary biology is more focused 3) evolutionary biology has more of a unifying theoretical framework.
I think most if not all the work associated with ecology and evolutionary biology (along with some other topics) can be roughly grouped into “biology of large scales/numbers”. Evolutionary biology appears to be a subset of this group, that is smaller than 50% of its topics/scope (option 2), and because of its more unifying theoretical framework (option 3) got its more clear distinction as a separate field. And given it’s smaller scope, has less papers (option 1). I am not sure this interpretation makes much sense, but I do think it’s difficult to set apart these 3 options.
Also, I think the repercussions of option 3 (unifying theory) is also not only about the issue of more focused questions that was mentioned. A unifying theory is related to the fact that things are more quantifiable — I would expect there are less disagreements about what to measure or about are the interpretation of results — and therefore meta-analyses are simply less necessary. To bring it to the extreme, how many meta-analyses are done in Physics? I don’t know the answer, but I suspect that far fewer.
I agree that it’s difficult to tell these options apart, in part because they can reinforce one another.
Interesting question as to whether there is less disagreement within evolutionary biology than ecology as to what to measure, or how to interpret results. Maybe there is. Or maybe not; perhaps ecologists *should* disagree with one another about these matters, but in practice they don’t. After all, if there’s no quantitative mathematical theory specifying exactly what you should measure, then maybe you don’t get disagreement about what to measure. If everyone’s trying to measure some vague, multifaceted concept like “biodiversity”, maybe everyone just picks different measures (all of which are equally plausible), and doesn’t get too bothered about others picking different measures. And if different measures lead to different results, people either aren’t aware of that fact, or else they are aware but just shrug it off. Because what are you gonna do? As This isn’t physics, it’s ecology.
I don’t endorse that attitude myself. And I’m actually not sure how many ecologists would endorse it. But I suspect that more than a few ecologists would endorse it.
Good point about the likely lack of meta-analyses in physics. I too bet there aren’t many. Arguably, meta-analysis is what you do when your field lacks both quantitative theory, and precise measurements of precisely-defined concepts.
Which if so, illustrates why one shouldn’t often expect moderator variables to explain all that much variation in ecological meta-analyses. If we could come up with a short list of moderator variables that would explain most of the variation in the effect of interest, we likely wouldn’t need meta-analysis. We’d likely have a quantitative theory of the effect of interest, and very precise measurements of the effect of interest.
NSF in the U.S. stopped funding the National Center for Ecological Analysis and Synthesis (NCEAS) almost ten years ago (sadly), but that started some time in the 90s and I would guess that it had a pretty substantial effect on the field of ecology generally because pretty much all funded working groups had a focus on meta-analysis (plus they had postdocs in their heyday). I have been involved in a few meta-analyses at this point, and can trace them all back to NCEAS meetings circa 2011. Maybe it didn’t make as much of a difference for the field as I think, but it made a big difference for me.
That’s a great hypothesis. I think that’s definitely part of it. But I’m not sure it’s the whole story. Because other fields, including evolutionary biology, subsequently copied NCEAS–but with much less success and influence. So I feel like NCEAS was both cause and consequence here. It pushed ecologists towards doing more meta-analyses–but it seems like something about ecology allowed that push to succeed. A similar push in evolution didn’t succeed to nearly the same extent, I don’t think.