I’ve been writing a lot lately about just how variable ecological effect sizes are (e.g., here, here, here). Different studies of the same effect often report very different effect sizes, and not just because of sampling error. That’s why most meta-analyses in ecology these days incorporate moderator variables–covariates that may explain some of that massive variation in effect size.
As I’ve written before, my anecdotal impression is that moderator variables rarely explain much variation in effect size in ecological meta-analyses. Even the moderator variables that explain a statistically significant fraction of the variation in effect size rarely explain all that much variation in an absolute sense.
But maybe there are some exceptions? I mean, ecologists have now done hundreds of meta-analyses, many of which included moderator variables. Surely, a few of them found moderator variables that explained a lot of variation in effect size. Which are they? What are the most successful applications of moderator variables in the history of ecological meta-analyses?
I’ll go first. Based on extensive research*, I’ll nominate two moderator variables from the Borer et al. (2005) meta-analysis of trophic cascades. Judging from inspection of Fig. 4, it looks to me like “study duration measured in plant generations” and “terrestrial vs. aquatic habitat” together explain a fair bit of variation in mean trophic cascade strength.
My other nominee is from evolutionary ecology. It looks like “plant population size” explains a fair bit of variation in the mean strength of local adaptation in plant populations (Leimu & Fischer 2008, Fig. 2a). Larger plant populations exhibit appreciably stronger local adaptation on average than small ones do.
At the risk of massively overgeneralizing from three moderator variables, it’s interesting to me that, for all three of those moderator variables, there were good theoretical and/or empirical reasons to expect those moderator variables to have big effects. Many community ecologists had long thought that trophic cascades were stronger in aquatic systems than in terrestrial systems. They thought that both for theoretical reasons, and because even before any meta-analyses had been done, they’d noticed that trophic cascades often seemed to be strong in experiments conducted in aquatic systems. And if you want to observe a big effect of trophic cascades on plants, obviously you have to run your study for enough plant generations for a big effect to show up. And if you were asked to list the variables that govern whether evolution by natural selection will lead to adaptation, population size would be right at the top of the list. You need a big population size in order to count on selection overcoming genetic drift. In contrast, I feel like in most meta-analyses, there’s not such a strong reason to expect the chosen moderator variables to have a big effect (there’s often a vague, hand-wavy reason…).
But perhaps my casual impression here is too driven by a few examples I happen to remember. So you tell me–what are the most successful applications of moderator variables in the history of ecological meta-analysis? Looking forward to learning from your comments.
*”extensive research” being operationally defined as “I consulted my memory for 15 seconds”.