Did ecologists cherry-pick all the big effects years ago, so that now we’re stuck studying small effects?

No. From my fairly comprehensive database of 476 ecological meta-analyses, here’s a graph of weighted mean effect size vs. meta-analysis publication year:

This graph is very crude. It includes meta-analyses based on different effect size measures. And it’s not showing you error bars around those means (as you’d expect, the means that are farther from zero tend to have bigger error bars). But better graphs wouldn’t change the basic picture: there’s no trend over time. It’s not the case that ecologists decades ago discovered all the big effects, so that now we’ve all moved on to studying smaller effects.

This result doesn’t surprise me, and I’m guessing it doesn’t surprise most of you either. There’s an infinity of effects that ecologists could study. The supply of big effects is not some exhaustible finite resource. And ecologists’ choices about what effects to study don’t have much to do with how big those effects are on average.

7 thoughts on “Did ecologists cherry-pick all the big effects years ago, so that now we’re stuck studying small effects?

  1. “And ecologists’ choices about what effects to study don’t have much to do with how big those effects are on average.”

    Really? I’d rather hope that the opposite is true. That we intentionally go after the biggest influences in our system first. Pick off the biggest explanatory variables and high r2 first and then go after lesser effects later in some systematic fashion. The first time I work in a new system my question is what explains most of the variance. Indeed its an organizing theme in the book I’m writing. But dunno, maybe I’m strange.

    • “The first time I work in a new system my question is what explains most of the variance. Indeed its an organizing theme in the book I’m writing. But dunno, maybe I’m strange.”

      I think you’re strange. 🙂 But maybe we need a poll on this.

  2. Not sure whether you would regard this as an ecological or evolutionary study, and we did not tabulate effect sizes – though they can be calculated from the results presented. Just out: Givnish, T. J., R. Kriebel, J. G. Zaborsky, J. P. Rose, D. M. Waller, K. M. Cameron, and K. J. Sytsma. 2020. Adaptive associations among life history, flowering, and mating system traits in native and introduced angiosperm floras. American Journal of Botany 107: 1677-1692.
    This paper tests several hypotheses using the native and introduced floras of Wisconsin, and employing phylogenetic corrections. It’s the first extensive study using that approach to test several of Baker’s hypotheses re introduced plants.

    • Thanks Thomas, but I stopped adding to my compilation a few months ago. It’s becoming a less-comprehensive compilation by the week, unfortunately! 🙂 But I needed to stop adding to the compilation so that I could focus on writing up some analyses of the compilation for publication.

  3. To which: 15,000 effect sizes? Meh. My compilation has over 114,000! 🙂

    (I’m joking, of course. Not criticizing the project to which Jeff Ollerton linked. I know very little about that project–only skimmed it very briefly. The goals are different than for my project, so it’s probably fine that it “only” has 15,000 effect sizes.)

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.