The latest on diversity-productivity relationships: getting past a zombie idea

I’m a bit late to this (sorry, got busy then it slipped my mind), but better late than never. Writing in Nature, Grace et al. 2016 try to slay the zombie idea of a humped diversity-productivity relationship by integrating multiple theories into a single structural equation meta-model that they then fit to data. I don’t ordinarily comment on individual papers from the recent literature. But I’ve been following work on this topic so I thought I’d say a few things. I think there are some important larger lessons here for how to do good ecology.

First of all, “slay” isn’t really the right word for what Grace et al. are out to do. One can read their paper not so much as a test of the “hump-backed model” of diversity-productivity relationships, as a demonstration that the hump-backed model isn’t worth testing any more. That it’s no longer useful as a working hypothesis, as a motivation for empirical work, or as a guide to interpreting empirical work. Put bluntly: we should stop looking for humps in bivariate diversity-productivity relationships, because we’re not learning anything. We can get far more statistical explanatory power, develop a far richer empirical basis for future theory, and have far more informative tests of that future theory, by asking “what determines diversity and why?” than by asking “is diversity a humped function of productivity?”

I like any study in which people test ideas from all angles, including testing assumptions as well as predictions (see here, here, and here). That’s what the best ecology papers do. So I like Grace et al., though in some ways it’s not my cup of tea. I don’t think one can really “test” verbal or conceptual models, no matter how statistically sophisticated the test. And so I worry a little that structural equation modeling is ripe for abuse by investigators less careful than Grace et al., who might use it as a way to go straight from box and arrow diagrams to data analysis. Skipping the difficult but very powerful intermediate step of converting one’s verbal or diagrammatic models into proper process-based mathematical models. But your mileage may vary on that. Plus every approach to science has its own “failure modes”–the characteristic ways in which it tends to go wrong, in those cases where it goes wrong. In any case, whether you think of Grace et al. as a development and test of theory, or as a descriptive study giving future theory a target to shoot at (which is how I think of it), it seems to me like a big advance on previous work.

And if you think that moving to the sort of multicausal approach Grace et al. advocate means giving up on the search for generalizations in ecology, well, there are many roads to “generality” in ecology. Getting beyond univariate theories like the hump-backed model doesn’t amount to Balkanizing ecology into a collection of unique, unrelated, overwhelmingly contingent special cases.

One benefit of moving beyond the hump-backed model would be that nobody would ever think that they must’ve done something wrong if they fail to find a hump in their own data. Or have others think that they must’ve done something wrong. Angela Moles has talked about this in a different context. About how ecological research has been held back by widespread belief in a general pattern that just isn’t there, or at least is noisier and less general than many people think (see also). Belief so strong that many people simply disbelieve contrary data that absolutely should be believed. And if you reply to Angela’s point by saying “extraordinary claims demand extraordinary evidence”, well, ecologists are the last people who should ever see any claim as extraordinary. Ecology isn’t particle physics. Even the strongest and most general patterns in ecology take the form of statistical trends that only explain a modest fraction of the variation in the data. We shouldn’t be at all surprised if those trends fail to turn up in any given dataset. An ecologist who fails to find, say, a humped diversity-productivity relationship is not at all in the same position as a physicist who claims to have found particles that travel faster than light, and should not adopt or meet with the same skepticism of their data.

Grace et al. 2016 isn’t the only recent paper on the hump-backed model. Tredennick et al. 2016 critique statistical and interpretive mistakes in a recent high-profile paper claiming vindication of the “hump-backed model” from bivariate data. I don’t have much to say about that because I think in the grand scheme of things statistical issues aren’t the important issues here. We ecologists too often try to fix or paper over problems with our hypotheses by using better statistical techniques (or more data). I’ve talked about this in the context of the null model wars. Here, if a hypothesis is so vague that different investigators draw opposite conclusions about it from the same data, that suggests to me that the root problem is the hypothesis, not our statistics. “Do you see what I see?” is a starting point for endless arguments, not progressive science, and it’s not something you can fix statistically. So while I agree with Tredennick et al., I don’t think the right response to their points is “continue to look for humps in bivariate diversity-productivity relationships, but using some other statistical approach than we’ve used in the past.”

My sense, and hope, is that there’s a developing consensus that we need to get beyond the hump-backed model. Stop trying to prove or refute it. Just set it aside and do other things–ask related but different questions, develop new theories, etc. That’s what people on both sides of the debate about the hump-backed model have now called for. I’d add that, if that is indeed the direction the field goes, that we ought to stop citing and teaching the hump-backed model as well. Once the hump-backed model stops playing an active role in our science, there’s no reason to let it linger on as a ghost, as seems to have happened with the intermediate disturbance hypothesis. I’m increasingly of the view that our undergraduate teaching and our citation habits are too slow to change, continuing to give nods to ideas that were important in their day but that are now past the end of their natural lives. Ecology students who go on to grad school should not discover (to their surprise, I presume) that many of the big ideas they were taught as undergrads have been refuted or set aside. This isn’t to criticize those ideas or show disrespect to the ecologists who developed them, any more than the fact that no one uses allozymes any more is a critique of allozymes or disrespectful of Dick Lewontin. Science moves on, and should move on. And while it’s very useful for any scientist or science student to know something of their history of their field, you can’t make effective use of that history by treating it as a fixed point around which current work revolves or should revolve. Part of what Brian’s called a “problem solving mentality” is the willingness to completely abandon an idea.

If indeed the field does move on from the hump-backed model, it’ll be one more illustration that you influence the direction of the field not just by suggesting new approaches, but showing that those approaches work on real data. I doubt that my concern trolling about whether Grace et al. 2016 are really testing proper theoretical models is going to cut much ice with most ecologists, who are going to look at Grace et al.’s results and go “R^2 of 0.61! Wow!” 🙂

Which leads to a final, broader question, to which I don’t know the answer. In future, will papers like Grace et al. 2016 push ecology towards putting a premium on high R^2 values? To the point that any model (statistical or otherwise) that fails to achieve a high R^2 will be dismissed or devalued? There are times when we should worry about the combination of high enthusiasm + low R^2. But are there also times when we should worry about the combination of low enthusiasm and low R^2? This kind of gets back to my old post on small effects and when they’re worth studying. Conversely, there certainly are times when we’re too easily impressed with high R^2 (see here [scroll down to #5] and here). (I’m leaving to one side here worries that R^2 itself is a zombie idea.)

p.s. In case it needs saying, nothing in this post or previous posts on this topic is a personal criticism of anyone who has different scientific views than I do. Disagreement, even quite strong disagreement, is a normal part of science.