Trying to save a zombie idea (UPDATED)

In a previous post I commented that it would be interesting to see whether the very nice paper by Adler et al. (2011) on diversity-productivity relationships in terrestrial grasslands would finally kill off the zombie idea that diversity generally is a humped function of productivity. I also suggested that this zombie might be more of an apparent zombie than a real one, because hardly anyone these days still believes in this zombie idea, it’s just that everyone thinks everyone else believes in it.

I was wrong on both counts (you’d think I of all people wouldn’t be so quick to see a zombie idea as dead or dying!) Writing in Science this week, Pan et al. and Fridley et al. criticize Adler et al., and the reply is here. Pan et al. claim that, properly analyzed, the Adler et al. data show a strongly linear diversity-productivity relationship, while Fridley et al. claim both that the data are “clearly deficient” as a test for a humped diversity-productivity relationship, and that they show a “clear” humped diversity-productivity relationship*.

In my view, the comments are a striking illustration of just how far even very good ecologists will go in an attempt to save a pet hypothesis in which they are heavily invested. I should emphasize in saying that that I don’t have a dog in this fight. The question of how plant diversity varies as a function of primary productivity is a purely empirical question. Addressing it properly requires careful study design to sample the full range of natural variation, control for confounding factors, etc. And personally I don’t care what the answer is. I just care that we get the right answer.

I think Adler et al. is the right answer, for grasslands. Pan et al. don’t, but as Adler et al. point out their comment basically amounts to cherry-picking data to obtain a linear relationship. Not consciously cherry-picking of course. But as a rule, in any large and complex dataset there is always some subset of data showing a relationship between variables different than that shown by the dataset as a whole, and you can always find some more-or-less-plausible post-hoc reason to pay special attention to that subset. As Grace et al. point out in their reply to Pan et al., Fridley et al. actually prefer to focus on a different subset of the data. That strongly suggests that both sets of commenters are, unconsciously, just looking for reasons to make the data show what they think the data ought to show.

As noted above, the Fridley et al. comment is, on it’s face, self-contradictory*. Fridley et al. also try to change the question, arguing that “productivity”, or any index of it, should be defined to include leaf litter, which would be contrary to the bulk of previous empirical and theoretical studies. They also argue that Adler et al. should have conducted various alternative analyses…that Adler et al. actually did conduct and presented in their paper. And finally, as Grace et al. show, what little “humpiness” there is in the Adler et al. data reflects the log-normal distribution of the data, so it looks slightly humped on an arithmetic scale.

I’m not an expert on this stuff (I’m not a plant guy), but I’m confident I know enough about it to come to an informed (not infallible, but informed) opinion. And in my view, the Adler et al. paper is totally unscathed. In grasslands, I think we now have the best data we can ever expect on the diversity-productivity relationship–collected at a large number of sites around the world, with good coverage of a big productivity range, sampled using a consistent protocol. And those data have been very carefully and thoroughly analyzed. The answer is clear: the diversity-productivity relationship is very weak and noisy, and it’s not humped. So we’d better learn to deal with it.

Even if you want to say that the humped diversity-productivity relationship is so well-established that we need extraordinary evidence to reject it, well, I disagree with your premise, plus Adler et al. is extraordinary evidence.

At the end of their comment, Fridley et al. conclude with some quite striking rhetoric. They claim that the hump-backed diversity-productivity relationship is a “cornerstone of plant ecology”, backed by “decades of careful mechanistic analysis”, and is “used by plant conservationists and restoration ecologists, as well as theoretical ecologists.” The first claim must surely be loose language on the part of Fridley et al., because taken at face value it’s patently false. Discarding the humped diversity-productivity hypothesis would not cause plant ecology to collapse the way a building would if you removed its cornerstone. If you think otherwise, your view of what “plant ecology” consists of is way too narrow. The vast bulk of research on plant physiological, population, and community ecology has nothing to do with the diversity-productivity relationship, humped or otherwise. I’m not sure what the second claim means, but I hope they’re not claiming that we’ve proven mechanistically that the diversity-productivity relationship has to be humped. There’s nothing in ecology like, say, the kinetic theory of gases that let’s us rigorously derive a diversity-productivity equivalent of the ideal gas law. And their other claims are irrelevant. If conservationists, restorationists, and theoreticians based their work on the assumption that the earth was flat, that wouldn’t make the earth flat (and yes, it is possible for many experts to be wrong about a matter of empirical fact). Worse, do you really want to argue that conservation and restoration practices, or theoretical modelers, should never adjust what they do in light of new data? Please don’t misunderstand me: the authors of the Fridley et al. comment include many very good, very smart ecologists (and Jason Fridley for one is a friend of this blog). I just find it striking that so many very good, very smart ecologists could be so reluctant to reconsider their position on what ought to be a straightforward empirical matter, and so willing to resort to what looks like quite poorly-supported rhetoric. Basically their rhetoric at the end amounts to saying “this zombie idea has been around for a long time–so therefore it can’t possibly be a zombie”.

UPDATE: Commentary from one of the co-authors of Adler et al. here.

*Grace et al. also note that Fridley et al.’s comment is self-contradictory. I agree with that characterization.

19 thoughts on “Trying to save a zombie idea (UPDATED)

  1. The cosigners on the Fridley comment are certainly some big names in the subfield, but I was going to mention that if we could use the number of people making the statements as an indicator of the weight of belief, the original authors hold a slight majority. Maybe this was the ‘last stand’ of the hump-shaped diversity-productivity model (and now I’m imagining Peter Adler and James Grace as Wyatt Earp and Doc Holliday…).

    Then again, I don’t know what it means when 1/3 of the original coauthors neglect to add their names to a response comment; did they just not reply in time, or did they intentionally not want their names on the response?

    • I actually don’t think measuring “weight of belief” is the way to go at all. But then, I would say that. But if you insist on looking for ways independent of the data to judge who’s right here, I don’t know that I’d look at number of signatures. I’d be more inclined to look at things like citation patterns–does one side mostly cite themselves for instance? I’d also look at things like which side has published more, and more prominently, on the topic. *Not* to determine who has more “expertise”, but to determine who’s invested a lot in a certain position that they’re therefore more likely to dig in and defend. Some of the authors of Fridley et al. are indeed big names–but they’re big names who’ve built their careers on the hump-backed diversity-productivity claim.

      That some of the original coauthors didn’t sign the reply almost certainly just indicates that they were too busy or didn’t care enough to bother, and trusted their colleagues to handle it. I’m sure it doesn’t mean that they don’t actually believe in the reply. But if you’re really interested, I know some of the authors of the reply and could ask them…

      As to whether this was the last stand of the hump-backed diversity-productivity claim, I don’t think zombie ideas really have last stands. They just fade away, as their proponents retire. Which takes a loooonnnnngggggg time, because the proponents of any idea will have students and close colleagues who will tend to become proponents themselves, who will have their own students and close colleagues…only when the “R0” of the zombies is <1 can we say that the zombie outbreak is starting to fade. Not sure if we're there yet for this zombie.

      • Right, to determine which side is closer to the truth we need to consider the vested interests of both sides and whether or not they make their arguments with clear, empirical statements. That part is pretty settled for me (I agree that “the diversity-productivity relationship is very weak and noisy, and it’s not humped”).

        I know very little about invasion analysis but this seems like a good application of it (or at least a good opportunity for me to learn some about it). If the R0 of one trait/idea is greater than the other, would it matter if the R0 of the other is less than 1? Would the new idea still ‘win’ even if the zombie idea was still around?

      • If the R0 of one trait/idea is greater than the other, would it matter if the R0 of the other is less than 1? Would the new idea still ‘win’ even if the zombie idea was still around?

        Depending on the particular parameter values, we might get coexistence between zomA’s and zomB’s

      • Mike: If I understand correctly, you’re asking about the “relative fitness” of alternative ideas rather than the absolute fitness of a single idea. If we’re thinking of relative fitness, then the idea with the highest relative fitness will win in the sense that it will replace the alternatives and “sweep to fixation” in the “population” of ideas. With the caveat that, if the absolute fitness of every competing idea is less than 1, the winning idea itself will eventually die out (e.g., everybody stops caring about the diversity-productivity relationship, and so nobody any longer has any ideas about the form of the relationship).

        To get stable coexistence of competing ideas, their relative fitnesses would have to be frequency-dependent for some reason. It’s hard to see why that would be the case.

  2. Aaron, as you guessed, the reason not every original author signed is that many did not respond in time. Science requires authors to fill out quite a few forms, and the deadlines on the Comments/Replies are tight. No one asked to have their name removed.

  3. Pingback: Yes, IDH zombies are still worth worrying about « Oikos Blog

  4. Pingback: A key problem in interpreting observational data « Oikos Blog

  5. Pingback: ESA 2012 preview: all the good talks are on Friday | Dynamic Ecology

  6. Pingback: Peter Abrams: suggested readings | Dynamic Ecology

  7. Pingback: Zombie ideas in ecology: the TREE paper is now online | Dynamic Ecology

  8. Pingback: Want to bet? | Dynamic Ecology

  9. Pingback: Major screw-up in economics, and what it means for ecology | Dynamic Ecology

  10. Pingback: Zombie ideas in ecology: the local-regional richness relationship | Dynamic Ecology

  11. Pingback: Pseudo-Humps (and my first play with Shiny) | Andrew Letten

  12. Pingback: When, if ever, is it ok for a paper to gloss over or ignore criticisms of the authors’ approach? | Dynamic Ecology

  13. Pingback: On progress in ecology | Dynamic Ecology

  14. Pingback: Should your paper include alternative analyses as “robustness checks”? If so, where? Take a quick poll! | Dynamic Ecology

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.