Why ecologists should refight the 'null model wars' (UPDATED)

Sometimes, scientific debates get resolved in favor of one side or the other. Modern birds are descended from dinosaurs, and those who thought otherwise were incorrect. Sometimes, debates get resolved in favor of some intermediate or synthetic position. The neutralist-selectionist debate in evolution has given way to a sophisticated appreciation of the importance of both processes. Something similar might be said about the density-dependence vs. -independence debate in population ecology. Clements and Gleason disagreed as to whether ecological communities are highly integrated ‘superorganisms’ or nearly random assemblages of ‘individualistic’ species, but the modern view sees interacting species as highly non-independent even though interspecific interactions don’t produce anything like a Clementsian superorganism. And sometimes, debates just stop because the original question is no longer relevant, perhaps because it was ill-posed in the first place (think of alchemical debates).

But sometimes, debates remain relevant, but don’t get resolved. Instead, we just stop talking about them. Sometimes this is because everyone involved recognizes that the available data and analytical methods aren’t sufficient to settle the issue. But sometimes, relevant debates just stop simply because neither side has anything new to say. The ‘SLOSS’ debate, over whether it is better to have a single large nature reserve, or several small ones of equal total area, is an example. As a graduate student, I recall David Ehrenfeld, the first EiC of Conservation Biology, telling a class that he’d decided to stop publishing papers on the SLOSS debate because no one had anything new to say about it. The debate between frequentist and Bayesian statisticians, especially concerning the proper interpretation of ‘probability’, seems like another possible example. Certainly, Brian Dennis’ passionate argument for the relevance of this debate to ecologists (and in favor of the frequentist position) seems not to have sparked an ongoing public discussion in ecology as far I can tell.

Selective journals naturally don’t want to publish repetitive papers. More broadly, scientists (especially those not directly involved in a debate) get tired of seeing the same old arguments rehashed. We tend to see these kinds of debates as pointless, as an indication that a fruitless dead end has been reached and its time to direct our energies elsewhere. But I’m not so sure this is the right response. Just because a debate can’t be resolved (or can’t be resolved by data) doesn’t mean it ought to be ignored. Debates between liberal and conservative political views have been going on for centuries, but that doesn’t mean they can or should just be abandoned. The issues are too important, and too unavoidable, for that. Analogously, I think there are some arguments in science that we need to keep having, not because we hope to resolve them, but because they are arguments about important issues on which every one of us ought to have a thoughtful, considered view.

So what irresolvable old debates should we keep having? In an attempt to provoke some comments from my fellow Oikos editor Nick Gotelli, I suggest that ecologists need to refight the ‘null model wars’ (That high-pitched sound you hear is a collective scream of anguish from every community ecologist who was active in the late ‘70s and early ‘80s) For those of you who aren’t aware, in 1975 Jared Diamond (yes, the Jared Diamond) published a paper on the distribution of bird species on islands in Papua New Guinea. Diamond argued that species were distributed so as to obey ‘assembly rules’ generated by interspecific competition (e.g., some ‘forbidden’ combinations of species never coexist on the same island). Connor and Simberloff (1979) sharply criticized Diamond’s proposed rules, arguing that some were trivial tautologies, and others were actually consistent with what would be expected under a null model in which species were distributed randomly. This touched off years of vociferous debate on a range of interrelated issues, from the appropriate choice of null model, to the value of hypothesis testing as a research approach, to the relative advantages of observational vs. experimental data (see, e.g., the famous Nov. 1983 issue of American Naturalist, the edited volumes by Strong et al. 1984 and Diamond and Case 1986, and numerous Oikos papers such as this one). In the aftermath of this debate many community ecologists refocused on small-scale field experiments as a rigorous way to test for competition. More recently, interest in using null models to infer the causes of observed patterns in species distributions has revived, thanks largely to Nick’s work. Null models are now being applied to new problems, including species richness gradients (the ‘mid-domain effect’) and phylogenetic community ecology.

But this revival in interest in null models seems not to have been accompanied by revived interest in the very serious conceptual issues which were debated, and never fully resolved, during the null model wars. In all null model work, the goal is to compare observed data, which were presumably generated by the combined action of various processes, to data generated by a null model which omits the effect of some process of interest but retain the effects of the other processes. Obviously, the choice of null model is absolutely crucial here. You will be seriously misled if your null model inadvertently retains some effects of the process of interest (the ‘Narcissus effect’; Colwell and Winkler 1984), and/or removes effects of other processes. In light of that, here are some questions that I think we ought to be (re-)debating:

1. How do we identify the appropriate null expectation? Most recent work assumes that appropriate null expectations can be generated simply by randomizing the observed data, typically in some constrained way. Nick Gotelli has done fine work testing the ability of different randomization methods, choices of constraints, and test statistics to recover known patterns in artificial data. But that work doesn’t address the deeper issue of what patterns we should expect our data to exhibit in the absence of the process of interest. For instance, in a spatiotemporally heterogeneous world, in which species have finite dispersal abilities, do we really expect species to be distributed randomly with respect to one another even in the absence of competition?

2. How do we generate data conforming to our null expectation? Again, most recent work assumes that constrained randomization of the observed data is the way to go. But is it? For instance, if we want to detect effects of interspecific competition, it is not at all obvious that the standard sorts of constraints on randomization do a good job of removing all effects of competition (no one’s ever entirely solved the Narcissus effect), or of capturing the effects of all the non-competitive processes. Maybe it would be better to develop an explicit mechanistic model of the processes we believe generated the data, and then use that model to generate expected data when the process of interest is set to zero. For instance, this is basically how models of neutral drift and migration are used to generate null expectations in population genetics and macroecology, thereby (hopefully) allowing effects of selection to be detected. Here’s a paper of Nick’s that touches on this issue.

3. What do we do if different processes have highly non-additive effects, so that the effects of removing a given process are highly context-dependent? In particular, what do we do if an observed pattern is ‘overdetermined’, so that removal of any one process doesn’t change the pattern? Is a hypothesis-testing approach even the right way to go in these kinds of situations? (not that alternative approaches are any picnic in such situations either…)

4.  In cases where questions 1-3 don’t have fully-satisfactory answers, what do we do? Do we just forge ahead with admittedly-imperfect methods on the grounds that they’re the best available? Do we abandon null model approaches entirely and focus on alternative approaches like field experiments? Do we find creative ways to combine different, complementary lines of evidence, each of which compensates for the limitations of the others? I suspect this is the really irresolvable, eternal question, the one about the relative strengths and weaknesses of different research approaches and the best ways to combine them.

I do mean these questions as real, honest questions, not rhetorical ones. Hopefully folks will chime in with some responses in the comments. Certainly, these questions are being discussed–in particular cases. For instance, Connolly (2005) argues that the mid-domain effect is really a Narcissus effect, which largely vanishes when an appropriate mechanistic model is used to generate the null expectation (UPDATE: McClain et al. (2007) is another nice paper discussing these questions in the context of the mid-domain effect). Recent attempts to use randomized null models to detect effects of competition on species distributions have used increasingly large datasets and increasingly complex constraints in an attempt to address some of the issues I’ve raised. But there hasn’t been any broader-based debate the way there was in the late 70s and early 80s. I think that’s a shame, especially for those too young to have participated (or maybe even heard of!) the first null model wars. Indeed, I’m too young myself to have experienced the first null model wars—but I hope I get to experience NMW II.

p.s. Just so no one gets the wrong idea and thinks I’m throwing stones from inside a glass house, I happily agree that similarly challenging questions can be, and should be, asked about any research approach. I’ve certainly been asked hard questions about my own research approach (laboratory microcosms), which is absolutely fair. Indeed, my own reasons for doing what I do have shifted over the years in response to such questions, which I plan to talk about in a future post at some point. I appreciate being obliged to think hard about why I do what I do, and I hope the questions I’ve posed in this post will be taken in the same spirit.