Ecologists often want to study the relative importance or strength of different variables or factors. Which is stronger: top down or bottom up effects? Where does community X fall on a continuum from “drift dominated” to “niche dominated”? What’s the relative importance of density-dependent vs. density-independent factors in explaining temporal variance in species’ abundances? What’s the relative importance of ecological vs. evolutionary determinants of species’ range limits? Etc.
Questions about the relative importance of different variables or factors often are very sensible questions to ask in a multicausal world (see also). But if you’re not careful, it’s very easy to ask what seems like a sensible question, but actually makes no sense at all. In general, just because dependent variable Y is affected by more than one thing does not mean that it makes sense to ask about the relative strength or importance of those things!
Here are some common pitfalls in studies of the relative importance or strength of different variables in ecology:
- Arbitrary or contestable definition of “importance”, when the answer is sensitive to one’s choice of definition. See Rees 2013 Eco Letts for a great critique of one instance of this problem. More broadly, see this old post and the excellent comment thread for discussion of different measures or indices of the “same” thing (here, the “strength” or “importance” of some variable).
- Asking where natural systems fall along some purported continuum that is not actually a continuum. See: all attempts to treat “niches” and “drift” as the two ends of a continuum. That’s like trying to treat loud sounds and high pitched sounds as the two ends of a continuum. There’s a continuum from neutrality to selection, and from drift to, um, lack of drift. But there’s no continuum from drift to selection.
- Asking about the relative importance of things that do not combine additively. Rather, they combine in some non-additive way, so there’s an interaction term. In which case the effect of any given variable (either in an absolute sense, or relative to the effects of the others) depends on the values of the other variables. For instance, the strength of top down effects depends on the strength of bottom up effects: the change plant biomass following top predator removal depends on resource enrichment. (Aside: in general, asking whether variables A and B have additive effects is a bad scientific question if you define “additive” statistically. Whether or not there’s a statistical interaction term between two variables is very sensitive to your data transformation and other factors having nothing to do with how the world actually is. Statistics is a tool to help you answer your scientific questions, it shouldn’t define your scientific questions. See #6 in this post.)
- Asking about the relative importance of different things that, when combined, yield “emergent” effects. Example: in population dynamics, the combination of deterministic and stochastic forces often yields surprising emergent outcomes. Think of quasi-cycles: regular or nearly regular population cycles that emerge when a population that would exhibit damped oscillations to a stable equilibrium in the absence of stochasticity is subjected to ongoing, non-cyclic stochasticity. Asking about the relative importance of determinism vs. stochasticity in generating quasi-cycles is like asking about the relative importance of sugar, flour, mixing time, and oven temperature to the baking of a cake. It doesn’t make sense. And any attempt to try and force it to make sense is going to be highly artificial and unhelpful. Much the same could be said about attempts to partition the relative strength or importance of determinism and stochasticity in many other contexts in ecology, such as certain approaches to metacommunity dynamics.
Dynamic Ecology 
Are we going to get a “How TO study the relative importance…” post? Been grappling with that one…
Maybe. This post is just quickie filler–it’s an edited version of a comment I made on an old post. It’s a lot easier to write posts you’ve already written!
Got any thoughts you want to share, since you’ve been grappling with this?
Tough nut to crack. Need to grapple some more before having anything useful to put down in writing…
Hi Jeremy, I agree that in situations where state C is only possible if both A and B are true (emergent properties) then talking about the ‘relative importance’ of A and B doesn’t seem very useful. For the first three contexts I’m not as convinced. On the first, importance is almost always going to be possible to define in multiple ways – as long as you’re clear about how you’re defining it, other people can decide if they agree that’s it’s a reasonable dimension of ‘importance’. I would be shocked if we could ever find a single definition of ‘importance’ that all ecologists would agree on. On the second, I don’t see a continuum as being necessary to investigate which of two drivers is more important – volume and frequency may not be on a continuum but I think it would still be reasonable to ask “Is volume or frequency more important?” On the third, interactions don’t necessarily imply that one driver isn’t always more important than another – it just implies that the difference in importance between two drivers isn’t constant. So, it could still be reasonable to conclude one driver is more important than another even if an interaction occurs. And, of course, the interaction itself is interesting – how the relative importance varies at different levels of the two drivers.
As for how to measure relative importance – not surprisingly I’m going to see predictive ability as one potential arbiter. If A and B both have an effect on C but knowing A allows me to make much better predictions than knowing B then I would say that A is more important than B. I suspect there are many dimensions to ‘important’ but that seems like one reasonable one. Best, Jeff H
“On the first, importance is almost always going to be possible to define in multiple ways – as long as you’re clear about how you’re defining it, other people can decide if they agree that’s it’s a reasonable dimension of ‘importance’. I would be shocked if we could ever find a single definition of ‘importance’ that all ecologists would agree on.”
I agree. But the problem comes in when people think it’s worth arguing about the “right” definition anyway, leading to endless sterile debates. See that Rees 2013 paper for an example. I think the point generalizes to any situation in which people are trying to operationalize some vaguely-defined verbal concept: How do you make research progress on a topic when different ways of defining key terms lead to different conclusions, but there’s no decisive consideration favoring any particular definition over any of the others? See, e.g., endless debates over the “right” way to define and partition alpha and beta diversity.
Thinking about it just now, part of the problem in these endless debates is that they aren’t *totally* pointless. It’s not like an argument about whether chocolate ice cream is better than vanilla, or red is a better color than blue–choices that are obviously just a matter of personal preference and that it would obviously be silly to argue about. Rather, there are some rational considerations one can appeal to. For instance, any sensible alpha diversity measure should say that a local site with two species of equal abundance has higher alpha diversity than a site with just one species. The trouble sets in when those rational considerations aren’t completely decisive, or when different rational considerations argue for different conclusions and there’s no agreed way to weight those different considerations. So you have a situation where it sure *seems* like there’s a worthwhile rational debate to be had–but yet there’s no way for that debate to reach a resolution.
“On the second, I don’t see a continuum as being necessary to investigate which of two drivers is more important – volume and frequency may not be on a continuum but I think it would still be reasonable to ask “Is volume or frequency more important?” ”
This is a good point I now wish I’d talked about in the post, or talked about more clearly. There are different ways of framing questions about relative importance, and sometimes one framing can make perfect sense while another framing makes no sense. For instance, it makes perfect sense to ask how much some response variable of interest changes in response to a change in the strength of selection, and to compare that to how much that same response variable changes in response to a change in population size (thereby changing the strength of drift). (You’d of course probably want to standardize everything so that your answer wasn’t dependent on the different units in which selection and population size are measured; this is the idea behind elasticity analysis.) But another way to ask about relative importance is to ask where some variable or system falls on some purported continuum. Which makes no sense when that purported continuum isn’t actually a continuum (e.g., there’s really a 2-D space of possibilities with four corners, but without realizing it you’re trying to collapse it to a 1-D space with two endpoints).
Pingback: How to assess “importance”? | Dynamic Ecology