Note from Jeremy: this is a guest post from the one and only Mark Vellend.
The thing you study is underappreciated. Maybe it’s facilitation, or parasites, or time lags, or precipitation, or nitrogen. We’ve all written papers arguing that the thing we study is more important than people think. But what does it mean to be “important”? This was one of many questions raised by the poll results – and ensuing discussion (see comment by Shan Kothari in particular) – about ecologists’ views on potentially controversial issues. Maybe “true” and “false” answers to the same question reflect, at least in part, different conceptions of how to assess importance.
Debates in ecology often focus on relative importance of different processes or factors: niche vs. neutral effects on community structure, stabilizing vs. equalizing processes underlying coexistence, local vs. regional processes in explaining diversity patterns. Many of these debates come with the added complexity (and confusion) that the things we measure (e.g., spatial position) map highly imperfectly onto the concepts we hope to capture (e.g., neutral stuff). There are also questions that seem answerable via an assessment of relative importance, but that actually are not. Those are debates for another day. To think about assessments of “importance” we can start much more simply, focusing on a situation where at least some quantitative comparison is clearly possible.
Scenario: two factors that influence one outcome (forget about interactions).
Y ~ X1 + X2
Productivity ~ Temperature + Nitrogen
How do we assess the relative importance of temperature and nitrogen in determining productivity? Intuitively is seems like we’d want to calculate [Change in productivity] / [Unit of temperature] and [Change in productivity] / [Unit of nitrogen], but this doesn’t work since the values are not in the same units. To do this quantitatively we need numbers in the same units. What to do?
Option 1: Manipulate nitrogen and temperature, or observe places that vary in nitrogen and temperature, and assess which has a bigger effect on productivity.
The main challenge here concerns the range of variation in X1 and X2. If I observe vegetation from the southern to the northern tip of Canada, temperature is almost certain to appear most important. If I observe a gradient of forests from sandy to clay-loam soils in one region, nutrients are likely to appear most important. These results are valid, but unsatisfying and hard to compare given their very narrow domains of application and profound dependency on sampling design. Same thing with experiments. Years ago I had fun debates with Roy Turkington about the relative merits of experimental and observational studies, with one specific question concerning the appropriate levels of experimental treatments. His approach (assuming I’m remembering right) was to aim to “relieve” the system of a given limitation. So, if it’s herbivory, you eliminate the herbivores. If it’s nutrients, you figure out how much you need to add so that nutrients cease to be a limiting factor. This is like a presence-absence variant of option 1.
Option 2: Assess productivity changes over ranges of nitrogen and temperature that represent realistic past or future changes to the system under study.
This is what a global change scientist would probably want to do. In one sense, it is like a variant of option 1 in which the ranges of variation in X1 and X2 are chosen strategically. I consider it separately because with a study including broad ranges of X1 and X2, the two options can be implemented simultaneously, and most importantly, they can lead to opposite conclusions.
In the following hypothetical example, the overall experimental (or region-wide observational) effect of nitrogen is greater than the effect of temperature (assume y-axes on the same scale), as shown by comparing the red curly brackets on the right. In the model on the full data asset (P ~ N + T), N comes out the best predictor, and so we say N is a more “important” determinant of productivity. But in the hypothetical region where the study was conducted, there is very little expected change in nitrogen, but a large expected increase in temperature. So if we calculate realistic future changes in productivity due to N or T, this time temperature comes out looking more important, as shown by the blue curly brackets on the right.
Which perspective is the right one? Well, probably neither. As with so many issues in ecology, it depends on the question, which is another way of saying that the initial question (“which of N or temperature is the more important determinant of productivity?”) wasn’t specific enough to even really be answerable in a general way. If you want to predict productivity at a random location in the study area (assuming the data were observational), what’s the first thing you want to know? Nitrogen. What variable is most likely to cause future changes to productivity? Temperature. My sense is that we have far more studies of the first kind, which could mislead us to focus on the wrong variables if we’re interested in predicting future changes to a system.
In one sense it is probably “progress” when we convert a black vs. white question to a question of relative importance. But the next step is probably often a need to assess the importance of variable X3 in determining the relative importance of X1 and X2, and so on. And at some point we get confused or bored and try to find a new black vs. white question that we can study for a while before it gets converted to a meta-meta-question. Or maybe there are far better ways to assess relative importance so that debates can be resolved in ways that satisfy most people. What do you think?