Drilling down vs. scaling up

Biological Posteriors asks a good question: how far down the [mechanistic] rabbit hole should one go to get an answer to any question? For instance, if you want to understand plant distributions, do you need to study plant physiology? Or even plant biochemistry?

Briefly, I’d say it depends on how you’ve framed the question, what sort of answer you’re looking for (e.g., a quantitative vs. a qualitative answer), and whether there’s anything comprehensible at the bottom of the rabbit hole.

But here I want to respond by asking a question of my own: why assume that you can only find the right mechanistic “level” by starting at a high level and then drilling down? Why not go the other way? Why not scale up? That is, start with a (possibly very detailed) “low level” mechanistic description of the physiology, life history, and behavior of individual organisms, and then ask about its higher level implications for density-dependence of population growth rate, coexistence, ecosystem function, etc.? There are lots of successful examples of this approach, indeed too many to list.

Note that this approach need not restrict you to building and simulating very computationally-intensive individual-based models. For instance, it may well be possible to derive a tractable analytical, high level approximation to your individual-based low level simulation. Importantly, that high level model, although simple, may well be different than the simple high level model you would’ve invented if you hadn’t first done the low level model and then scaled up. The work of Drew Purves, Steve Pacala and colleagues approximating the famous SORTIE model of forest dynamics is a fine example (Purves et al. 2008, Strigul et al. 2008).

So how do you decide whether to start high and drill down, or start low and scale up? Well, it’s often good to start at a level at which you already know, or can easily find out, a fair bit. In other words, don’t think about whether to drill down or scale up, think about starting from what you know and then working (upwards or downwards) towards something you don’t know.

It’s also worth noting that, if you don’t know how to drill down, you often won’t know how to scale up either, and vice-versa. This is something I wish a lot of macroecologists would take to heart. Macroecologists often argue that we don’t know how to scale up from individual- and population-level mechanisms to their macroecological consequences. Which is true enough. But they seem to take that as an argument for starting at the macroecological level and then drilling down. Which I confess I don’t understand. For instance, writing in the most recent issue of Oikos, Gotelli and Ulrich argue that we don’t know how to specify and parameterize system-specific process-based models of species interactions and dispersal.* But they present this as a reason to focus on null models that test for certain non-random patterns in presence-absence matrices (data matrices indicating which species are present at which sites). But if we don’t know how to build and parameterize low-level process-based models, why should we be at all confident in our ability to build high level null models that omit the effects of certain processes (such as interspecific competition)? Especially null models that putatively apply, not just to one specific system, but very generally? Because take my word for it, it is really easy to come up with very plausible low-level competition models in which competition generates presence-absence matrices that look nothing like those tested for by any of the standard null models. And conversely, it’s surprisingly difficult to come up with generally-applicable low-level process-based models that produce some of the high-level patterns that null models often test for (such as “checkerboard distributions”, where sites contain species A or species B, but never both). To be fair, I think Gotelli and Ulrich are aware of this issue, although they don’t put it quite this starkly. But I’m not sure even they have fully taken to heart the notion that, if we don’t how to scale up from microecology to macroecology, we don’t know how to drill down either.

*Grouchy aside: I also don’t understand why macroecologists harp on the purported impossibility of specifying and parameterizing low-level models for many species. First of all, as the example of SORTIE (and other examples) shows, it’s perfectly possible to build and parameterize very detailed process-based individual-level models of entire communities, or of dynamically-sufficient subsets of those communities. Second of all, why would anyone think that scaling from microecology to macroecology is totally impossible unless we have a fully-specified and parameterized model of the low-level microecological processes? For instance, you don’t need to build such a model to show experimentally that local communities are effectively closed to colonization (e.g., Shurin 2000). Which is all you need to show in order to refute the once-common macroecological claim that linear local-regional richness relationships imply that local communities are highly open to colonization. I guess I must be missing something here, because very smart macroecologists whose work I really respect keep emphasizing the claim that we can’t build and parameterize low-level process-based models of community dynamics. Which just seems like such an obvious straw man. Hopefully folks will weigh in in the comments on this and set me straight.

13 thoughts on “Drilling down vs. scaling up

  1. These are great ideas, Jeremy. I like your suggestion to start where you know and work towards something that you don’t know. It seems like that is how we can make unique discoveries – by drawing connections that others have not done before. It also makes the way we practice science flexible.

    To add to your suggestion, I would say that frequently the level at which “we can easily find out a fair bit” is at a micro level. Much of what we do as individual researchers (historically) is done at a single site or with a few populations.

    • Hmm…increasingly, the level at which we can easily find out a fair bit is the macro level. Much easier to these days to collect, compile and analyze data on macroecological patterns, thanks to remote sensing, digitization of range maps, etc. Hasn’t gotten much easier or faster to run field experiments.

  2. If that’s not an outstanding scientific question, then I don’t know what is.

    Unfortunately, can’t help you with an answer though, unless you appreciate the sign that says: “Answers to questions $1. Answers requiring thought, $10. Correct answers, $50” and have $1 on you. [Note that this is the updated version of the original sign, adjusted for inflation.]

  3. Also, should you happen to have a ten on you I could probably muster 1/5 of the correct answer, or you could settle for the popular ten pack…just want you to know your options.

  4. Seriously though, I have an answer, and it even required some thought!

    I think the answer is basically that when you model from the bottom up, which is to say, attempt to explain more complex phenomena with theories derived from simpler systems, you quickly run into chaotic-type situations where your predictions simply go right off the rails. That may be due to truly chaotic behavior, i.e. sensitivity to initial conditions, or something that resembles it but is just due to the system encountering new variables or states that you didn’t (maybe couldn’t) account for in your theorizing. Either way, the predictions fail terribly at some critical point, often abruptly, and it’s because your world view was limited in some important way when you theorized.

    When you go in the other direction, your theories are going to derive from observations (on your variables of interest) that have been inherently constrained by *all* of the actual factors operating on them. You thus have a *much* more realistically constrained view of where the higher level (i.e. more complex) system “ends up”. You can then start evaluating potential causative factors that caused it to get there, statistically, and start saying “aha, factor x is really only explaining about 20% of the observed variation here, not the 75% we would have expected based on first principle considerations (i.e. mechanistic theory). So what else is going on here, explaining the remaining 80%. Hmmm, well, let’s check out factors y and z”

    In other words, you’re less likely to go wildly off the rails catastrophically. However, you pay for it by wallowing around in a lot of confusion about what exactly is driving (and feeding back) on what as you go through a kind of process of elimination of possible causative explanations.

    In other words, the reason why weather forecasting is accurate only about 5 days ahead, and then goes directly down the tubes, versus climate modeling, which can reasonably predict certain meterorological variables over vastly larger scales of time and space.

    This is probably not what you are looking for but it’s going to cost you $10 nevertheless.

  5. Another way to phrase this might be that when you start from the bottom and try to go up, you can rather abruptly find that “We’re not in Kansas anymore Toto”, whereas when you start from the top and go down, you know you’re in Kansas, but damned if you’re sure what caused that tornado.

    There is no charge for this answer.

    • On the contrary, a good analogy is often better than a lengthy non-analogical explanation. I’ve sent a check for a bazillion dollars via passenger pigeon.

      • Excellent! I will use it buy a supercomputer so I predict community stability from the physics of subatomic particles, which I would otherwise have resolved long ago.

  6. Pingback: Survey on mathematical training for ecologists | Just Simple Enough: The Art of Mathematical Modelling

  7. Pingback: Ecology is mostly not like billiards (but lots of people think it is) « Oikos Blog

  8. Pingback: Scaling up is hard to do: a response | Dynamic Ecology

  9. Pingback: Steven Frank on how to explain biological patterns | Dynamic Ecology

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.