Note from Jeremy: this is a guest post, written by Karen Abbott after soliciting thoughts and discussion from Lauren Sullivan, Chris Stieha, Robin Snyder, Lauren Shoemaker, Sean Satterlee, Ben Nolting, Brent Mortensen, Chris Moore, Brett Melbourne, Brian Lerch, Geoff Legault, Aubrie James, Katie Dixon, and Sam Catella.
Karen adds: This was very much a group effort and these contributors (listed in reverse alphabetical order because regular alphabetical order feels a bit tyrannical when imposed by an Abbott) each made this post significantly more interesting than anything I would have come up with on my own – thanks to all.
Over the past five years or so, I have spent more time than I should probably admit feeling hung up about what stochasticity really means. When my thoughts start to fall down a rabbit hole of semantics or to philosophical questions of determinism versus free will, I pull myself back. I’m not interested in those things, important as they may be. I’m interested in gaining a deep understanding of the role that “stochasticity” — the conceptual construct — plays in ecological thinking, as well as the role that actual stochasticity plays in real ecological systems.
Lots of other people think about these things too (particularly the latter question on the role of actual stochasticity), so I asked a non-random group of colleagues, collaborators, and lab members to share their thoughts on what stochasticity means and where or how stochasticity is important. I’m not sure if this exercise has made me feel more or less hung up, but it was really fun and a number of interesting themes emerged:
1) Collectively, we have lots of ideas about what stochasticity is, and what it’s not. These ideas can be roughly organized by whether stochasticity contributes as a mechanistic driver of observable patterns or whether it exists outside of (deterministic) drivers to add variance to observations.
2) Questions of scale are ubiquitous.
3) Semantics aside, there are some meaningful differences in how ecologists apply the concept of stochasticity. “Stochasticity” is a rather precise-sounding, technical word that may hide a lack of conceptual precision.
Taking these one at a time —
1) What stochasticity is, and what it’s not:
There was universal consensus that stochasticity involves uncertainty, but a bit of divergence on the origin of that uncertainty. I tend to use the word “stochasticity” synonymously with “randomness” and “noise” and many contributors also defined stochasticity as randomness. Detractors on this point questioned whether anything in ecology is truly random. If we had perfect knowledge of all causal factors, would there ever be more than one possible outcome? An equal but opposite argument is: since we never can have perfect knowledge, is anything in ecology truly deterministic? There’s that pesky rabbit hole again. So, sidestepping that, we could say that stochasticity is anything with a probabilistic aspect, defined by not only a mean but also a variance and distributional shape. The uncertainty associated with stochasticity then comes from randomly sampling from this distribution. (Additional uncertainty might come from not fully understanding the distribution itself, but I wouldn’t consider that to be stochasticity, per se, because it doesn’t stem from randomness. Some of my colleagues might disagree here, though, as per the next paragraph.)
What if nothing in ecology really is random? In this worldview, stochasticity is a stand-in for anything we can’t attribute to measurable deterministic processes. More cynically, it’s a scapegoat when deterministic arguments fail to explain what we see. The implications of changing from a probabilistic definition of stochasticity to one based on the limits to our knowledge are profound. Here, we could make a system less stochastic by measuring more things. I find that last statement incredibly unsettling, and quite contrary to my stochasticity-as-randomness view, but this is where my head starts to hurt. I can usually pretty happily ignore the question of whether anything in ecology is truly random because regardless of the answer, our finite ability to know things will always render ecological systems effectively random. With this thinking, I can’t logically reject the idea that increasing knowledge decreases stochasticity, but the idea still doesn’t fully sit right with me.
Certainly, it matters what we think an ecological pattern is random with respect to. In plant community ecology, for example, the spatial arrangement of different species may be effectively random with respect to one another, but not with respect to fine-scale environmental heterogeneity (or vice versa). Here again, whether a pattern appears to be stochastic or not depends more on whether we’ve measured the right drivers, than on any true random or probabilistic element.
In sum, stochasticity may be randomness, or a probabilistic representation of non-random things we don’t or can’t know. The important distinction between these viewpoints is that in the first, stochasticity is an integral part of the mechanism that generates ecological variation and uncertainty. In the second, stochasticity is something that blurs an inherently predictable pattern, making it only appear uncertain. In the second view, if we could remove stochasticity, we would fully understand the system whereas in the first, stochasticity cannot really be removed nor can the system be fully understood without it.
So, we collectively believe that stochasticity is one or both of these opposing things. There are also several things that we believe stochasticity isn’t. It isn’t variation per se; some ecological variation has deterministic roots, such as population-level variation in demographic rates due to age structure. An individual’s age is deterministic, and so the component of demographic variation due only to age is also deterministic. Stochasticity also isn’t equivalent to neutrality; despite the strong influence of stochasticity on neutral dynamics, stochastic community patterns need not be neutral. Most of us feel that measurement uncertainty and the statistical treatment of error structures in data fall outside the umbrella of stochasticity. (Although I have pondered whether there are hidden biological insights in the way statisticians deal with process error. Folks in my lab meeting didn’t find this particularly compelling, but I will probably continue to ponder, if for no other reason than to build a better conceptual bridge for myself between model-fitting and theory.) Next, stochasticity is not simply the application of stochastic computational tools, like MCMC. Several of us study stochastic processes using deterministic tools, an approach that conserves the idea of a distribution of outcomes without actually sampling from that distribution. Lastly, stochasticity isn’t chaos, although the two might be effectively indistinguishable in a practical sense. (The interaction between stochasticity and nonlinear phenomena like chaos came up a lot in discussion, and is a topic near and dear to my heart and the hearts of many others listed above. To me, this is where the effects of stochasticity get the most interesting: it generates the storage effect, reactivity, and long or recurring transient patterns, for example.)
2) Questions of scale:
I asked my colleagues to explain the origin of stochasticity in ecology, and several pointed to heterogeneity at smaller scales. Heterogeneity of cells within an individual generates stochasticity at the scale of the organism, heterogeneity among individuals generates stochasticity at the scale of the population, and the magnitude of interspecific differences influences the importance of stochasticity at the scale of the community. In other words, deterministic processes that generate heterogeneity at small scales generate variance that is effectively stochastic at larger scales. (Stochastic processes at small scales also surely contribute to stochastic variation at larger scales, but this statement is less interesting.) Conversely, when we measure variance at a particular scale, we tend to attribute it to lower-level processes because, ideally, variation in higher-level processes was controlled in the study design.
As responsible ecologists, we all know the importance of defining the spatial and temporal scale at which a particular idea is relevant. By the same token, I think we should define our “scale of determinism” – the scale outside of which we chalk all variation up to stochasticity, whether or not it’s inherently random. I’m not quite sure how to implement this, but it’s a cute idea. Perhaps the scale of determinism is internally determined: at small scales everything appears random (think of “population dynamics” on a spatial patch so small individuals are continually walking in and out) and at large scales everything appears constant (imagine summing up the population dynamics on a large network of asynchronous local patches). In between is the scale of determinism. Of course, the dynamics at that intermediate scale still won’t be fully deterministic, so perhaps this is a misleading way to think of things. In any case, a “scale of determinism” need not be a spatial scale as in this example; it could be a biological level of organization, or even simply an explicit list of processes (like density dependence) that we consider to be deterministic within the context of a particular study.
3) The term “stochasticity” gives a false sense of precision:
The inherent probabilistic nature means that we can never precisely predict stochastic outcomes. One contributor suggested that this might be why we opt to use the more pretentious-sounding word “stochastic” instead of just saying “random”. At least in common parlance, “random” has connotations of hopelessness. To try to understand something random feels futile; it’s just random. But of course, we can understand many things about stochastic/random processes, despite this inherent unpredictability, by studying the probability distributions.
So, the study of stochasticity lies somewhere between futility and precision. Personally, I’m comfortable in this neighborhood. Still, it’s worth remembering that a solid mathematical description of stochasticity and a biological understanding of stochasticity are two different things. I don’t have any objection to our use of the term “stochasticity”, but I do wonder if we let ourselves get away with muddled terminology in ecology because we rest assured that the mathematicians have sorted it out (“ah yes, ‘stochastic processes’ – people know how to deal with those”). Semantics aside, we do seem to have an awful lot of at least somewhat divergent ideas about stochasticity, even with a hand-picked group of like-minded people.
Closing thoughts: I initiated this blog post, and the conversations that led up to it, hoping to finally get over some of the conceptual issues that have been nagging at me for years. As I wrote, I revised my goal to be to get you, the readers, to feel just as angsty about stochasticity as me. Isn’t it an exciting feeling?! I mean that seriously, no sarcasm: isn’t it exciting? There’s a big, wide, stochastic world out there full of monumentally important questions yet to be answered.
[Bonus material: Chris Stieha took the above text and ran it through a program he wrote that assembled random sentences from clauses in my original text. The result: a stochastic essay on stochasticity. Here are my favorite quotes —
“stochasticity is anything with a hand-picked group of colleagues, collaborators, and lab members”
“stochasticity is an integral part of the term “stochasticity”, but I wouldn’t consider that to be answered.”
“There’s a big, wide, stochastic world out there full of monumentally important questions yet to be deterministic within the context of a distribution of outcomes without actually sampling from this distribution”]
Shamelessly self-promoting p.s. from Jeremy: I have an old post on what stochasticity is, with the terrible title “ignorance is bliss (sometimes)“. 🙂