Favorite good (or bad?) examples of “operationalizing” vague ecological concepts?

“Operationalization” is the term for taking a concept that’s vague or abstract and making it more precise and concrete, so that it can be put to practical use. Like many scientific and social scientific fields that aren’t physics or chemistry, ecology has many concepts that are only vaguely defined, or at least were only vaguely defined when they were first proposed. “Niche” is an infamous example. Or think of how one response to my critique of the intermediate disturbance hypothesis was to question whether the ideas I was critiquing were “really” part of the intermediate disturbance hypothesis, properly defined. Few big ideas are born fully formed, so most new ideas have to go through some refinement and elaboration to make them operational

Sometimes, the process of operationalization is successful, meaning that eventually everyone agrees on the definition of the concept and can go out and apply it. For instance, everybody agrees what “gross primary productivity” is. There might be practical obstacles to measuring it in any particular case, and different ways of measuring it might be prone to different sorts of errors. But those are practical obstacles, not conceptual ones.

But sometimes, the process of operationalization fails.

For instance, many plant ecologists starting with Welden and Slauson (1986 Q. Rev. Biol.) have argued that there is a crucial distinction to be made between the “intensity” of competition (roughly, the absolute effect of competition on the physiology of the affected individual plant) and the “importance” of competition (roughly, the long-term ecological effect of competition, relative to other processes). Plant ecologists have tried to operationalize these concepts by proposing and applying quantitative indices of competitive “intensity” and “importance”–more than 50 such indices, by one count! Of course, this proliferation of indices is a sign of failed operationalization. If we really knew what competitive “intensity” and “importance” were, we wouldn’t keep proposing new indices of them and arguing over which indices are best. Freckleton et al. (2009) and Rees et al. (2012) argue–to my mind completely convincingly–that the concepts of competitive “intensity” and “importance” are too vague and disconnected from mathematical theory to be made operational. Just because you have a plausible-seeming way to put a number on some vague verbal idea doesn’t mean that vague verbal idea is now “operational” (see Brooker et al. 2013 for the opposing view).

To elaborate a bit on Rees et al.’s point, if you have a mathematical model and use it properly, the problem of operationalization doesn’t arise. You just measure whatever quantity the model tells you to measure. For instance, in my own work on spatial synchrony, my collaborator and I developed a mathematical model that made some predictions about how spatial synchrony should behave. When we tested those predictions experimentally we made sure to calculate from our data the same measure of spatial synchrony as we’d used in the model (the cross-correlation; Vasseur & Fox 2009). So the fact that there are various other measures of synchrony that are consistent with the vague verbal idea of “synchrony” was irrelevant. As another example, lots of recent work on diversity and stability operationalizes “diversity” and “stability” as “whatever meaning those terms have in the mathematical model I used to generate the predictions I’m testing.”

Another example: think of the debate over how to partition alpha and beta diversity. If we all agreed on what alpha and beta diversity are, would there be any debate over how to partition them?

But the above examples are merely the first ones that occurred to me off the top of my head; perhaps they’re not representative. And I’m having trouble thinking of any other general principles to guide successful operationalization, besides “don’t try to go straight from a vague verbal concept to a quantitative, model-independent measure of that concept”. That’s where you come in. Tell me: what are your favorite examples of successful and failed operationalization in ecology? Let’s try to compile some examples and draw some general lessons.

(p.s. Yes, I’m aware that psychologists and other social scientists worry a lot about this issue and have a large literature on principal components and latent variables and such. I know that literature a bit, in part as a reader of Andrew Gelman’s blog–which does not inspire much confidence in me that psychologists have the operationalization problem solved. For instance. So feel free to comment that there’s an easy technical solution here, but be prepared for some pushback. Don’t get me wrong, statistical techniques like PCA and latent variables have their place. But I doubt the one-size-fits-all solution to problems of operationalization is “come up with a bunch of indices of whatever vaguely-defined abstract concept it is you’re trying to measure, then run them through a PCA and take the first principal component”.)

Related old post:

The most common way to fish for statistical significance in ecology

41 thoughts on “Favorite good (or bad?) examples of “operationalizing” vague ecological concepts?

  1. Great question. OK, here’s one to kick off: “specialisation”. In my area (and I get the sense elsewhere across EEB, too), the notion of what it means for a species to be a “specialist” or “generalist” has multiple meanings, for example:

    – the number of other species with which it interacts in a given relationship (predator prey, mutualism, etc.)

    – the number of species with which it interacts weighted by the strength of that interaction (used in network studies)

    – the turnover of the number of species with which it interacts across multiple populations

    – the number of broad/functional groups of taxa with which it interacts

    – the possession of specific adaptations that allow it to interact in a particular way

    – limited distribution in a particular habitat type

    I’m sure there are others that I’ve missed. I can provide some literature behind this if you want to follow it up.

    • Hmm. That sounds to me like a more different case–a single term to refer to a bunch of different things that aren’t necessarily related. Rather like how “selection” can refer to resource choice by individual organisms (see Emilie’s comment), or natural selection. Each of those two uses of “selection” has its own operationalization. “Stability” is kind of the same–there are different operationalizations of different uses of the term.

      • No, I disagree, at least for part of my list. Much of the debate and disagreement in pollination ecology about the relative frequencies of specialized and generalized interactions, pollination syndromes, etc. revolves around how to interpret the world “specialized” to understand the same set of interactions. Here’s an example:

        Plant species x is an orchid with highly specialised pollen delivery (as pollinia) = specialist

        But:

        Plant species x is pollinated by 20 different species in one population = generalist

        But:

        All of those 20 pollinators are medium sized bees, therefore plant species x is pollinated by a single functional group of species = specialised

        But:

        In different populations across the full range of plant species x the effective pollinators include large hoverflies and wasps = generalist

        All of these examples can be found in the literature going back into the 19th century, and the distinction is particularly acute for those with botanical versus ecological training, where the former emphasises floral adaptations, the latter ecological interactions and context.

        I’m sure similar discussions have gone on in other parts of ecology, e.g. predator-prey interactions.

      • Ok I buy that as a failed operationalization.

        This gets back to your old post with Angela Moles on whether species interactions are stronger and more specialized in the tropics. A lot of the pushback you got was arguing about operationalizations.

      • Yes, agreed. I suspect that part of what you’ll uncover with the failed ops question is that one of the root causes is ecologists coming to a subject from very different backgrounds/traditions.

      • “I suspect that part of what you’ll uncover with the failed ops question is that one of the root causes is ecologists coming to a subject from very different backgrounds/traditions.”

        Can you elaborate? Do you mean, failed operationalizations happen when ecologists lack a background in the topic and so make mistakes that someone with more background wouldn’t make? Or do you mean failed operationalizations happen when a mix of ecologists of different backgrounds all work on the same topic?

        And can you give an example?

      • I was really thinking along the lines of “failed operationalizations happen when a mix of ecologists of different backgrounds all work on the same topic”, but I suppose that the first explanation could apply in some cases.

        The example I’m most familiar with is the one that I’ve already mentioned: “specialisation” as applied to plant-pollinator interactions. I’ve seen examples of botanically-trained scientists applying “specialised”to a plant’s pollination system when referring to the floral phenotype (e.g. orchids, asclepiads) regardless of the diversity of pollinators involved. I’m also aware of examples where no consideration was made of sampling effort and species termed “specialised” based on limited observations of pollinators, again by botanically-trained researchers.

  2. One of my favourites is “stochasticity”, operationalized by McShea and Brandon (“Biology’s first law”, U. Chicago Press) in the “with respect to” sense. Organisms don’t live and die at random (e.g., there is always a cause of death), but if demographic events occur at random with respect to the identifies of alleles at a given locus, you get stochastic genetic drift at that locus. Replace “identities of alleles at a given locus” with “species identity”, and you’ve got an operational definition of stochastic community drift. Arguments about whether anything in the universe is truly stochastic will go on forever, but one can in principle actually measure whether event X occurs at random with respect to situation Y.

  3. Selection might be a good example, especially in relation to preference. These vague concepts are well defined in Johnson 1980 (Ecology 61: 65-71), but have been operationalized with selection ressource analyses (see Manly, et al. 2002. Resource selection by animals: statistical design and analysis for field studies).
    However, I don’t know the historical context of that.

  4. Ecological resilience (and not engineering) is in its early stage of operationalization, so I don’t think it’s fair to judge failed/not failed just yet. There have been numerous and apparently successful attempts at operationalizing specified resilience, and (at least on short time scales) frameworks for operationalizing the management of systems for resilience.

    As an emergent property, can we ever operationalize (i.e., prove) resilience? Probably not.

  5. I should do a poll on this: ecosystem health, successful or failed operationalization?

    I’d say failed just because I reject the analogy on which the original verbal idea is based. I only think it makes sense to speak of healthy or unhealthy individual organisms, not healthy or unhealthy ecosystems (or cities, or other non-organisms that sometimes get described as “healthy”). So since I reject the vague verbal idea, I don’t think it can be operationalized. I think indices of ecosystem “health” are better thought of as measuring, I dunno…lack of human impact, or “is this ecosystem in the state we humans would like to see it in?”, or something.

    I now await somebody who works on ecosystem health to tell me I’m totally wrong (seriously, I’d be very happy to get pushback on this.)

    • I totally agree with “failed”. What makes things more complicated is that “esosystem health” comes up every now and then in management issues and we ecology consultants are often asked to assess an ecosystem as “healthy” or not. In this case in my opinion we have two (2) failed operationalizations: “ecosystem” AND “health of an ecosystem”!!

    • It seems like a first step is to have an operationalization of “health”, which is not a small topic nor free of debate, but could be defined as a multidimensional assessment of physiological functions compared to the state we humans would like/expect to see it in. If one can accept that definition, then I think it would be hard to have a problem with the analogy to ecosystem health as: a multidimensional assessment of ecological functions compared to the state we humans would like/expect to see it in. Other operationalizations of “heath” may not translate as well. Do we now digress into operationalizations of “physiological” and “ecological” function”?

      • “a multidimensional assessment of physiological functions compared to the state we humans would like/expect to see it in.”

        Sure, that’s fine. I just object to using the word “health” to refer to “whatever it is (certain) people happen to like”. If you’re giving a technical meaning to a word that’s in common use, you should try to avoid giving a technical meaning that’s almost the opposite of the common connotation of the word. “Health” does not have the connotation of “being in a state that some people happen to like”. The common connotation of “health” is much more objective and has normative force that mere personal preferences lack. That isn’t to deny that what’s “healthy” can be debatable. It’s just to say that calling this sort of index ecosystem “health” is to give the index a sheen of objectivity and normative force that it doesn’t deserve. At least not to my mind.

        Political advocacy groups promoting their causes play this game all the time. Come up with some arbitrary index of something, name it with a term in common use that has some normative force, and then draw a political conclusion from the results. A conclusion that’s hard to question because if you question it it looks like you’re questioning widely-accepted norms. Think of indices of “freedom” or “well being” or “livability” or whatever. I just instinctively don’t like even a whiff of that sort of thing showing up in science.

        Don’t misunderstand, I actually don’t read the ecosystem health literature at all, so perhaps I’m coming from a place of ignorance here. That rant is just an expression of a general, deep-seated, knee jerk aversion on my part. I’m averse to trying to operationalize ecosystem “health” for more or less the same reason I don’t consider sports in which the winner is determined by judges evaluating the competitors’ conformity to some arbitrary standard to be proper sports (e.g., figure skating, rhythmic gymnastics).

        I now look forward to getting pushback both from readers who work on ecosystem health, and readers who like figure skating. 🙂

    • Yes, disturbance is a interesting one. It is clear what it is in most models, but hard to define in a clear (include-exclude) manner in the real world. The range of definitions is broad and even gets into strange territory as when a change in a disturbance regimes (such as a reduction in fire frequency) is labelled as a disturbance. Nonetheless, the concept of disturbance is so essential to so much (at least in forest ecology) that we seem to get on with it and just accept that definitions vary by study. So it appears we have operationalised the confusion.

    • This is not a peer-reviewed scientific literature but…Nassim Taleb argues in his book “Antifragile” about this: Stress are constant and don’t enable the system to recover, whereas disturbances occur rapidly and enable the system to be “antifragile” (opposite of fragile = “stronger”). One may argues that the concept of “healthy ecosystem” is highly linked to disturbance/stress and resilience concepts.

  6. I am not seeing many answers to your “successful” half of the question. I could add a dozen more to the failed list. And would definitely have ranted on alpha & beta diversity if you hadn’t.

    But successful, lets see …
    Population density and even abundance
    Occupancy
    Relative abundance
    Transition probabilities (sensu stage/age matrices)
    Biome (and vegetation classification) (admittedly more borderline than the others)
    Any number of ecosystem concepts such as mineralization rates, N:P ratios, carbon flux
    Would have said NPP but there was just a paper by Simova & Storch really calling this into question
    A number of physiological properties (e.g. A_max = maximum photosynthetic rate although it is widely recognized and handled that this depends on spatial & temporal scale, metabolic rate, maximum velocity, etc)
    Diet preference (again a bit more borderline)

    There is a theme here – most subdisciplines of ecology are better operationalized than community ecology …

    • Brian, some of your examples strike me as cases where the “operationalization” is trivial. What else could “relative abundance” be besides” abundance of species X divided by total abundance of all species”? Same question for occupancy, population density, abundance, and several others. What are the obstacles to operationalization? Is it stuff like deciding how to count “individuals” of a plant that sends out runners underground?

      Sounds like I should look up Simova and Storch, since I had thought we all knew what NPP is, operationally.

      • I’m not going to disagree with you that they’re trivial. But I will say:
        a) There is a danger of being tautological – anything we’ve successfully operationalized is trivial
        b) population abundance and occupancy are rarely directly measured – they are normally estimated by ever increasingly sophisticated methods – there are literally whole books on how to measure them. And lots of debate about appropriate and relevant scales and scale-dependence of methods. Is that trivial? Similarly there is a whole book on ecosystem measurement methods.

        And if population size, N, is trivial, why isn’t species richness, S, trivial? (My answer because N scales linearly but S doesn’t). But they seem equally trivial.

      • @Brian:
        “There is a danger of being tautological – anything we’ve successfully operationalized is trivial”

        Agreed, but thinking about it, I don’t *think* I’m falling into that trap. For instance, I said in another comment that I think “competition” has been operationalized reasonably well. And I don’t think that was trivial. I would say operationalizing “competition” was a lot harder than operationalizing, say, abundance.

        Re: your b), yeah, I would say that using sophisticated methods to estimate (say) abundance is just a matter of overcoming technical challenges. Same for estimating species richness or ecosystem variables or whatever. Those technical challenges aren’t trivial in the sense of being easy to overcome; apologies for not being clearer on that. But I do think they are technical challenges, not *conceptual* ones. Indeed, it’s hard for me to see how we could push so far with increasingly sophisticated technical methods to estimate abundance unless we’d already operationalized abundance conceptually. I think the conceptual challenges to operationalizing “abundance” are mostly trivial. (Though in saying that perhaps I’m being too quick to write off organisms that lack discrete individuals as rare exceptions; such organisms do raise conceptual problems for how to measure abundance operationally.)

  7. Ecosystem health is an interesting example for sure. The underlying issue is only partly in the word “health”, more generally it’s in the question whether we think good to bad can be arrayed along a single dimension. (A precursor in rangeland science is the concept of “range condition”.) For a few decades now it’s been pretty well understood by most researchers that actually there are multiple desirability scales that don’t necessarily align with each other. But usually there is strong pressure from the minister for environment for a single scale that you can manage towards and build a coloured map from, and this in turn translates through the land or water management agency to pressure on researchers. Seems to me to be an area where at least so far, the research community hasn’t been able to persuade the applied agencies there are more sophisticated ways to look at what we’re doing.

    • Yes, one cause of dubious operationalization is the felt need to boil things down for some non-scientific audience or some policy purpose. As someone who’s own work and reading is far removed from policy, this isn’t an issue I’m well-qualified to comment on.

  8. It seems to me there are a lot of parallels between “operationalization” (which I take as measuring in a repeatable way a variable that is useful to know) and language overloading. Could one reasonably claim that words/concepts that have a single meaning are easy to operationalize and all the things that are vague/ambiguous/overloaded like “competition”, “specialist/generalist”, “disturbance”, “stress”, “ecosystem health”, are impossible to operationalize because they are not one thing and thus it is nonsensical to repeatably measure them.

    • Hmm. “Competition” is one that I think has been operationalized reasonably well. But yes, I agree with your broad point. The root of failed operationalization is trying to summarize in one number something that’s either vague or multifaceted.

  9. This may be a bit off topic sorry but concepts becoming ‘operationalized’ made me think about when a concept, or theory, becomes a law. Are some ecological relationships able to be considered laws? I had been thinking about metabolism and the -3/4 scaling relationship just before reading this… Could that be considered an ecological law?

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s