Favorite good (or bad?) examples of “operationalizing” vague ecological concepts?

“Operationalization” is the term for taking a concept that’s vague or abstract and making it more precise and concrete, so that it can be put to practical use. Like many scientific and social scientific fields that aren’t physics or chemistry, ecology has many concepts that are only vaguely defined, or at least were only vaguely defined when they were first proposed. “Niche” is an infamous example. Or think of how one response to my critique of the intermediate disturbance hypothesis was to question whether the ideas I was critiquing were “really” part of the intermediate disturbance hypothesis, properly defined. Few big ideas are born fully formed, so most new ideas have to go through some refinement and elaboration to make them operational

Sometimes, the process of operationalization is successful, meaning that eventually everyone agrees on the definition of the concept and can go out and apply it. For instance, everybody agrees what “gross primary productivity” is. There might be practical obstacles to measuring it in any particular case, and different ways of measuring it might be prone to different sorts of errors. But those are practical obstacles, not conceptual ones.

But sometimes, the process of operationalization fails.

For instance, many plant ecologists starting with Welden and Slauson (1986 Q. Rev. Biol.) have argued that there is a crucial distinction to be made between the “intensity” of competition (roughly, the absolute effect of competition on the physiology of the affected individual plant) and the “importance” of competition (roughly, the long-term ecological effect of competition, relative to other processes). Plant ecologists have tried to operationalize these concepts by proposing and applying quantitative indices of competitive “intensity” and “importance”–more than 50 such indices, by one count! Of course, this proliferation of indices is a sign of failed operationalization. If we really knew what competitive “intensity” and “importance” were, we wouldn’t keep proposing new indices of them and arguing over which indices are best. Freckleton et al. (2009) and Rees et al. (2012) argue–to my mind completely convincingly–that the concepts of competitive “intensity” and “importance” are too vague and disconnected from mathematical theory to be made operational. Just because you have a plausible-seeming way to put a number on some vague verbal idea doesn’t mean that vague verbal idea is now “operational” (see Brooker et al. 2013 for the opposing view).

To elaborate a bit on Rees et al.’s point, if you have a mathematical model and use it properly, the problem of operationalization doesn’t arise. You just measure whatever quantity the model tells you to measure. For instance, in my own work on spatial synchrony, my collaborator and I developed a mathematical model that made some predictions about how spatial synchrony should behave. When we tested those predictions experimentally we made sure to calculate from our data the same measure of spatial synchrony as we’d used in the model (the cross-correlation; Vasseur & Fox 2009). So the fact that there are various other measures of synchrony that are consistent with the vague verbal idea of “synchrony” was irrelevant. As another example, lots of recent work on diversity and stability operationalizes “diversity” and “stability” as “whatever meaning those terms have in the mathematical model I used to generate the predictions I’m testing.”

Another example: think of the debate over how to partition alpha and beta diversity. If we all agreed on what alpha and beta diversity are, would there be any debate over how to partition them?

But the above examples are merely the first ones that occurred to me off the top of my head; perhaps they’re not representative. And I’m having trouble thinking of any other general principles to guide successful operationalization, besides “don’t try to go straight from a vague verbal concept to a quantitative, model-independent measure of that concept”. That’s where you come in. Tell me: what are your favorite examples of successful and failed operationalization in ecology? Let’s try to compile some examples and draw some general lessons.

(p.s. Yes, I’m aware that psychologists and other social scientists worry a lot about this issue and have a large literature on principal components and latent variables and such. I know that literature a bit, in part as a reader of Andrew Gelman’s blog–which does not inspire much confidence in me that psychologists have the operationalization problem solved. For instance. So feel free to comment that there’s an easy technical solution here, but be prepared for some pushback. Don’t get me wrong, statistical techniques like PCA and latent variables have their place. But I doubt the one-size-fits-all solution to problems of operationalization is “come up with a bunch of indices of whatever vaguely-defined abstract concept it is you’re trying to measure, then run them through a PCA and take the first principal component”.)

Related old post:

The most common way to fish for statistical significance in ecology