Many ecologists expect competing species to exhibit compensatory dynamics, meaning that the densities of any two competing species should be negatively correlated over time or across space. If your competitor increases in abundance, you ought to decline, right? After all, to the extent that two species are competing, that means that when one increases, it’s at the expense of the other, right?
Um, no. Or rather, not necessarily. For instance, environmental fluctuations can cause competing species to exhibit positive rather than negative correlations in abundance. Think of a drought which causes the density of every plant species to decline, even though they’re all competing. But there’s a deeper reason why you should not necessarily expect the densities of competing species to all be strongly negatively correlated with one another: in general, it’s mathematically impossible. I don’t think this fact is as well-known as it should be, so I thought I’d post on it.
Say you have just two competitors, each of whose densities you’ve measured at a bunch of different time points, or a bunch of different spatial locations. In this special case, the correlation coefficient (Pearson’s correlation or rank correlation) between the density of species 1 and the density of species 2 can indeed take on any value from +1 to -1. So depending on how strongly the species compete and other factors, it’s possible that their densities could be perfectly compensatory (correlation = -1). So for the sake of illustration, let’s assume that the correlation between their densities is -1.
Now imagine that there’s a third competitor. How will its densities correlate with those of species 1 and 2? Well, to answer that, you’d have to specify more information about the ecology of all three species. But without knowing anything about the ecology, I can tell you what the answer won’t be. Species 3 won’t have a correlation of -1 with both species 1 and 2. Because that’s mathematically impossible. For instance, if species 1 and 3 have a correlation of -1, then by definition species 2 and 3 must have a correlation of +1, i.e. perfectly synchronous rather than perfectly compensatory dynamics. Conversely, if species 3 has correlations of -1 with both species 1 and 2, then by definition species 1 and 2 must have a correlation of +1.
This three species case is a simple illustration of a general principle: the more species you have, the less-compensatory their dynamics can possibly be. It’s mathematically possible for any number of species to all be perfectly in sync with one another. But the more species you have, the less density compensation they can possibly exhibit, on average. In general, we can describe the pairwise correlations among s competitors with a correlation matrix, a square matrix with s rows and s columns, one row and column for each species. The number in row i and column j gives the correlation between species i and j, and of course the same number will appear in row j and column i since the correlation between species i and j is the same as that between j and i. The numbers on the diagonal will all be +1, since by definition any variable is perfectly correlated with itself. Now, as a matter of mathematical necessity, correlation matrices are positive semidefinite. Which turns out to imply that, the larger s is, the less-negative the off-diagonal elements of the correlation matrix can possibly be, on average.
For instance, in the special case when every pair of species has the same correlation, the minimum possible value of that correlation equals -1/(s-1). Here’s the graph for that special case:
As you can see, even with as few as 5 species, in this special case the minimum possible correlation is only -0.25, which is pretty weakly compensatory. In the limit, as s goes to infinity, the minimum possible correlation goes to 0 (i.e. species fluctuate independently of one another).
Of course, in reality the pairwise correlations won’t all be equal, and so even with many competing species it’s possible that some pair of them might have strongly compensatory dynamics. But if they do, that just implies that some other pair of them must have strongly synchronous dynamics. On average, the pairwise correlations can’t be more than slightly negative when you have more than a few species.
Note as well that the same basic point holds for other measures of synchrony. For instance, the exact same points hold if you want to analyze synchrony in the frequency domain by looking at phase differences.
This mathematical fact is certainly familiar to folks who do a lot of work on this stuff, like my collaborator Dave Vasseur. But it deserves to be more widely known. Lots of ecologists have the vague sense that competitors ought to exhibit compensatory dynamics, and so are somewhat surprised to learn that compensatory dynamics are actually quite rare in nature. But the reason they’re rare is mathematical, not ecological. Which means you cannot use the rarity of compensatory dynamics as evidence for anything about ecology. For instance, you can’t say “These species only exhibit weakly compensatory dynamics, so they must not be competing very strongly”. You can’t even say “These species only exhibit weakly compensatory dynamics, so environmental fluctuations must be generating synchrony that overrides the strongly compensatory dynamics that would otherwise occur.”
Just to be clear, there absolutely is scope for the strength of synchrony or compensation to vary among communities, and among different pairs of species, for all kinds of interesting ecological reasons. But if you aren’t clear on what dynamics are possible, you’re liable to misinterpret actual dynamics.
Dynamic Ecology 
Not sure I’m following this one. You switched from the question of whether two species can compensate each other to the question of whether more than two can do so. Those of course are not going to be the same thing, but this is completely trivial. You can still get any two species from some larger group that compensate.
That’s the point of the post. Sorry if it wasn’t as clearly written as it should have been.
I’m glad the point of the post is obvious to you, but based on admittedly anecdotal evidence I think you’re in the minority.
It’s a very important point that goes beyond species compensation in community ecology (although the basic misconception was the main reason we wrote this* and that* a few years ago).
Any time there is spatial heterogeneity across population patches, the correlation in habitat quality is limited in the same way for >2 patches.
Jeremy, I could probably dig up some reviewer reports (and at least one paper) to flesh out your anecdotes with ‘data’ if you like, it apparently isn’t as obvious to some people as it needs to be.
Jim, I wonder if the problem here stems from the fact that you can have a very simple competitive community, where all species are otherwise identical at the within and between species levels (e.g., as in the ‘diffuse competition’ communities of Roughgarden & Hughes 2000), yet simply moving from a 2 to 3 species community (adding another ‘identical’ species) limits the potential correlation structure of species (or environmental) fluctuations. Some more confusion might arise from the differences in population vs per-capita growth fluctuations.
* Some excellent, indirect journal plugging by our favourite (blogging) Oikos Ed ;o)
Thank you Jeremy for your interesting post. However, I was wondering about some of your statements that is “… the larger s is, the less-negative the off-diagonal elements of the correlation matrix can possibly be, on average.” This statement apparently contradicts what has been argued about the compensatory mechanisms responsible to stabilize aggregate community properties as species richness increase.
The work of Robert May (1972) and further reviews (Cottingham et al, 2001) have emphasized that as species richness increase, compensatory dynamics among populations also increase (correlations become more negative), and this is responsible for the stabilizing effect observed at the community level.
Why this pattern is apparent? It is something that people working with diversity-stability relationship are not considering for?
I will be glad if you or any other could do some comment about that.
Thnaks,
Adriano
Hi Adriano,
When you write “as species richness increases, compensatory dynamics among populations also increase (correlations become more negative”, I think you’re articulating the widespread but mistaken view this post is pointing out. I think if you look carefully at both the references you cite, you’ll find that they *don’t* say this. I know May (1972) doesn’t. May (1972) actually has *nothing* to do with density compensation at all! May (1972) is about “stability” in a *very* different sense. To be precise, May (1972) is about how the real part of the largest eigenvalue of the community (Jacobian) matrix changes as a function of species richness and the fraction of non-zero matrix elements when the non-zero matrix elements are chosen randomly from a normal distribution with zero mean and a particular standard deviation. To be more informal, May (1972) is about how whether, and how quickly, a community returns to equilibrium following a small perturbation that changes species’ densities only slightly from their equilibrium values. None of which has *anything* to do with how those species’ densities covary. For instance, species could fluctuate synchronously or asynchronously as their densities return to equilibrium, but May’s results don’t tell you anything about this. And while I haven’t read Cottingham et al. 2001 in a while, and do think that some ideas in that article aren’t explained in the best way, I’m pretty sure that Cottingham et al. don’t make a flat-out mistake like claiming that, as species richness increases, the average pairwise correlation between two species becomes increasingly negative.
The larger issue here is that the “diversity-stability” literature is unfortunately a mess. Lots of totally-unrelated ideas have all been lumped together under the heading of “diversity-stability”. Existence (feasibility) of equilibria, rates of return to equilibria, patterns of covariation, variability of aggregate variables like total biomass and primary productivity…It’s the kind of thing that begs for a blog post to sort out–except that it would probably take several posts! And even then I doubt they’d succeed. Back in the early 80s Stuart Pimm wrote what was at the time a very well-cited article, describing all the different senses of “stability” in the ecological literature and pointing out that they basically had nothing to do with one another. And here we are years later, and people are only more confused about than they were in the 80s. So when I complain about the imprecision of words and the very real confusion it causes, the diversity-stability literature is what I’m thinking of.
Frankly, unless you know some mathematics you probably have little hope of making heads or tails of the diversity-stability literature. You’ll almost inevitably struggle to understand any bit of it, much less how the different bits relate (or in many cases, *don’t* relate) to one another, or even realize that they *are* different bits. And the worst part is, you may well *not even realize that you’re confused*. Mere words are simply too imprecise to do justice to the math, especially for readers who are just looking to “get the gist” of the math. Because the same words that “give you the gist” (i.e. a highly-simplified summary) of one bit of math can also be used to “give you the gist” *of some completely different and unrelated bit of math*.
In fact, I think it´s easy to argue that the eigenvalues and Jacobian matrix are directly linked to covariance patterns in species fluctuations. At least, an awful lot of work in locally stable discrete time systems points in that direction, e.g. Ives et al (1999, Science; 2000, Ecology Letters) and the Hughes and Roughgarden paper I linked to above are good places to start with pretty simple community structures.
A really excellent, more general, though probably underappreciated paper dealing with population covariance matrices in more complex systems is Greenman & Benton (2005, Theoretical Population Biology). It does a couple of odd things I still don´t fully understand, but really shows how the Jacobian links with the environmental covariance matrix to drive population covariances for locally stable and ´less´ stable systems, even in coloured environments!
Of course, some mathematical knowledge is required to get the most out of these papers, but the earlier ones are certainly a good place to start.
Yes, I’m aware of those papers. A couple of technical points (which I know you know Mike, but probably many readers don’t). May’s results are for a deterministic system, and they don’t assume any particular model–they just assume that there is some (possibly nonlinear) model that one could write down that would have an interior equilibrium and an associated Jacobian with the sorts of properties May assumes that his Jacobians have. In contrast, results like those of Ives, Hughes, etc. are, if memory serves for stochastic Lotka-Volterra systems, i.e. a specific model is assumed.
Technical quibbling aside, yes, you’re right that, in general, the ways in which species interact, as summarized by a Jacobian matrix, will have consequences for how they covary over time. I intentionally avoided saying this, because it would have undermined the point of the post. The point of the post is to force people to try to force people to think precisely about this stuff and actually get to grips with the math. Rather than just settling for verbal summaries of the math which are imprecise at best and seriously misleading at worst. If you tell people who are not mathematically inclined “there are close linkages between Jacobian matrices and the resulting patterns of correlation in species’ abundances”, what people hear is something like “Jacobian matrices and correlation matrices are basically the same”. And unfortunately, even if you follow up by telling people “Go read Ives, Hughes, and Benton to see what I mean”, most people who are not mathematically inclined will either not bother, or will read them and misunderstand them (or understand them only very superficially, which amounts to misunderstanding them). Wish I knew what to do about this, but I don’t.
A depressing line of thought, I know. I guess I’m just grouchier than usual today.
Make them required reading for some of the courses you’re teaching 😉
Being really brave, one could build an entire course around Greenman & Benton (2005)!
(this would probably make Ripa & Ives (2003, TPB) a prerequisite course)
p.s. Just to be clear Adriano, my reply is not at all meant as a criticism of you or anything you wrote. My reply is a lament that this literature is so difficult to make sense of, especially for anyone who doesn’t know a lot of math.
Thank you Jeremy for your elucidating reply! I totally agree with the confusion around the term stability means. My post was intended to bring into the discussion a subject that has been hotly debated in literature over the years and in my opinion have elements which are related to the message of your post.
Thank you.
Hrm. Does this apply for predator-prey interactions as well? Might this be a simpler explanation behind the whole weak-interactions-are-the-rule pattern seen in so many papers?
I also wonder to what extent that those low correlations at high diversity (as that’s what we’re talking about here) could in part be driven by supression effects from the competitive network in and of itself.
Hi Jarrett,
Not sure what you mean by “apply for predator-prey interactions as well”. But it applies to any correlation matrix. So for instance, if for some reason you were looking at the correlations in abundance among a whole bunch of predator and prey species (or a bunch of mutualists, or whatever), then the average pairwise correlation can’t possibly be much below zero. This is a mathematical fact about correlation matrices, which is *totally* independent of the underlying mechanisms generating the variability. If you have a matrix giving the correlations among a bunch of variables, that matrix will be positive semidefinite. Which means that, if your matrix has a whole bunch of variables, the average off-diagonal element of the matrix can’t *possibly* be more than slightly negative. Mike Fowler’s comment emphasizes this, and perhaps the post wasn’t as clear about this as it should’ve been. The post is about the relevance of this mathematical fact to compensatory dynamics, but it’s a fact about any correlation matrix, not just matrices of the correlations of species’ abundances.
“I also wonder to what extent that those low correlations at high diversity (as that’s what we’re talking about here) could in part be driven by supression effects from the competitive network in and of itself.”
Afraid I don’t know what you mean by “suppression effects”. Can you clarify? In any case, it’s not clear to me how you would tell if such “suppression effects,” whatever they might be, are operating in a many-species system. Since it’s mathematically impossible for the average pairwise correlation to be very negative in a many-species system, I’m not sure it’s even meaningful to talk about biological mechanisms that prevent the average pairwise correlation from being very negative in a many-species system. How can it be meaningful to talk about the cause of something that was impossible anyway? That’s like trying to talk about the cause of the fact that 2+2 does not equal 5. Maybe I’m misunderstanding what you mean here. You could talk about mechanisms that prevent *any given* pairwise correlation from being very negative (or very positive)–is that what you mean?
Aside: do you think this fact about correlation matrices has any implications for the interpretation of structural equation models (which typically are based on correlation matrices, IIRC)? Perhaps not, I don’t know.
Re: the application to “weak interactions”, there isn’t one, unless you want to define “interactions” as “pairwise correlations”. Which I for one don’t! 😉
There are various formal definitions of “interaction strength” in the theoretical literature. See Laska and Wootton’s 1998 paper in Ecology for a review. (And of course there are various informal and vague definitions in the heads of empiricists, which, as usual with informal and vague definitions, mostly leads to confusion.) But unless you’re *defining* interaction strengths as correlations (which no formal definition does), there’s no reason why a matrix of interaction strengths has to be positive semidefinite. Indeed, for most definitions of “interaction strength” of which I’m aware, an interaction strength matrix need not be any special kind of matrix at all. It need not be symmetrical around the diagonal, for instance, or have all the diagonal entries be identical, or etc. In principle, any element in the matrix can take on any real numbered value.
Pingback: Advice: 20 different stability concepts « Oikos Blog
For folks who are interesting in reading more about this, there is a nice book chapter on it from a few years back:
Brown JH, Bedrick EJ, Ernest SKM, Catron JE, Kelly JF. 2004. Constraints on negative relationships: mathematical causes and ecological consequences. in The nature of scientific evidence: empirical, statistical, and philosophical considerations. Pages 298-308.
This chapter was part of what Jim Brown and I talked about while I was interviewing for a graduate position in his lab… in 1999. So, if you’re frustrated by journal turn around times, just remember that it can be a whole lot worse.
I think this post confounds the problem of using correlations as evidence of compensatory dynamics with the problem of using evidence of compensatory dynamics to make inferences about ecological processes. It is true that any community, regardless of species richness, can show ‘perfect’ compensatory dynamics (i.e. the community abundance does not change at all over time) regardless of the minimum possible correlation. In a two-species community, ‘perfect’ compensatory dynamics would imply a negative correlation of -1 and as you show the minimum possible correlation would decline as species richness increased. It is also true that any community can show ‘perfect’ lack of compensatory dynamics, when all populations are perfectly synchronous. Those are the two extremes. The fact that correlations don’t capture those extremes very well is the problem, not that where a community lies on that continuum is not informative about ecological processes.
The extremes are actually reasonably simple to interpret– where we see large population changes but zero variance in community abundance we have evidence that processes that cause populations to co-vary negatively are important. One important process that causes negative co-variation is competition. Negative co-variance alone can’t lead us to competition as an important process but it leaves it in the mix (while leaving out ‘common response to environmental fluctuations’ as an important process).
When we see perfectly synchronous changes in population size we have evidence that processes that cause populations to co-vary positively are important. We can rule out inter-specific competition as an important process.
And, where a community lies along that continuum is a measure (albeit, not a perfect one) of the relative importance of interspecific competition.
My key point here is that the extremes of compensatory dynamics are clear and do not vary with species richness –
‘Perfect’ compensatory dynamics – population fluctuations but no fluctuation in community abundance
‘Perfect’ lack of compensatory dynamics – completely synchronous population fluctuations.
Coming up with an ideal index (e.g. symmetrical around zero, not affected by differences in population/community abundances) might be trickier but there is little doubt in my mind that summaries of population covariances can tell us a great deal about the roles of difference processes in regulating population and community abundances.
So, can you say “These species only exhibit weakly compensatory dynamics, so they must not be competing very strongly”? I’m not sure because ‘compete strongly’ can mean so many things. How about “These species only exhibit weakly compensatory dynamics, so environmental fluctuations must be generating synchrony that overrides the strongly compensatory dynamics that would otherwise occur.”? Well ‘must’ is a strong word but I think this statement is reasonable…as long as you use an index of compensatory dynamics that makes sense. And ‘mean correlation of all species-pairs’ doesn’t make much sense (although, some kind of normalization that takes into account the minimum possible correlation might work) .
Hi Jeff,
Thanks for taking the time to comment. But I’m afraid I’m going to have to respectfully but strongly disagree with most of what you’ve said.
Afraid I don’t understand why you wish to define “perfect compensation” as “population fluctuations but no fluctuation in community abundance”. If you define “perfect compensation” in that way, then mathematically it can only occur in a two species community, since that’s the only community in which all species can exhibit pairwise correlations of -1. Perfect compensation, as you define it, is mathematically impossible in communities with >2 species. And I’m afraid I don’t understand at all why you would want to compare the behavior of a community with >2 species to a standard that it’s mathematically impossible for that community to achieve.
On the other hand, if by “perfect compensation” you simply mean “the most strongly-compensatory dynamics a given community could possibly achieve, given the number of species in it and any other mathematically-relevant variables”, then I certainly agree that this lower bound must be kept in mind.
It is true that different measures of compensation behave differently, or at least appear to. But they’re all mathematically interrelated in well-known ways, so the problem identified in the post can’t be made to go away simply by changing one’s index of compensation. For instance, in the past you’ve used the summed covariance as a measure of compensation. The summed covariance for a given community has upper as well as lower mathematical bounds, which depend on both species richness and on species’ variances. This just a restatement of the mathematical facts noted in the post, and reflects the fact that correlation coefficients are calculated from species’ variances and covariances.
I’m afraid I don’t see why you think the mean pairwise correlation doesn’t “make sense” as an index of compensation. Again, the same issues crop up with any other index one might choose, including the one you’ve used in your work in the past. Just to be clear: I’m not saying or implying that you’ve misinterpred your data, merely that the index you’ve used in the past is no better than the one I (arbitrarily) chose to illustrate the point of the post. And while I have no strong feelings about the appropriate choice of index (because you can always convert from one index to another), I would note that something like the summed covariance, with both upper and lower mathematical bounds that depend on multiple factors, seems to me to be, if anything, less easily-interpretable than the mean pairwise correlation.
You suggest that an “idea” index would be symmetrical around zero and have bounds independent of species richness. No such index exists as far as I am aware, and I question whether it’s even possible (not just “tricky”) to define such an index. I also question whether such an index is desirable even if it is possible, because it would effectively define away the issue raised in the post. The issue raised in the post is a real fact that applies to any and all natural communities. That it is a mathematical fact rather than an empirical fact does not make it an artifact. It seems to me that we should interpret our data with this fact at the front of our minds, rather than trying to come up with indices that try to hide this fact or define it out of existence.
I don’t think this mathematical fact makes it impossible to interpret data on community variability. For instance, it’s rare to find communities with strongly-synchronous dynamics, even though such dynamics are mathematically possible; that’s a suggestive empirical fact. But I do think community variability data are much more difficult to interpret than most people believe. I think by far the most important and robust conclusion of the relevant theory is that all sorts of different combinations of mechanisms can give rise to the same level of compensatory dynamics. Just as one simple example, a many-species community in which species exhibit independent fluctuations (mean pairwise correlation =0) is equally consistent with a community in which interspecific competition is very strong (however one chooses to measure it), and a community in which interspecific competition is completely absent. Unfortunately, this is an area in which many people have very strong intuitions that are contradicted by theory and theory-independent mathematical facts.
Hi Jeremy, I’ll have to think about some of the points you made before I reply but one quick comment – it’s not clear to me why it is only possible to have population fluctuations but no change in community abundance in a 2-species system. A simple thought experiment shows that’s not true.
Year 1: Species A – 50 individuals, Species B – 70 individuals, Species C – 90 individuals
Year 2: Species A – 90 individuals, Species B – 50 individuals, Species C – 70 individuals
And I’m comfortable calling that ‘perfect’ compensation because the populations (in sum) fluctuate in a manner that results in a community where the summed abundance is constant. It seems to me that the term ‘compensatory dynamics’ implies the idea that populations co-vary in a manner that results in the community having less variability than the average variability of the populations (i.e. changes population A ‘compensating’ for changes in population B so that the overall change in community abundance is buffered). Further, that implies that ‘perfect’ compensation would occur when populations co-vary in a manner that results in a constant community abundance. That said, I don’t care if we call it perfect or maximal or whatever. The key point is that the measure of ‘perfect’ compensation doesn’t vary with species richness – it is Community variance = 0, regardless of species richness.
Jeff H
Thanks for the clarification Jeff, I figured I had to be misunderstanding what you meant by “perfect” compensation.
Re: zero variability in total abundance as a useful benchmark, yes, possibly. But it still wouldn’t get you around the problems identified in the post. To continue with your thought experiment, if you went through the exercise of specifying hypothetical data that exhibited zero variability in total abundance, you’d find that as you went to communities with increasing numbers of species, that you were increasingly constrained in terms of what the data have to look like in order to achieve zero community variability. To have zero community variability, the summed covariance needs to be equal in magnitude but opposite in sign to the summed variance. That’s true by definition–the variance of the sum of a set of random variables equals the sum of the elements of their variance-covariance matrix, which is just the sum of the variances plus the sum of the covariances. As species richness goes up, you are going to be increasingly constrained in terms of precisely what features your data can have, in order for those data to exhibit summed covariances that precisely cancel the summed variances.
Hi Jeremy, I would disagree that you are more constrained as you increase in species richness. I’m not as confident of my footing here but I’ll go with another thought experiment. Let’s imagine a two species system where each species can take 1 of two values (let’s say, the values are 1 and 2 – I know this is completely artificial but I think it will illustrate the point).
Year 1: Species A – 1 individual, Species B – 2 individuals
There are only two possibilities in year 2 if the community abundance is going to stay constant
Species A – 1 individual, Species B – 2 individuals or,
Species A – 2 individual, Species B – 1 individuals
If we look at a 3 species community
Species A – 1 individual, Species B – 2 individuals, Species C- 2 individuals
There are 3 combinations that would result in constant community abundance. And the number of combinations would increase as species richness increased. I think this would be true even if we released species from the artificial ‘1 or 2 individuals’ constraint. In fact, I think the key point is that in natural communities with 5, 10, or 20 species and a reasonably large range of possible abundances for each species, the number of possible combinations that could result in
Summed variances = Summed Covariances
is going to be very large.
Yes, I wasn’t clear enough on what I meant by “constrained”. What I basically meant was the sorts of patterns of variances and covariances one can have as species richness increases.
Hi Jeremy. So, to address the issues you raised more comprehensively. I think if we can agree that compensatory dynamics have a logical midpoint (i.e. 0 covariance) and are bounded in ways that are not dependent on species richness (i.e. the lower boundary is when sum of variances = – 2 * sum of covariances and the upper boundary is when all populations are perfectly correlated) then we can identify some indices as less useful than others. For example, I think it would be desirable if the point where there is 0 covariance (i.e. 0 correlation on average between populations) is midway between the upper and lower boundary. This is not a characteristic of correlation so I think it is less than ideal. It would also be nice if the index was not dependent on species richness. This is not a characteristic of correlation. It would also be nice if the index was not dependent on community abundance and relative abundance of species within a community. I think this might be a characteristic of correlation (although I’m not absolutely sure). Of course, this all depends on the initial assertion that there is a logical midpoint and logical upper and lower boundaries that can be defined without knowing species richness or community abundance or, in fact, anything about the community. I do believe that (and I’ve described above what those boundaries are) but if one doesn’t, then coming up with ‘better’ indices than mean correlation probably don’t make sense.
So everything that follows is based on the assertion that there are these boundaries. You questioned whether this question of symmetry around zero is actually ideal because it would ‘define away the issue’. It’s not clear to me what issue this defines away. If there is a logical midpoint and an upper and lower boundary then it seems reasonable to prefer that the index is symmetrical around the midpoint (in this case, 0). The fact that correlation doesn’t do this suggests it’s not an ideal index. It is an artifact of the index not an inherent quality of the phenomenon we are trying to measure. The concept of compensatory dynamics has a logical midpoint and an upper and lower boundary and correlation does not capture this in an ideal way.
If mean correlation was what we were fundamentally concerned about then we would have to deal with the mathematically inevitable relationship between the lower boundary of correlation and species richness. But we are simply trying to find an index that captures ‘compensatory dynamics’. We have used correlation and it doesn’t work very well. Well, let’s find something better.
You’ve suggested that it would be more than ‘tricky’ to develop an index that is symmetrical and independent of species richness but I think it would actually be fairly simple to come up with something better (if not perfect). Here’s a quick and superficial stab. I believe the lower boundary that is independent of species richness is when community variance = 0, which is when sum of variances = -2 * sum of covariances. Therefore, an index for negative covariances, which would be bounded between 0 and -1 would be the 2 * the sum of covariances divided by the sum of variances. This doesn’t work for positive covariances because there is no clearly defined upper boundary for the index when covariances are positive. We could use mean correlations (perhaps weighted by abundance) for positive covariances although there is something a bit discomfiting about an index that is calculated one way for values between 0 and -1 and a different way for values between 0 and 1. A way that might be better is to use the empirical variances and estimate what the sum of covariances would have been if the species were maximally correlated. The index would be observed sum of covariances/estimated maximum sum of covariances. This still results in an index that is calculated differently between 0 and -1 and 0 and +1, which isn’t great. But, I think it would work better than correlation.
Your last paragraph takes on a much different issue. It’s captured by the statement ‘I think by far the most important and robust conclusion of the relevant theory is that all sorts of different combinations of mechanisms can give rise to the same level of compensatory dynamics.’ This is the point where I am always hesitant to put my thoughts in writing because tone is so important, but here goes – if, as ecologists, our most important and robust conclusion is that all sorts of mechanisms can give rise to the same level of compensatory dynamics, then we should be embarrassed. Our job is to describe how the world of living organisms works not the probability space in which it might work. This is the ‘The world is complex, all things are possible’ conclusion. This conclusion, while true, contributes almost nothing to what people know or would have known without ecologists. Further, I think we can do better. I won’t pretend that I have examined the detailed output of all the theoretical models that provide output on variances and covariances but I have examined some. You write that
Just as one simple example, a many-species community in which species exhibit independent fluctuations (mean pairwise correlation =0) is equally consistent with a community in which interspecific competition is very strong (however one chooses to measure it), and a community in which interspecific competition is completely absent
Really? Independent fluctuations are equally consistent with models with and without inter-specific competition? I could buy that they are possible in both contexts but ‘equally consistent’? The only models I have seen where there is strong inter-specific competition and positive covariance is where some other process that causes positive covariance is more important. So, yes, if you build a model with a strong common response to environmental changes and inter-specific competition then you can get positive covariances. That shouldn’t be a surprise. If you build in strong density dependence, that might result in positive covariances (although I’m not sure), but only because the model has been constructed so that intra-specific competition is more important than inter-specific competition. I have never seen a model that only incorporates inter-specific competition that does not result in negative covariance (although that may be a function of insufficient reading) unless we build in some very unlikely phenomena (e.g. small fluctuations in species A cause large fluctuations in the opposite direction in species B, C, and D species but the fluctuations of B,C, and D have no effect on each other) But, I think it is safe to say that in communities where species are competing there will only be positive covariances when 1. Some other process that causes positive covariances is more important than interspecific competition or 2. The way interspecific competition works is very counter-intuitive.
The world is a complex place but when that becomes our take home message then ecology is a science of the ‘possible’ rather than the ‘probable’. People don’t want to know what could be true, they want to know what is true or the next best thing, what is most likely to be true. So, if we see large negative covariance can we conclude definitively what is causing that? No. But there are two things we can do. One, we can make the fairly safe conclusion that ‘a common response to the environment’ is not the most important process governing population dynamics. Is that conclusion necessarily true? Maybe not. Is it likely true? Yes. Somebody may be able to develop a model where a common response to the environment is the most important process governing population dynamics and demonstrates negative covariance (although I am skeptical that it is possible) but I am almost certain that if somebody did, the world they would have built would have a bunch of assumptions and characteristics that would be very unlikely to be found in nature. Two, we can develop a list of potential processes and combinations of processes that might result in positive covariance and look for predictions that would distinguish among the competing hypotheses.
So, my main reason for responding to this post is that I disagree with the messages – I think the problem with correlations is a technical not a fundamental one and the take home message that the reason compensatory dynamics are rare in nature is mathematical isn’t correct (I don’t think). If we define compensatory dynamics as negative summed covariance then in communities with random fluctuations we should see negative summed covariances about 50% of the time. The lower boundary for the mean correlation would be small in a species-rich community compared to a species-poor community but the probability of negative covariance would not change – it would be 50%. And so, I absolutely do believe that the rarity of negative covariances can be used to say something about ecology. The reason we rarely see negative summed covariances (i.e. compensatory dynamics) is not mathematical or at least, not in the way it was described in the original post.
Hi Jeff,
Re: designing indices, I find it telling that, in taking even a tentative stab at producing an index with the properties you’d like, you are forced to define a very ad hoc index which is defined and scaled differently on either side of zero. That’s a symptom of the point of the post. I’m glad you’re uncomfortable with that, because I am too! I’m curious (seriously): did you expect to have to make the ad hoc choice to impose different scalings on either side of zero, and does being forced to do so make you inclined to rethink your views on what’s desirable in an index of compensatory dynamics?
Now, at some level I admit that what constitutes a “useful” or “desirable” index is going to be a to-may-to, to-mah-to kind of thing. I have an old post on how choice of index is often somewhat, or even entirely, arbitrary. So if you were to decide to go with one of the indices you’ve suggested, I might mildly disagree. But as long as your choice of index didn’t lead you into any serious errors I wouldn’t really be bothered by it.
You and I may have to agree to disagree on what ecologists should consider embarrassing; perhaps I’ve misunderstood you? I think the job of ecologists is to discover the truth (theoretical as well as empirical truths). If the truth is complex (e.g., because many possible mechanisms could explain the same observed pattern), then saying so is not only not embarrassing, it’s obligatory. If nothing else, it prevents empiricists from incorrectly interpreting what’s actually observed. You misunderstand what’s probable if you misunderstand what’s possible. I have various old posts that are relevant here, including this one. I’ll also note that, by choosing to ask appropriate questions, we can sometimes force a simple answer rather than a complex one.
“I have never seen a model that only incorporates inter-specific competition that does not result in negative covariance (although that may be a function of insufficient reading) unless we build in some very unlikely phenomena (e.g. small fluctuations in species A cause large fluctuations in the opposite direction in species B, C, and D species but the fluctuations of B,C, and D have no effect on each other)”
That statement surprises me, which makes me suspect I must be misunderstanding exactly what you mean, because off the top of my head I can think of some quite well-known and plausible models in which species just compete but yet don’t necessarily all negatively covary with each other. Perhaps if you say what models you are thinking of?
“Two, we can develop a list of potential processes and combinations of processes that might result in positive covariance and look for predictions that would distinguish among the competing hypotheses.”
Sure, but I’m having trouble thinking of papers off the top of my head that correctly enumerate all the combinations of reasons why we might see the patterns of covariance that we see in nature, and then do the experiments required to tease apart those possibilities. Doesn’t mean there aren’t any, but they’re at best a fraction of the large amount of empirical work on community variability. In my experience, it’s much more common for people to casually assume (and even to write papers explicitly or implicitly based on their casual assumption) that there’s a very simple mapping from pattern to process here. A common view, I think, is that negative covariances come from competition (meaning that, unless shared responses to environmental fluctuations intervene, every pair of species in a competitive community should be expected to negatively covary), positive covariances only come from shared responses to the environment and not any other factor, the effects of competition and environmental fluctuations are “additive” in some sense, and the way in which the effects of competition and environmental fluctuations can potentially “sum up” is independent of species richness. I think that common view is greatly oversimplified, in part for the reason outlined in the post and in part based on my reading of the theoretical literature.
Hi Jeremy, yes it caught me by surprise that I had to use different scalings but it only makes me a little uncomfortable. Well, actually I would have to play around with it a little and see what its properties were before I would decide how uncomfortable I was. And it makes up for some serious flaws that arise from using correlations. (And in retrospect I probably shouldn’t have been surprised – correlations work in a way I’m perfectly happy with from 0 to 1 but not from 0 to -1. I think this implies that we may have to do different things to one side than the other and I would say that it is not completely uncommon to run into mathematical techniques that require one approach for positive numbers and another for negative numbers although I’m on pretty shaky ground here because my mathematical training is pretty bad.). The key flaw being the lack of symmetry around 0. That may seem like a trivial issue except that I think it can result in your conclusion that the explanation for the rarity of compensatory dynamics is mathematical rather than biological and I’m not convinced that conclusion is correct. Because the minimum mean correlation is so close to zero in species-rich communities I think the implicit (but erroneous) assumption is that if population fluctuations are random, negative summed covariances will still be rarer than positive, but they won’t be. It would still be true that 50% of the summed covariances would be positive and 50% negative but the positive mean correlations would be distributed between 0 and 1 and the negative mean correlations would be crammed in between 0 and whatever the lower boundary is. So, by focusing on the properties of correlations we end up concluding that we should expect compensatory dynamics to be rare even where populations are fluctuating independently but I think that’s a mistake.
The key place where this can get confusing is that when I talk about positive and negative covariances I am always talking about summed covariances across all species pairs in a community. And so when I talk about 50% of summed covariances being negative and 50% positive that implies I am talking about multiple communities not covariances within a community (e.g. if we looked at fish communities in 20 lakes and in all lakes populations fluctuated independently we would expect 10 lakes to have positive summed covariances and 10 to have negative summed covariances with a mean around 0). That’s why my statement about models is confusing. I completely agree that lots of models that only include inter-specific competition can results in some species pairs co-varying positively but the summed covariances will always be negative. That is, on average species will co-vary negatively. And I define compensatory dynamics as dynamics where the variance of the sum (i.e. community) is smaller than the sum of the variances (i.e. populations). That seems like a pretty reasonable way to define compensation to me but others may disagree. Using this definition means that compensatory dynamics occur when the summed covariances are negative. Now, we have a tool that we can use to look at many different communities and ask how often do we see negative summed covariances (i.e. compensatory dynamics)? And if we rarely see negative summed covariances, I’m reasonably sure it is not because of any mathematical issue related to correlations – at least not as it was presented in the original post.
I realize that I’m a bit of a dog on a bone on this but I’m concerned about a message that states that we would expect compensatory dynamics (i.e. summed negative covariances) to be rare in nature because of the mathematical properties of correlations. I don’t think that conclusion is correct. But if we believe it, we don’t investigate the biological explanations for the rarity of compensatory dynamics because we don’t think biological explanations are required.
Jeff Houlahan
PS Thanks for providing the forum for this, Jeremy. There is an immediacy to this that is much more satisfying than waiting around for papers to get published.
Hi Jeff,
Thanks for making use of the forum. It’s good to have new voices, the bulk of the comments tend to come from relatively few commenters, most of whom are bloggers themselves.
Re: the bounds on negative correlation coefficients in this context, I don’t know that I have much else to say. It sounds like you’re a little less confident than you were before that this is just an uninteresting artifact that can and should be scaled away. I’d merely note again that there are mathematical bounds on the summed covariance too, it’s just that they’re set by the variances as well as by the species richness. It’s unclear to me why mathematical bounds on the summed covariance that are set by species’ variances are any more or less artifactual than bounds on the mean pairwise correlation that are set by species richness. I think if you keep thinking about it you’ll become equally comfortable with the various indices and converting back and forth between them, so that the species richness-dependent lower bound on the mean pairwise correlation will eventually stop making you uncomfortable. So when you say at the end that “if we rarely see negative summed covariances, I’m reasonably sure it is not because of any mathematical issue related to correlations,” that’s equivalent to saying “if we rarely see negative summed covariances, I’m reasonably sure it’s not because of any mathematical issues related to species richness and species’ variances”.
Let me clear up one minor misunderstanding: in general, mathematical (or non-mathematical) bounds on a quantity by themselves imply nothing about the distribution of that quantity among its possible values. I’m not making any implicit assumptions about this distribution, and I’m sorry if something I wrote suggested otherwise. Mathematically, it’s impossible for the mean pairwise correlation to be more than very, very slightly negative in a species-rich community. But it’s a purely empirical question where within the range of mathematical possibilities any given real community is likely to fall. As I think I noted in a previous comment, real communities in fact don’t seem to be distributed uniformly within the space of mathematical possibilities, which is an important fact. (Though in and of itself I don’t think this fact implies much about the underlying ecological mechanisms. As an aside, I suspect many empirically-oriented ecologists might be surprised by various aspects of the behavior of community variability in even quite simple competition models, if they just went and played around with simulations of those models.) A similar point could be made in other contexts. For instance, the fact that it’s biologically impossible to have negative population sizes does not in and of itself imply anything about the mean-variance scaling of population size. The fact that it’s mathematically impossible for gamma diversity (however defined) to be smaller than alpha diversity (however defined) doesn’t in and of itself imply anything further about the relationship between alpha and gamma diversity. Etc.
Which does actually lead to an important substantive point, which I probably should’ve included in the original post. Given that constraints on the possible values of variables, by themselves, don’t imply anything about the distribution of those variables within the space of possibilities, it’s rather difficult to say exactly how one would detect the effect of those constraints. It’s even difficult to say precisely what it would mean for there to be such an effect. Particularly since one can imagine that there could be “interactions” in some sense between the bounds, and the mechanisms generating the data. In the case of physical or biological bounds, those interactions can be tractable. I’m thinking of cases like Connelly’s work showing that hard bounds on species’ geographic ranges (e.g., land-ocean boundaries) combined with plausible individual-based models of birth, death and movement within the ranges (e.g., individuals “bounce off” the hard boundaries) only give rise to an extremely weak “mid-domain effect”. But in the case of mathematical bounds, while I do think those bounds are important and not mere “artifacts”, I admit it’s hard to say precisely how the bounds affect the data we actually observe.
Silly of me to miss an opportunity for a relevant self-citation: See Fox and Vasseur 2007 Ecol. Lett. for a model in which interspecific competition causes species that are subjected to environmental fluctuations to covary more positively than they otherwise would. And no, it’s not a weird model, or an ad hoc model specifically designed to generate positive covariances. It’s a very simple, standard competition model, and we weren’t expecting to find that result at all.
Click to access Vasseur%20and%20Fox%202007%20ELE.pdf
Hi Jeremy, thanks for pointing me to the paper. I’ve only had a quick read through it but it’s not clear to me how this is evidence against my point but it may be that I am missing something. So, presumably Figure 2a is the figure we should be focusing on here. How I interpret that figure is that, in the absence of environmental fluctuations competing species are perfectly negatively correlated. As you introduce environmental fluctuations the species covariation becomes less negative. That’s consistent with what I would have expected – that as the importance of environmental fluctuations became more important the signal of competition would get weaker. The interesting thing is that even when the response to environmental fluctuations is negatively correlated the signal of competition gets weaker. But when I think about that it makes sense – at the extremes the two competitors are perfectly negatively correlated so unless the environmental fluctuations are synchronised perfectly with the competitive effects then the addition of environmental effects will lower the negative corrrelation between the two competitors. Once again, as the importance of environmental fluctuations increases the signal of competition declines.
Your characterisation of these results as ‘ a model in which interspecific competition causes species that are subjected to environmental fluctuations to covary more positively than they otherwise would’ (I’m assuming that by ‘otherwise would’ you mean ‘in the absence of competition’) doesn’t seem to capture exactly what these results show (although, once again, it’s more than possible I’m missing something). I don’t see any results that are showing what the results would be without competition so it’s not clear to me how I can make any comment about what interspecific competition causes because competition doesn’t vary in the model as far as I can see. It seems to me that to make that statement with confidence you would have to have shown me how the species varied when they didn’t compete and when they did compete and show that adding environmental fluctuations made the competing species co-vary more positively than the non-competing species. What I believe you’ve shown here as that adding environmental fluctuations results in competing species co-varying more positively, which is what I would have expected. Jeff H.
Hi Jeff,
Thanks for taking the time to have a look at that paper. But no, sorry, that’s not a correct interpretation of the results.
If I understand you correctly, you’re appealing purely to the species’ covarying as negatively as possible with no environmental fluctuations, so that changing anything about the system–like adding environmental fluctuations of any sort, positively or negatively correlated–necessarily makes the species less negatively correlated. Sorry, but that doesn’t fully, or even mainly, explain what’s going on. It doesn’t explain why you keep getting less and less negative covariation (up to a point) even after you’ve moved well away from the lower bound.
More deeply, your story ignores the underlying mechanism we demonstrated in Fig. 3 and 4. You can’t explain any features of Fig. 3 and 4 based on your story. When species are subjected to an environmental perturbation–negatively or positively correlated–their transient response often is very positively correlated–it bumps up against the opposite bound. Why would that ever be the case on your story? Further, the nature of the consumers’ transient response depends on where they are on the limit cycle when the perturbation occurs, as well as on how positively or negatively correlated their response to environmental fluctuations is. Your story doesn’t explain that. The mathematical bounds on how the system can behave are the same everywhere on the limit cycle, so can’t explain variation along the limit cycle in how the system responds to perturbations. Your story also ignores effects on stability, which also arise from the mechanism demonstrated in Fig. 3 and 4. Fig. 3 and 4 go hand in hand with Fig. 2.
Bottom line: yes, mathematical bounds restrict how systems can behave. But this system behaves in a very specific way within the much broader range of possible dynamics permitted by mathematical bounds on its behavior.
Apologies if I’ve misinterpreted what you’re trying to get at. But even if I have, I think you’ve got the wrong end of the stick on this paper–your story just doesn’t sound to me anything like what’s actually going on in that paper. But it’s very early in the morning and I’ve got a fussy toddler here, so maybe my brain isn’t operating at 100% peak performance. 😉
Jeez, my brain definitely isn’t on. I also forgot to point out Appendix S1–did you have a look at that? There, we vary the strength of competition between the consumers, by introducing additional resources and varying how much they feed on shared resources vs. exclusive resources. That appendix shows that its competition between the consumers that explains the positive (or less negative) synchrony they exhibit when environmental fluctuations are introduced.
Hi again Jeff,
Thought I’d take a minute to pick out the key bits of that appendix, it’s long and some of it isn’t relevant to what we’re discussing.
Section ii(b) is relevant. That section considers a case in which, in the absence of environmental noise, the dynamics reach a stable equilibrium. So, no fluctuations in species’ densities at all, so it’s not meaningful to talk about synchrony. As you add environmental noise, the species exhibit persistent fluctuations because they’re repeatedly being perturbed, with each perturbation generating a transient dynamical response that is then interrupted by the next perturbation. Those transients tend to be synchronous initially, as in the main ms, no matter what the environmental correlation. And so the consumer cross correlation responds to increasing fluctuation strength in exactly the same way as in the main ms. Afraid I don’t see how your alternative story can explain that, as in the absence of noise, or with extremely weak noise, the consumer cross-correlation isn’t up against the lower bound.
Section ii(d) is relevant, and is the one I was thinking of when I suggested having a look at the appendix. This section shows that there’s a threshold strength of competition for consumers to exhibit anti-synchronous fluctuations in a constant environment. Weakly competing consumers fluctuate synchronously, not asynchronously, in the absence of environmental noise. And that’s despite the fact that they still share a predator, so still interact in a mutually-negative way via apparent competition. In other words, weakening competition doesn’t eventually lead to a point where the consumers covary in a weakly negative or independent fashion–it leads to a point where they covary positively.
This section also shows that, when consumers only compete weakly for resources, adding increasingly strong environmental noise (of whatever correlation) to the system makes the consumers fluctuate increasingly asynchronously with one another. Yes, with no noise, the system starts out close to the upper bound of maximally high synchrony. But again, why should increasingly strong noise keep moving the system further and further away from that upper bound? You need to be able to explain that, it’s not a trivial feature of the results. In general, when there’s a “hard” bound on the behavior of a dynamical system, and some factor pushes the system’s behavior away from that bound, it’s not necessarily the case that increasingly high levels of that factor will keep pushing the system’s behavior away from the bound.
Hope this helps, looking forward to any further thoughts you have. I’m flattered that you’re taking the time to dig into that old paper of mine.
Pingback: Old Oikos Blog posts are archived here | Dynamic Ecology
Pingback: ESA 2012 preview: talks to see on Thursday | Dynamic Ecology
Hi Jeremy, no, I didn’t see Appendix S1 – I’ll have a look at that.
Jeff.
Pingback: Help Jeremy decide what posts to write next! (plus a preview of coming attractions) | Dynamic Ecology
Pingback: Mathematical constraints in ecology and evolution, part 1: species can’t all covary negatively | Dynamic Ecology