Recently we polled readers on their views on some of the most controversial–or maybe just seemingly controversial!–ideas in ecology. Here are the results!
This is a long post, but you should stick with it. I found the results very interesting and I think you will too. Below the fold there’s an overview of the respondents, the results, and then some commentary.
Overview of the respondents
We got 251 responses (thanks everyone!), split about evenly between grad students (27%), postdocs (30%), and faculty (34%), along with a smaller number of other ecologists (10%). Obviously they’re not a random sample of ecologists, or even of readers of this blog, though they’re probably fairly representative of our regular readers. But they’re a big enough and diverse enough sample of ecologists to be worth talking about, I think.
I suspect the main way that they’re unrepresentative of ecologists more broadly is that they’re probably a bit more likely than randomly-chosen ecologists to agree with my views on topics I’ve blogged about. (Then again, some of the results suggest that’s not true…) They may also be a bit more likely to agree with one another than randomly-chosen ecologists. But I’m speculating on that.
Below are histograms of the distribution of opinion about each idea. Responses are on a 5-point scale from 1 (definitely false) to 5 (definitely true). Respondents were asked to skip any idea with which they were completely unfamiliar. Respondents also were asked to indicate their level of expertise about each idea, choosing from the options “I’m an expert”, “I know some”, “I know a bit (e.g., learned it in a class)”, and “I know nothing”. In the histograms, I omitted the opinions of anyone who indicated they knew nothing about the idea (those people weren’t supposed to provide an opinion, and when they did it was usually “3”). I color-coded the opinions of others by their level of expertise: light blue for experts, blue for people who know some, dark blue for people who know a little. Note that the y-axis scale varies among histograms, because the number of respondents varied among ideas.
As best I can tell, respondents honestly evaluated their own expertise. Nobody claimed expertise on most or all of the ideas (which would be implausible), a different set of people claimed expertise on each idea, and only a minority of respondents claimed to be experts on any given idea.
I’ve ordered the ideas roughly from most to least controversial. More controversial ideas are those with a mean opinion near 3, and with a high variance in opinion, especially a bimodal distribution.
So without further ado, the most controversial idea in ecology (well, at least among these respondents, from the choices I provided) is….
Species interactions typically are stronger and more specialized in the tropics!
In second place (or first place, if you prefer to quantify “controversy” via variance of opinion rather than bimodality of opinion): the idea that local biodiversity is declining in most or all localities:
Aside: I am proud that the two most controversial ideas in ecology are ideas we’ve blogged about. 🙂
Next, several ideas that are all quite controversial, but that you could rank in different orders depending on how exactly you quantified “controversial”:
Species’ poleward geographic range limits typically are set by abiotic factors, not species interactions:
(Aside: I did not expect ideas about range limits to be among the most controversial ideas on the list.)
The intermediate disturbance hypothesis (empirical version): species diversity typically peaks at intermediate disturbance frequency or intensity:
The intermediate disturbance hypothesis (one theoretical version): disturbances promote diversity by interrupting competitive exclusion and preventing the community from achieving equilibrium:
Biodiversity is among the most important determinants of ecosystem function:
Habitat fragmentation per se (as distinct from habitat loss) typically reduces biodiversity:
(Aside: that is a weird distribution of opinion–a very clear mode somewhere other than “3”, but also a high variance.)
Interspecific competition can be reliably inferred from information about functional trait similarity:
(Aside: that is another weird distribution of opinion.)
Species diversity typically peaks at intermediate productivity:
Next, in rough descending order, some ideas I’d call moderately controversial.
Species richness on continents is dominated by ecological limits, as opposed to evolutionary limits:
(Aside: apparently, the ASN debate on limits to continental-scale species richness was a tie in the minds of ecologists. 🙂 )
Higher biodiversity will buffer ecosystem function against climate change:
Predator functional responses typically are ratio-dependent:
(Aside: that last graph shows the biggest split between expert and non-expert opinion on the list. I’ll come back to that.)
Top-down effects typically are stronger and more important than bottom-up effects:
Species’ equatorward geographic range limits typically are set by species interactions, not abiotic factors:
The dilution effect: host biodiversity generally reduces disease prevalence or infection risk:
Ecological networks (e.g., food webs, plant-pollinator networks) have the structures they do because other structures would be unstable:
The enemy release hypothesis: invasive species often spread and become abundant relative to natives because enemies attack the invasives less:
And finally, in rough descending order, some ideas that are not particularly controversial:
Hubbell’s neutral model: community dynamics typically are dominated by drift, because all individuals of all species typically are demographically equivalent:
Interspecific competition and facilitation can be reliably inferred from co-occurrence matrices or correlation matrices of species’ abundances:
(Aside: that last one and the next one are the two for which I wonder the most if there’s more consensus among our readers than there is among ecologists as a whole. But that’s pure speculation on my part.)
Interspecific competition can be reliably inferred from information about phylogenetic relatedness:
(Aside: Is that last one a sign that the phylogenetic community ecology bandwagon–or at least, one big important piece of it–got stopped in its tracks by criticism?)
Population growth typically is density-dependent:
And finally, the least-controversial idea on the list: Earth is currently experiencing a 6th mass extinction:
Experts vs. others
There are some very interesting patterns in the views of experts. vs. non-experts. Below is a scatterplot of mean expert opinion on each idea vs. mean opinion of everyone (including experts). Each point is one idea, the line is the 1:1 line.
Expert views correlate quite well with everybody’s views; there’s no idea in this poll on which experts as a group have a massively different view than everyone else. And since experts are only a small minority of respondents for any idea, that positive correlation mostly isn’t an artifactual part-whole correlation. The positive correlation would still be about as strong if I was less lazy and plotted mean expert opinion against mean non-expert opinion.
But here’s the really interesting result: experts think ideas are more likely to be false (or less likely to be true) than do all respondents. That is, the points in the previous graph mostly fall below the 1:1 line. It’s generally not a huge effect, but it’s so consistent across so many different ideas that it’s clearly a real trend. Surprisingly, the trend holds for ideas that the crowd tends to believe, for ideas the crowd tends to disbelieve, and ideas for which the crowd is on the fence. So it’s not that experts are generally more cautious than non-experts–it’s not that mean expert opinion is always closer to 3 than mean crowd opinion. Nor is it that that experts are always more certain than non-experts–it’s not that the experts choose 1 when the non-experts choose 2 and the experts choose 5 when the non-experts choose 4. I did not expect this result, and I’m not sure what to say about it. Really looking forward to your thoughts on this result!
The next plot shows the variance in expert opinion about each idea vs. the variance in everyone’s opinion. Again, the line is the 1:1 line.
As you might expect, there’s a positive correlation, though it’s pretty noisy. Ideas that are controversial among all ecologists tend to be controversial among experts.
But here’s the interesting thing: expert opinion about any given idea almost invariably exhibits greater variance than does all opinion. I don’t think this is just a reflection of the smaller number of experts, at least not in all cases. If you estimate a population variance with a sample variance, your estimate is not biased upward by a small sample size. And in the most extreme cases, the high variance of expert opinion aligns with my own impressions of the literature, so I don’t think we’re just seeing sampling error here:
- Expert opinion about ratio-dependent functional responses in the poll is pretty bimodal, with the larger group of experts rubbishing the idea. That aligns with my subjective impression of the field, that expert opinion is split between a larger group of people who share Peter Abrams’ very negative opinion of ratio-dependent functional responses, and a smaller group of people who share Roger Arditi and Lev Ginzburg’s very positive opinion.
- Expert opinion about the ubiquity of local biodiversity declines is quite bimodal in the poll, with the larger group of expert respondents thinking that local biodiversity declines are not ubiquitous. The opinion of non-experts seems to skew a bit towards the opinion of the larger group of experts in this case. The poll results align with my impression that expert opinion on this issue really is divided into opposing camps. FWIW, in my view the experts arguing against ubiquitous local biodiversity declines (Mark Vellend, Maria Dornelas, our own Brian McGill, et al.) have much stronger evidence and arguments than the opposing experts (Brad Cardinale, Andrew Gonzalez, et al.).
- Expert opinion about the intermediate disturbance hypothesis also looks fairly bimodal, with one group of experts who think it’s definitely false and another group who think it might have some truth to it (hardly any expert respondents think it’s definitely true). I presume that’s a split between experts who think about the IDH more or less as I do, and those who think about it more or less as Doug Sheil and David Burslem do.
- Expert opinion about whether species interactions typically are stronger and more specialized in the tropics also is quite bimodal, although the two camps merely lean in opposing directions rather than being confident the idea is true or false. That’s the only idea on the list for which non-expert opinion also is bimodally distributed; no idea why.
- Expert opinion about the ubiquity of humped diversity-productivity relationships also is pretty bimodal. I’m not surprised, that’s a longstanding vociferous debate in the literature (see here and here).
- Our poll found a particularly high variance in expert opinion about whether Earth is in the midst of a 6th mass extinction. Not sure what to make of that. On the one hand, our poll respondents didn’t include many experts on that idea, so the high variance could well be a blip. On the other hand, the idea is at least somewhat controversial–prominent paleontologist and mass extinction expert Doug Erwin has publicly criticized the idea. I’d be very curious to know if the experts in our poll who disagreed with that idea are mostly paleontologists or people familiar with the paleo literature, and if the experts who agreed mostly work on extant species.
- There was one idea for which I expected a high variance of expert opinion, that didn’t show up in our poll: the dilution effect. I think most people familiar with that literature would say there are two opposing camps of experts on that issue, but you can’t see those camps in the poll data. But very few experts on the dilution effect completed the poll, so we didn’t have much power to detect bimodality of expert opinion on that idea.
- Hubbell’s neutral model is the only idea polled for which the variance in expert opinion is much lower than the variance in overall opinion. Expert opinion in our poll agrees quite strongly that community dynamics are not typically dominated by neutrality+drift. Which raises the question, why isn’t there an opposing camp of experts who think the world is dominated by neutrality+drift? Did the poll just miss them? Does Steve Hubbell not read this blog? 🙂 Or is there some more interesting reason? Speculation: perhaps no expert, not even Steve Hubbell or Graham Bell, ever really believed that the world is really neutral. Rather, maybe even neutral theory’s developers and staunchest proponents saw it not as true, but rather as a thought-provoking null model to consider, or as an approximation useful for certain limited purposes (e.g., explaining the shape of the species-abundance distribution). Maybe you only end up with polarized opposing camps of experts if you have polarized opposing camps from the get-go? That is, further research might sometimes fail to reduce polarization of expert opinion, but never creates it?
So one take-home message of this poll is that, if it seems to you like ecological opinion about topic X is divided into opposing camps, well, that’s probably just expert opinion. The bulk of ecological opinion on even the most controversial topics is hardly ever bimodally distributed. Presumably because non-experts who lack the time and/or inclination to do a deep dive into the literature on a controversial topic just withhold judgment or have vague mixed feelings. Which as an aside is perfectly rational of them. Nobody has time or a good reason to come to a deeply-informed opinion about every issue in ecology!
- Most every idea on this list was at least slightly controversial, meaning that every choice from 1-5 was chosen by someone. Proof, if proof were needed, that disagreement is a normal part of science. So if it makes you anxious to know some ecologist out there somewhere disagrees with you on X, well, sorry but you’d better get used to it! Note that this does not mean that every idea in ecology is controversial. I chose most of these ideas because I thought they might be controversial. I didn’t include ideas that I’m sure are non-controversial, like “species richness typically increases with area sampled”.
- But just because disagreement is a normal part of science doesn’t necessarily mean it should be. After all, why should any idea be controversial, at least among experts? If the available evidence and arguments are incomplete or point in different directions, shouldn’t everyone who’s aware of all the evidence and arguments have chosen “3”? Indeed, there’s an old joke that the answer to every question in ecology is “it depends”. So how come there’s no question in this poll for which respondents mostly agree that “it depends” is the correct answer? And how come people who know more about the issue typically disagree more than people who know less about it? Maybe the disagreements revealed by this survey, are a symptom of the fact that we all have our own intellectual biases, and that those matter most when we’re thinking about the issues we care most about–i.e. the issues on which we’re experts. From one perspective that is a depressing thought; arguably scientists could and should be more objective than that. But on the other hand, one could argue that is science as a whole actually better off with some level of controversy, even controversy arising from different scientists having different intellectual biases, their own “pet” hypotheses, etc. Arguably, science as a whole doesn’t need individual scientists to be intellectually unbiased, it just needs effective procedures for selecting among the views of different opposing camps. Maybe the variation in opinion among scientists provides the raw material for a process analogous to natural selection to drive scientific progress.
- Trouble is, that selection process seems to be pretty slow and not proceed to “fixation” in many cases. Ecologists are reluctant to say that an idea is definitely true. I’m struck by the number of ideas for which the modal response was 4, but for which 5 got few or no responses. There was a similar reluctance to choose 1, but that reluctance was evident for fewer ideas. I think this illustrates Brian’s old complaint that ecologists lack a “problem solving mentality”, and my complaints about “zombie ideas”. Ecologists are reluctant to ever completely rule an idea in or out. In some ways, that shows admirable intellectual caution. But the downside is that ecological ideas tend to live on long past the point when you’d think the field would’ve settled the issue.
- Case in point: BEF. Shouldn’t ecologists collectively be kind of embarrassed that, after decades of BEF being perhaps the single most intensively-researched topic in the field, that there’s no consensus on an empirical question as basic as “biodiversity is among the most important determinants of ecosystem function”? Another case in point: the IDH. The IDH has been around for 40 years. It’s in all the textbooks and is the subject of uncountably many papers. Isn’t it embarrassing that simple descriptive statements about the IDH are still controversial? Or perhaps not. Perhaps the timescale on which we should expect to be able to definitively answer these sorts of questions is centuries, not decades? How quickly should ecology, or any field of science, be expected to progress? If science progresses by a process analogous to evolution by natural selection, how long should we expect it to take for an idea to “go to fixation”?
- Speaking of the IDH, I’ve been banging on against the IDH online for 7 years, and I also have a widely-read TREE paper that grew out of those blog posts. And yet a plurality of my own readers still mostly agree with the IDH! Assuming that the respondents weren’t just trolling me (and if you were, well played :-)), that illustrates that you shouldn’t overestimate the influence of this blog–or any one person or paper–on ecologists’ collective thinking. I know that some ecologists don’t like my “zombie ideas” rhetoric, which is fair enough. But if the reason you don’t like it is because you think this blog is massively influential, well, you’ll be glad to know this blog is not nearly that influential. I cannot appreciably shift ecologists’ collective thinking about an idea merely by calling that idea a zombie. Even though I wish I could! 🙂 (I’m kidding, I’m actually glad neither I nor anybody else has that much power.) More broadly, influential individual scientists have some power to generate discussion, but no individual has the power to dictate the field’s collective thinking. (Aside: I don’t know if many people actually worry about this blog being overly influential, so maybe this bullet is attacking a straw man.)
- My own views mostly agreed with the crowd’s modal view, or were only one step removed. But there were some exceptions, and I confess I find them some combination of surprising, frustrating, worrisome, and depressing (What can I say? I’m me.) The biggest exception was the least-controversial idea on the list, that the Earth is experiencing a 6th mass extinction. I’m not an expert on this one, but FWIW I’m pretty convinced by Doug Erwin’s arguments against that idea. I really worry that respondents are expressing a much stronger consensus in favor of a currently-ongoing 6th mass extinction than can be justified by the empirical evidence. And “1” (definitely false) is the correct view on that theoretical version of the IDH, but was chosen by only a minority of respondents (Chesson & Huntly 1997, Fox 2012, and see this blog post, this one, and the series that starts here). More broadly, the degree to which an idea is controversial doesn’t line up all that well with my own views. The consensus occasionally is wrong, some ideas that are quite controversial really should not be, and others that aren’t really should be! I admit that bugs me (I know, I know, I should remember my own advice–disagreement is a normal part of science…) Anyway, I may do a follow-up post where I reveal my votes and explain my reasons for them, with links to key citations.
In conclusion, really looking forward to your comments on this one. I am especially interested to hear from readers whose views are far from the consensus on any of these ideas. Genuinely curious why you think as you do, especially if your views place you in a minority.
One thought about the “6th extinction” debate: the answer certainly depends a lot on how you think of it in terms of currently vs. eventually and evidence vs. belief. If the question was : “do we have empirical evidence that we are already experiencing a mass extinction similar to the five found in geological records” I suppose there would have been a vast majority of one. However, if people answered thinking of the question as “will the currently observed biodiversity decline eventually turn into a mass extinction”, they might be more willing to consider that there is no hope for reversal such that earth’s ecosystems will all eventually collapse in the long term (and then for sure we’ll be in big trouble…).
Yes, in general some of the controversy the poll turned up is surely due to people interpreting the stated ideas differently. But in the case of the 6th mass extinction, I doubt that’s the main explanation for why most people chose 4 or 5. The poll asked whether Earth is “currently experiencing” a 6th mass extinction, which to my mind is just a briefer version of your “we are already experiencing a mass extinction similar to the five found in geological records”. I could be wrong, but I doubt switching from the former phrasing to the latter would’ve caused most votes to switch from 4 or 5 to 1.
The phrasing of “currently experiencing” is difficult when the phenomenon (mass extinction) will likely take hundreds or thousands of years or longer. If the extinction rates we are currently experiencing continue, then yes, the earth will achieve a mass extinction at some point in the not too distant future (and thus we will have been “experiencing” it). This paper (by biologists and paleontologists) explores the rate issue and amount of time needed to accumulate enough extinctions to qualify as “mass”:
The conclusion being that if all species currently classified as critically endangered, endangered, and threatened go extinct in the next couple of hundred years (a definite possibility), we will get to the 6th mass extinction (Figure 3 is especially informative). So I wouldn’t say that all paleontologists agree with Erwin. (Erwin studies the Permian extinction which is the worst extinction on record. It seems we’re looking at more of a Cretaceous, dinosaur-ending mass extinction.)
Although conservation has made progress and the extinction rate is likely slower than it would be in its absence (http://science.sciencemag.org/content/early/2010/10/26/science.1194442), it’s unlikely that the current pressures on species will diminish in time to avert a mass extinction.
FWIW, I’m a conservation biologist!
Yes, I thought the same thing…!
I’m not surprised that “species interactions typically are stronger and more specialized in the tropics” is the most controversial topic as it’s something that’s really only started to be challenged in recent years. However I think that the evidence is clear and there are many examples where particular types of interactions are not stronger or more specialized, and you have to look at it on a system-by-system basis. I may be wrong but my feeling is that those scoring 4 or 5 on that topic would be people who define themselves as “tropical ecologists” with little or no experience of working outside of the tropics.
“I may be wrong but my feeling is that those scoring 4 or 5 on that topic would be people who define themselves as “tropical ecologists” with little or no experience of working outside of the tropics.”
That raises the broader issue of whether there are covariates that would explain controversy about other ideas. In the post I speculated that people who believe we’re in the midst of a 6th mass extinction might be mostly ecologists unfamiliar with paleontological data on mass extinctions, while the rare skeptics might be those respondents familiar with paleontological data.
Not sure about some of the others though. For instance, what’s the covariate that would explain controversy over the effects of habitat fragmentation on biodiversity? I have no idea. And I don’t see any obvious covariate that separates opposing camps regarding the ubiquity of local biodiversity declines.
For habitat fragmention, I could imaging that it is due to phrasing of the questions. You used per se in the orginal definition “on its own”, but at least in Germany it is also used as “in every case” “as a rule”. Maybe this caused some of the variance.
Possibly. Though we have very few readers and (presumably) very few poll respondents from Germany. So I doubt that the poll results would be skewed much by German respondents misunderstanding the phrasing of the question.
I don’t think it’s all “chiefly tropical biologists!” I have done both. Schemske has done a heck of a lot of both. Did you see our recent paper on Phytolacca americana in the eastern US? Strong support for the biotic interactions hypothesis, not tropical. Some tropical data are forthcoming and provide more support 🙂
Perhaps not, but then I’d be surprised if you or Doug had selected 4 or 5 for that question in that poll because you know the literature and you know that there are as many studies supporting one side of the argument as there are supporting the other side! So the response to “species interactions typically are stronger and more specialized in the tropics” has to be a 3 or a 2, depending on how one defines “typically”.
Yes, I saw the Phytolacca study, it’s very interesting, really nice piece of work. But as you say, the latitudinal range was modest and did not include the tropics. Plus, if I am reading your results correctly, populations at lower latitudes did not have fewer species of associated herbivore. Is that correct? In which case lower latitude populations are not more specialized as one would predict from the overall hypothesis. So again the evidence is mixed here.
Actually I definitely would have chosen a 5 😉 I think if you accept that everyone has tested the hypothesis equally well, it’s true there are a lot of non-supportive studies, but there have been a lot of limitations to approaches people have been using, and tests of the hypothesis that are truly outstanding and comprehensive tend to be more supportive. To some, paying more attention to certain studies seems like bias…to others, it is inappropriate to assume that all data are created equal.
I appreciate you reading my Phytolacca paper! Single-species studies of plants have to have “modest” latitudinal ranges because even wide-ranging species are not all over the globe. I will publish this year on Phytolacca rivinoides, a tropical congener, to extend the gradient to the tropics.
I’ve never thought that a gradient in specialization is a key part of the hypothesis, and that’s not something I was explicitly testing with this study. Good point to think about! I’m not really sure how one would glean that information from the data I have, but I’ll try! This is cool!
South of Georgia, four species were collected, and 95% of the collections were specialists (291/306).* North of Georgia, three species were collected (one falls out quickly north of Georgia), and 38% were specialists (13/34).* Does that seem like a good metric? I wish I had thought to put it in the paper!
It’s great to hear your thoughts and challenges! Thank you for engaging with me!
*These numbers assume that an ambiguous morphospecies turns over completely from Tennessee to Kentucky…everything we reared out from Tennessee south was specialized, and everything north of Tennessee that we reared out is generalized. “Specialist” means it eats a couple families in Caryophylalles, and “generalist” means it’s highly polyphagous. Not much is known yet about these herbivores–there were a lot of new host records! Yay new systems!
“tests of the hypothesis that are truly outstanding and comprehensive tend to be more supportive. ”
Is there a review paper that’s demonstrated that?
“tests of the hypothesis that are truly outstanding and comprehensive tend to be more supportive.”
As Jeremy notes, this needs to be properly reviewed and assessed because at the moment the argument sounds like “the studies that support my hypothesis have been well done; the studies that don’t support my hypothesis have been poorly done”. God forbid I should invoke the term “fake news” and all its connotations, but that’s how it comes across 🙂
All studies have their weaknesses and can be criticised on multiple levels, but ignoring work that doesn’t fit one’s world view is a dangerous strategy because one gets backed into an intellectual corner from which it’s hard to emerge.
Specialization: we’ve discussed this before and whether or not this is linked with the “intensity” part of the hypothesis is open to debate. But in its simplest form one could look at number of species involved in the interaction (which is what we have done with plant-pollinator data, correcting for sampling effort). This is an ongoing discussion in the literature: how does one measure “specialization”? Definitely worth thinking about for your plants though. However I would not define a herbivore that “eats a couple families in Caryophylalles” as a specialist! Herbivory can be highly species-specific so on a scale of specialist to generalist, that example is more to the generalist side.
There are some species that have temperate to tropical distributions that could make interesting systems to assess in the future. Bracken (Pteridium aquilinum) is one of them, but there are others, including in Australia.
Good luck with your thesis defence.
There is a continuum of subjective opinions on what constitutes a good test of a hypothesis, and yours is as subjective as mine but clearly different. It’s not unique to this issue that people disagree about how to test hypotheses. I believe that it’s more constructive to agree that these are differences in opinion than to compete for claims of superior objectivity. The tone of these conversations would be quite different, and frankly a lot more enjoyable and welcoming (dare I say, particularly for young scientists and URMs), if we shared that premise.
Daniel Anstett, Krystal Nunes, Peter Kotanen and I reviewed in TREE the many intricacies of why some approaches may be better than others (https://doi.org/10.1016/j.tree.2016.07.011). This is the closest thing to reviewing the quality of studies on the subject, but we did not attempt to set some kind of quality threshold and re-do a meta-analysis because we have different ideas of what that threshold should be. For example, I would argue that everyone should be measuring young leaves, which means I can count the herbivory studies on one hand that I consider proper tests of the hypothesis, but that’s not an opinion that my co-authors share. Another example: a quick/dirty/conservative way to dissect the lit on latitudinal patterns in defense would say that only bioassays count as good measures of defense because its rare that we understand so-called defensive traits well enough to be certain that comparing traits is biologically meaningful. And bioassays do support latitudinal patterns more than measures of traits—see Angela’s meta-analysis.
It was a great experience to think deeply with these folks to evaluate the pros and cons of a range of approaches. After all that, I would choose a 5 for your opinion poll, based on my opinions of what is a really good test, and based on evidence from less controversial interactions than herbivory. Considering and judging the available evidence to form an opinion is not the same thing as “ignoring work that doesn’t fit one’s worldview” or saying that “the studies that support my hypothesis have been well done; the studies that don’t support my hypothesis have been poorly done” or “fake news.” We just have different opinions for how to consider and judge the available evidence for this complex topic that isn’t fully understood.
Re: specialization in Phytolacca, sorry I wasn’t clear that Phytolacca americana is the only member of its family in North America, so its not that weird that even its relatively specialized herbivores are spread among multiple families. The “specialists” are recorded to eat 3 or 7 host species (I think they are saponin specialists, but not sure! Exciting! Like you say, kind of hazy to define specialization), versus the dozens of hosts for the “generalists.” If you counted the one eating 7 hosts as a generalist instead, to be conservative, the proportion of consumption by specialists becomes 46% in the six southernmost sites versus 0% in the four northern sites. Thank you for prodding me to quantify specialization as best as I can without a lot of information available about the herbivores.
PS Dissertation deadline is in a few weeks, so if I drop out of the conversation, it’s due to lack of time rather than desire for constructive dialogue.
“There is a continuum of subjective opinions on what constitutes a good test of a hypothesis,”
And there are are also objective ones, right? For instance, in this context larger sample sizes and longer latitudinal gradients are objectively better, right? I could certainly imagine someone doing a meta-analysis of latitudinal gradients in seed herbivory (or whatever) and using sample size and latitudinal range covered by the study as covariates.
Also, if everybody’s opinions as to which tests are better are “subjective”, I guess I’m unclear why we’d bother talking to each other about them. Can you elaborate? I mean, if you preferred a different ice cream flavor than me, there’d be no point in us discussing that, it’s just a subjective personal preference. Professional judgments as to how to interpret studies of the latitudinal gradient in species interactions arent’ completely objective in the same way that a judgment that 2+2=4 is completely objective. But surely they’re objective enough to be worth talking about, right?
“I believe that it’s more constructive to agree that these are differences in opinion than to compete for claims of superior objectivity. The tone of these conversations would be quite different, and frankly a lot more enjoyable and welcoming (dare I say, particularly for young scientists and URMs), if we shared that premise.”
Can you elaborate on your suggestion that we agree that there are differences in subjective opinion? Are you suggesting that scientists should agree to disagree about the latitudinal gradient in species interactions, full stop? And if so, what would you suggest that amount to in practice? For instance, do you think that opposing camps on latitudinal gradients in species interactions should agree to stop reviewing each other’s papers? Honest questions, not trying to put words into your mouth. Genuinely interested for you to expand further on the suggestion that scientists should agree to disagree on any aspect of a controversial scientific issue. That’s a response to scientific controversy that I’ve rarely seen expressed, and so I’m interested in it.
One problem with saying “let’s all agree that there are differences of opinion and that they’re purely subjective” is that it amounts to saying that, if a hypothesis is so vague that it gets interpreted or operationalized differently by different groups, well, that’s fine. Everybody can just agree to disagree on what the hypothesis is and how to test it. It’s not clear to me how hypotheses get refined or operationalized in a world in which everyone agrees to disagree. What are your thoughts on this? This is an issue I worry about a lot, outside the context of the latitudinal gradient in species interactions. My own impression is that ecologists in general have a rather poor track record of operationalizing vague concepts or hypotheses in way that everyone can agree on and lead to substantial progress (https://dynamicecology.wordpress.com/2017/02/15/name-the-most-successful-and-unsuccessful-examples-of-operationalizing-vague-or-abstract-ecological-concepts/).
And speaking as an ecologist and Am Nat handling editor who doesn’t work on latitudinal gradients in species interactions (and so doesn’t have a horse in this race), if leading researchers in the field (like you!) say that the ongoing controversy on this topic is just a matter of subjective differences of opinion on which we all should just agree to disagree, well, I confess I’m not clear why journals should keep publishing *anyone’s* work on this topic. I’m thinking back for instance to David Ehrenfeld’s decision, back when he was EiC of Conservation Biology, to stop publishing papers on the SLOSS debate because nobody had anything new to say and it was just authors repeating their own personal opinions at each other. But I guess I must be misreading your remarks, since I’m sure you wouldn’t recommend that journals stop publishing on the topic you work on! So I would welcome elaboration if you get a chance.
Carina, as always I very much appreciate your comments on this topic; thank you for taking the time with your defense fast approaching. Putting on my moderator hat, I want everyone who has something to say about any post topic to be confident they’ll be treated professionally. I’ve gone back and reread the entire thread. The tone looks fine to me. I can perhaps see hints of a bit of mutual frustration creeping in on your part and Jeff’s, but nothing out of line. If you have any concerns that you’d prefer to raise privately, you’re welcome to email me (firstname.lastname@example.org), or email Brian and/or Meghan if you feel that my own comments are problematic.
Hi Carinna – apologies if my tone came across as snarky, it wasn’t meant to, I was just trying to make a comparison, albeit clumsily.
Jeremy’s comments about subjectivity/objectivity nailed the issue I think, so I don’t have anything else to add to that.
But here’s where I see a problem. The poll statement was “Species interactions typically are stronger and more specialized in the tropics” not “Herbivory is typically stronger and more specialized in the tropics”. Now, there may indeed be issues with previous studies of latitudinal gradients in herbivory, I really can’t judge that, I don’t work on that topic. That indeed is an area where more consensus could be built and techniques improved, and in the future we might find that you are correct. But by scoring “Species interactions typically are stronger and more specialized in the tropics” as a 5 you are saying that there are issues with _all_ types of studies of strength of interaction that don’t support the latitude-biotic strength hypothesis, e.g. predator-prey, host-parasite, mutualism in its many forms, commensalism, etc. etc. Not withstanding that it’s also referring explicitly to specialization (which I do study and which, I can assure you, the findings are highly variable depending on study system). That seems to me to be a very extreme position to take, if it’s really what you think.
I’m sure your defence will go well, there will be lots to discuss!
I am certainly not suggesting that we just agree to disagree, and thus science grinds to a halt. This was an OPINION poll because we don’t know the answers to these questions yet. Admitting that both sides have valid “subjective opinions,” based on different readings of the literature, is not shutting down a conversation. It’s not giving up on discussion or getting data or writing papers or reviewing each others’ stuff. It’s opening up a conversation, for everyone to explain the many reasons and many forms of evidence that have formed that opinion. It is hard to imagine changing opinions of, say, me/Jeff/Angela, without another decade of strong evidence that skews one way or another…but if that decade happened, then hopefully minds would change! (And remember…what people consider “strong” evidence is also an opinion.) Nevertheless, agreeing that we disagree has not halted progress. The conversations in the literature from many voices have been really good for moving our field forward, and that would not change if we started a conversation with the premise that we all have “subjective opinions” about this complex topic.
What I’m saying is that more voices would feel welcomed to join the dialogue if the premise of or tone was “here is a controversial topic, on which people have disagreements and strong, divergent opinions, and here’s why” versus the current framing, which is “here is what objective people think, and everyone else is dogmatic/biased/loves zombies/spouts fake news.” The former invites dialogue and welcomes conversation. The latter is confrontational, requires defensive posturing, and feels bullying when you’re on the wrong end of the power dynamic (e.g., you’re junior/URM, not a gatekeeper/tenured professor/senior scientist). The idea that a young scientist would feel comfortable accusing a senior scientist of “fake news” or bias or dogma is LUDICROUS. As stubborn, outspoken, and thick-skinned as I am for a grad student, I would NEVER DREAM of saying something like that. But you guys can do it. And I really don’t have time to go back and pull out all the instances of this, but look back at the zombie idea guest post from a few years ago and the highlight of Angela’s “dogmatic” paper. Even if you guys never literally said “we are objective and everyone else isn’t,” that’s the implication of words like dogma/biased/zombie/fake.
How about have a poll that says, “If a famous blog in your field proposed a discussion of different viewpoints on a controversy in your tiny subfield, would you feel comfortable contributing to the discussion or comments?” and then ask “If a famous blog in your field staked a strong position opposite yours on a controversy in your tiny subfield, and then said anyone who believes otherwise is dogmatic/biased/loves zombies/spouts fake news, would you feel comfortable contributing to the discussion or comments?” Betcha more people would feel comfortable with the former. Especially young URMs. I just happen to be stubborn enough to say yes to both, apparently, but I really don’t enjoy the latter. Tone matters to people, and affects the voices that participate. I don’t think this sentiment is unique to me or this thread.
Jeff, like you, I have not read and absorbed every single paper on all biotic interactions and latitude. Wasn’t even aware that there was a vast literature on latitudinal patterns in commensalism, for example, so maybe you could share that library with me. These polls show that experts regularly choose 1 or 5. It’s not that weird.
We haven’t polled readers on precisely the question you suggest, but we could. In the past I’ve done posts inviting reader feedback on the tone of my own writing and my use of rhetoric (e.g., https://dynamicecology.wordpress.com/2013/04/17/on-the-tone-and-content-of-this-blog-feedback-encouraged/), but never a poll. A poll would have the advantage of making it easy and quick to respond anonymously. Thank you for the suggestion, I’ll take it up with Brian and Meghan.
We have polled on related questions, and compiled some related data, which I’ll take the opportunity to summarize and link to for any readers who are interested:
It is true that our commenters skew more senior than our readership: https://dynamicecology.wordpress.com/2018/01/17/a-crude-demographic-profile-of-our-commenters/. That seems to be true on all ecology blogs, including those like Small Pond Science and Scientist Sees Squirrel that write on different topic mixes than us and don’t use terms like “zombie ideas” and “statistical machismo”. The gender balance of our commenters matches the gender balance of our overall readership (which is a bit male-skewed), except that our posts on gender and equity issues draw a gender-balanced mix of commenters. And the seniority and gender balance of folks who comment on our posts on Twitter doesn’t seem to differ from the demography of folks who comment on our posts in our threads. So it’s not clear to me that we’d get more grad student commenters or commenters from historically underrepresented groups if we refrained from rhetoric like “zombie ideas”.
As with any blog, the vast majority of our readers never comment, or do so only very rarely. When asked to identify the most important reasons they don’t comment, only 5% of readers say it’s because they don’t want to get into an argument or worry about being criticized (https://dynamicecology.wordpress.com/2014/11/25/reader-survey-results-2/). Of course, we can’t survey the views of people who choose not to read us at all because of the rhetoric used in some of our posts. And of course, I would not suggest that just because only a minority of people refrain from commenting for reason X, that that reason is therefore unimportant or not worth addressing.
Only about 3% of our readers think that too many of our posts these days are just trolling, vs. 11% who think we’ve gotten too safe recently and used to be more critical and entertaining (https://dynamicecology.wordpress.com/2017/06/19/reader-survey-results-3/). Again, we of course can’t poll people who don’t read us or have stopped reading us because of the way some of our posts are written. But for me, that last result highlights the trade-offs here. Rhetoric like “zombie ideas” is not everyone’s cup of tea, especially not when coming from a senior prof like me (or Jeff or Angela), writing for a blog with a large readership. And both Brian and I have struggled with how terms like “zombie ideas” and “statistical machismo” read to others and get used (https://dynamicecology.wordpress.com/2015/04/09/peter-abrams-on-ratio-dependent-predation-as-a-zombie-idea/, https://dynamicecology.wordpress.com/2017/11/14/taking-statistical-machismo-back-out-of-twitter-bellicosity/). That’s why I’ve tried to tone down my own writing a bit over the last couple of years. My goal is to find a happy medium in terms of writing style, though there’s never going to be a way to please everyone. Of course, one could argue that people who feel, as you do, that rhetoric like “zombie ideas” has no place anywhere science is discussed, are the ones I (or any writer) *should* be aiming to please, and that people who enjoy more forceful rhetoric like “zombie ideas”, or at least don’t mind it, are wrong to enjoy it or not mind it.
If Angela, myself and others have adopted a combative or strident rhetorical tone in our writings on this topic (and not everyone would agree with your on that), but if we have, it’s because we care very deeply about this, because it matters. Not just matters in the sense of “let’s understand the fundamental ecological patterns and processes”, but matters in terms of real conservation. There is so much propaganda (and I use that word deliberately) being pushed out about how saving tropical ecosystems should be a priority because they are special, and contain specialized, irreplaceable interactions. Sure, they are, and they do. But so are other parts of the world, outside of the tropics, and especially outside of tropical rainforest. But they hardly get a mention and we are in danger of losing, for example, huge areas of grassland in South Africa that contain both astonishing plant diversity AND incredibly specialized sets of interactions between those plants and their pollinators.
Regarding your last comment, there isn’t a vast literature on this, and what there was I tried to summarise in the Oikos and Biotropica papers. But that’s the point: if there isn’t a huge literature pointing to greater specialization in the tropics, why would anyone select 5? Surely 3 is the logical choice?
Thanks Jeremy–this is really thoughtful! I’m glad you care about these issues.
Well, the disagreement makes me perk up and feel a bit of motivation to look into these issues a bit more. I’ve got too much to do already but I cannot help but think of how I might be able to get at some of these questions myself.
This is a really good post. Really meaty.
So interesting. You should do (or find a guest to do) something comparable with largely evolutionary questions. I’d do it myself if I knew what the right questions were. Hendry/Bolnick blog? Could even include a few of the same questions posed to a different if overlapping audience. The presence of both 1s and 5s for essentially all questions indicates not just a range of views, but polar opposite ones. My sense is that this is more common ecologists, but I’ve got no real evidence either way. Love to know the answer…
That’s a good idea, I’ll drop Dan and Andrew a line.
“If the available evidence and arguments are incomplete or point in different directions, shouldn’t everyone who’s aware of all the evidence and arguments have chosen “3”? Indeed, there’s an old joke that the answer to every question in ecology is “it depends”. I think the responses vary based on how one reads the assignment (and maybe I should go back and see if I didn’t follow directions).
For example, if one of the questions was:
Gap dynamics describes patterns of succession and regeneration in forests
I’d say the real answer is “it depends” but rather than answer 3, I’d answer 1, because I think gap dynamics is pretty good in some systems, but it’s useless in the Caribbean dry forests where I work. Since it is false there, I’d answer 1 or 2, because it’s not universally true and I’m not even certain that it’s mostly true, but I do think it’s basically true if light is your primary limiting resource. If people interpreted the assignment like I did, requiring a more absolute/universal criteria, then you’d get the “more likely to be false/less likely true.” When I thought there was no consensus, I put 3.
Covariates aside from tropical/temperate/polar (for the record I put 2 on strong species interaction closer to the equator–based on old lit I know from a past life in polar research).
Plant vs animal?
Primarily do Observational studies vs experimental vs model/theory
(or maybe these would explain more of the answer/no answer dynamics)
I LOVE this post. Of course, I am delighted to see the idea that “Species interactions typically are stronger and more specialized in the tropics” come up as highly controversial [there is now a slide about this survey in one of my third year lectures].
As Jeff Ollerton said, even questioning the idea that there is a general latitudinal gradient in the strength or specialization of species interactions is a relatively recent development. Changing people’s opinions on this much loved and widely accepted idea using evidence has been very interesting – it has felt a bit like trying to turn an ocean liner with a small oar.
An emeritus professor of conservation genetics [Richard Frankham] recently told me that most of his paradigm shifts occurred by him outliving the most vocal opposition. I am quite relieved to see that this doesn’t look like it will be necessary here!
Thanks Angela, I was hoping you’d comment. Interested to hear your sense that the controversy over a general latitudinal gradient in the strength/specialization of species interactions is of recent vintage. I think it would be interesting to repeat this poll in, say, 5 years and see if anything has changed.
Actually, if you could do it in 2005 instead, that would be awesome 🙂
Thanks for conducting this poll! I was especially interested to see the results for the dilution effect, and they don’t disappoint. The break down in terms of expert vs. novice opinion is quite telling. One aspect of this controversy that might not apply to many others is that we all WANT the dilution effect to be true. Most people (and ecologists in particular) would love to live in a world where we could protect our own health by protecting biodiversity. But in many cases, logic and data tell a different story.
We’re already planning to discuss the poll (in a general sense, not just the dilution effect) at an upcoming lab meeting!
“One aspect of this controversy that might not apply to many others is that we all WANT the dilution effect to be true. ”
Oh, I’m sure that applies to a bunch of the other ideas on the list! I’m sure lots of ecologists want it to be true that there’s a 6th mass extinction. Ok, not in the sense that they actually want lots of species to go extinct! But in the sense that they want everyone to believe that nature and humanity face an existential crisis. Same for the notion that local biodiversity is declining everywhere. Lots of ecologists want that to be true, because they think denying it amounts to downplaying an extinction crisis. I’m sure plenty of ecologists want it to be the case that biodiversity will buffer ecosystem function against climate change.
And controversy around several other ideas is due in large part to people wanting their pet ideas to be true. I’m sure Phil Grime and his academic descendants want species diversity to be a humped function of productivity. I’m sure Roger Arditi and Lev Ginzburg want ratio-dependent functional responses to be ubiquitous. Etc.
Glad you liked the post. Would be interested to hear any comments you care to share after that lab meeting.
I think there is a pretty simple explanation on why experts on average have a more critical opinion on the theories than non-experts. I would assume that even a lot of experts that disagree with some of the hypotheses might still think they are important enough to be taught in an introduction to ecology class. Neutral theory is probably the most clear example, it’s obviously false and no expert believes otherwise, yet its a beautiful theory which is nevertheless important, not least as a nullmodel. Therefore the neutral theory clearly should be taught to students. Now I believe students probably have a strong bias to believe things they get taught in class are true! So the question might be how do we need to teach theories which we believe to be false, yet still think students should know about and understand.
Hmm. Could be I guess. Though if so, that would mean the experts mostly misread or ignored the poll’s instructions.
I have various old posts on the strengths and weaknesses of “null” models. I’ll shamelessly take the opportunity to link back to a few of them. 🙂
I did not participate in the poll, so don’t know how that misreads the instructions. Could you explain?
The poll asked respondents whether the ideas were true or false. Not whether they should be taught in class because they’re elegant null models that sharpen our intuitions.
But that means the experts did not misread the instructions. My hypothesis: Experts correctly answered in your poll that a theory is false. Yet non-experts thought it was true, because they got taught the theory in class. That would explain why experts on average are more critical than non-experts.
“My hypothesis: Experts correctly answered in your poll that a theory is false. Yet non-experts thought it was true, because they got taught the theory in class. That would explain why experts on average are more critical than non-experts.”
Could be for many of these ideas. Not for all, I don’t think. For instance, I doubt ratio-dependent predation is taught in many undergraduate or graduate classes, and neutral theory probably isn’t taught to many undergrads. But you could well be on to something.
I know I’m very late to be jumping on this bandwagon, but I found my way here following the Evo-Bio poll.
I suggest a semantic explanation to the experts’ ‘skepticism’, other than on purely ecological grounds:
The questions/ideas in the poll are phrased as pretty short sentences, almost bullet-points. Ecological principals may be too complex to convey in a single sentence without if’s, but’s, exceptions or flexibility, even more so in the eyes of those who know a lot about the topic at hand. Constraining big ideas into a short sentence may omit parts that experts feel are indispensable (and are therefore less likely to agree with the sentence) whereas other respondents may overlook (what may seem to be minor) inaccuracies.
It’s obviously just an hypothesis and I do not refer to any particular question/s in the poll, nor did I test it in any way. It’s just a known source of bias in questionnaires (and the reason why some believe referendums, for example, are easily skewed and a problematic way of making ‘democratic’ decisions).
Could be. On the other hand, I deliberately stated most of these ideas as simple statements about what’s typically the case empirically. My hope was that that data-focused framing would limit the impact of the issue you identify.
I agree – the data-focused framing comes across clearly, and probably did limit any such biases. It was just my musing on something that really caught my eye.
This is a brilliant post, by the way, and a very interesting blog. Thank you!
You’re welcome, glad you liked it. This was the definitely the best post idea I’ve had in a while.
I will be defending my dissertation on the #1 most controversial topic in ecology in 5.5 weeks! Gulp. Here’s a preview https://onlinelibrary.wiley.com/doi/abs/10.1111/ele.12925
Interesting results! There is a fascinating body of literature on expert elicitation, but this result about experts being more bimodal was unexpected to me.
But not so sure how people interpreted the questions. For example, take the question “Species diversity typically peaks at intermediate productivity.” Say you think that in 70% of hypothetical situations you believe that diversity would peak at ID. I think you would answer a 5, based on my reading of the question, but not sure others would read it that way. I think the poll has a lot of linguistic ambiguity, but perhaps the sample size is large enough that these sorts of ambiguities wash away. However, based on the wording of the questions, I can see how the experts consistently averaging lower could be due to this ambiguity.
I would be fascinated to see the ‘Keynesian beauty contest’ version of this poll: How do ecologists think most other ecologists would respond to these question? I would be quite far off for some of these — for example, from inside the Minnesota bubble, it’s surprising that not everyone thinks that biodiversity is one of the most important determinants of ecosystem function.
I thought seriously about doing that, but it would’ve made the poll way too long.
I’m really curious about that too. Guess I could try polling people on which distributions of opinion surprised them most.
Also, I love that our commenters casually drop phrases like “Keynesian beauty contest”, confident that everyone reading will know what they’re talking about (or at least get the gist). Our commenters are the best. 🙂
This is why it’s generally considered a good idea not to do too many of one’s degrees or training stints at one institution. Conventional wisdom on a given topic, who are considered the “top” researchers in a given field, the “best” ways to answer certain questions – views on all of those things and others vary a lot from place to place. Of course they can vary from person to person in a given place, but local “cultures” do develop.
@ Mark Vellend:
Yup. Although I wonder if moving around helps as much as we might think, at least at certain career stages. For instance, if you do a PhD with prof X, you might well go on to go on to do a postdoc with someone prof X knows, who thinks about the world more or less as prof X does.
My modest hope for this blog is that it helps cut through insularity a little bit, particularly with unsuspecting grad students. If you wake up one morning to see that your favorite blog just did a post criticizing the idea that forms the basis of your thesis, or just did a poll in which 50% of respondents report disbelieving the premise of your thesis, well, that’s eye-opening! (Hopefully in a good way, rather than a scary way.)
The ability to communicate easily with people all over world certainly seems like it has countered any tendency towards insularity. Blogs are one part of that.
People often do go work with someone prof X knows, and it’s probably good for career advancement, but I’m not so sure it’s good for exposure to diverse perspectives, approaches, etc. Probably contributes to insularity (not within institutions, but within cliques, for lack of a better word).
As others have said – great survey! These results set me to thinking about what we know, what we teach, and the efficiency (or lack thereof) of empirical progress.
I’m a behavioral ecologist who dabbles widely. I also teach an undergrad ecology course (that contains no behavioral ecology). Thus I regularly invest time in trying to understand what’s known about topics outside of my area of expertise. Because I feel like I regularly encounter contradictory claims in the literature, your survey findings shouldn’t surprise me too much, but I actually found them really thought provoking.
I want to feel confident in my understanding of the world (especially when I’m teaching it!), and I’m frustrated when I feel like science isn’t working as well as it should. This happens a lot when I think about how bias can reduce reliability of results, and how an insufficient culture of replication means we often fail to rigorously assess both validity and generality of prior findings. Because I spend time thinking about those issues, its not surprising that I’m thinking that the (lack of) rigorous replication culture in ecology could be part of the explanation for the findings in your survey. Not only might some false findings persist, but a lack of consensus about how to test (replicate) certain ideas could lead to an accumulation of empirical data without a clear steps towards empirical resolution. I’m not saying I think this is THE explanation for your findings, but it seems like a plausible contributor. Of course ecology often deals with complex systems where causal paths may vary in importance (‘it depends’), so maybe empirical consensus in ecology is inevitably slow regardless of replication culture.
Shifting gears, I like the model of bringing together experts, especially those with strongly differing opinions, to build a consensus framework for evaluating a hypothesis. Agreeing how to interpret data before examining the data is a powerful tool. If no such agreement were reached, this could at least lead to a precise description of differences in assumptions, etc, which could then open those assumptions up for testing with a consensus framework. In theory, it should be possible for those in disagreement on an empirical matter to identify evidence that would resolve their disagreement. In some ways this seems like the sort of thing that happens in big physics. I don’t know how difficult this sort of effort is to implement, and what the track record of success is outside of physics. I wonder if anyone reading this has a sense of the extent to which NCEAS or other similar efforts have successfully resolved ecological debates?
“Thus I regularly invest time in trying to understand what’s known about topics outside of my area of expertise. Because I feel like I regularly encounter contradictory claims in the literature…”
Very interesting remark. I’d be very curious to hear from others on this. I have no relevant experience myself. My teaching has mostly been on topics on which I have pre-existing specialist expertise.
“insufficient culture of replication”
Do you think ecologists should start doing more high-powered, pre-registered replications, as psychologists seem to be doing? And what do you think we’d learn? Maybe we’d just (re)discover that the ecological world is pretty variable across space, time, and other dimensions?
“a lack of consensus about how to test (replicate) certain ideas ”
I have some old posts wishing for more “adversarial collaborations” in which people from opposing camps would agree a study that would settle their differences and then collaborate to conduct it. It’s been done a couple of times–often with a depressing amount of backsliding by whichever camp doesn’t like the results. But that’s ok, I think–the purpose of an adversarial collaboration isn’t to get the adversaries to agree, it’s to settle matters in the eyes of others. Which is probably one reason why it’s hard to get adversaries to agree to such collaborations–there’s too much risk you’ll “lose”. Some old posts on this:
“I like the model of bringing together experts, especially those with strongly differing opinions, to build a consensus framework for evaluating a hypothesis. ”
I agree 100% that that would be desirable in many cases. But history suggests it’s really hard. It was done for BEF in a “consensus” paper in Science around 2002 or so if memory serves. I think it was somewhat helpful, although at least one of the famous co-authors immediately started backsliding from it in subsequent talks. And Peter Abrams, Roger Arditi, and Lev Ginzburg did a joint “here’s where we agree, here’s where we agree to disagree, here are our agreed future research directions” paper on ratio-dependent functional responses in TREE in 2000. Peter Abrams is now accusing Arditi and Ginzburg of subsequently intentionally misrepresenting what that paper said (https://dynamicecology.wordpress.com/2015/04/09/peter-abrams-on-ratio-dependent-predation-as-a-zombie-idea/). I think there may be some moves afoot to develop some kind of consensus statement or agreed future research directions on local biodiversity declines. But I might be wrong about that; Brian would know.
Part of the problem with coming to an agreement on how to test some of these ideas is that they’re so vague and open to interpretation that even *with* good will on all sides everybody might not be able to come to agreement as to what the idea is and how to test it fruitfully. And I really don’t know how you go from a vague verbal idea to a concrete, usefully-testable one. Ecologists don’t have a great track record on that (https://dynamicecology.wordpress.com/2017/02/15/name-the-most-successful-and-unsuccessful-examples-of-operationalizing-vague-or-abstract-ecological-concepts/).
Hi Jeremy, Thanks for all these links! My take-home for the day is that you’ve written thoughtfully on a staggering range of topics. I wasn’t aware of any of these particular adversarial collaborations you mentioned. As and outsider, I find them convincing. Their failure to convince those most invested in the debates is disappointing – I like to think that scientists are more open to revising their opinions in light of evidence than are most other people – but I guess not really surprising.
In response to your question, I do see value in high-powered pre-registered replications. At least in behavioral ecology, much of what we think we know is often based on understanding from a relatively small set systems, and our understanding of those systems can turn out to be really poor under close examination (I showed this in systematic review of sexual selection of plumage color in a European songbird https://doi.org/10.1111/brv.12013). Thus assessing validity of prior results in those systems has major value (see these two recent papers that I’m not involved with: https://doi.org/10.1111/evo.13459 , https://doi.org/10.1101/283150 ). Of course the existence of variability (temporal, spatial, taxonomic, etc.) can reduce reproducibility, but a practice of deliberate and frequent replication can provide insights there. Too often I see divergent results explained away with references to environmental variability. It seems to me that environmental variability isn’t a robust explanation for failure to reproduce results until you can reliably identify the conditions under which the phenomenon holds or does not hold. This is rarely done in behavioral ecology. I think behavioral ecology (and maybe other disciplines of ecology and evolutionary biology) are much more difficult than many researchers realize.
Another possible way forward is formal Oxford-style debates, and papers that grow out of them:
The hope is that the formal debate format obliges those with opposing views to respond to each other point-by-point rather than talk past one another or just focus on their own favored “talking points”.
Oh, and also thought of another “adversarial collaboration paper” aiming to isolate areas of agreement and disagreement between opposing camps. This one’s from evolutionary biology, and concerns the concept of “niche construction”: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4261998/
Oh, and good question at the end of your comment re: has an NCEAS working group ever resolved a contentious ecological debate. I’m not sure. I think Shurin et al. meta-analysis of trophic cascade experiments in EcoLetts grew out of an NCEAS working group, and my vague sense is that that settled debate on *some* questions about trophic cascades. After that review, it was no longer tenable to argue that trophic cascades are mostly trivially weak, or that they are only strong in aquatic systems. But that’s just a vague sense on my part, perhaps others might have different impressions.
Starting a new subthread to continue the conversation about “adversarial collaboration” as a way to resolve scientific controversy…
In a pair of old posts I asked, what’s the biggest ecological idea you’ve ever changed your mind about?
Here I’ll pose a slightly different question, in the hopes of getting answers interesting enough to turn into a blog post. 🙂 What’s the biggest idea in ecology *any* ecologist has *ever* changed her mind about?
Like, imagine if AJ Nicholson had said “on reflection, Andrewartha and Birch were right, density-dependence is irrelevant for population dynamics.” Imagine Dave Tilman going “Actually, biodiversity is pretty unimportant for ecosystem function, my Cedar Creek results are a weird exception to that general rule.” Has any ecologist ever done anything like this? I can’t think of anyone.
The only example I can think of off the top of my head is from philosophy. Ludwig Wittgenstein famously developed two opposing philosophical systems, the second of which refutes the first.
p.s. Here’s my own answer to the question “What’s the biggest scientific claim you’ve changed your mind about?”: https://dynamicecology.wordpress.com/2013/12/26/from-the-archives-on-changing-your-mind/
This is not the biggest idea, I’m sure, but if I recall correctly, Dave Tilman now believes his speculations about invasibility of biogeographic realms from his MacArthur Award lecture (and the resulting paper) are incorrect. I believe it was reading Vermeij and other paleontologists that led him to change his mind.
Interesting, I didn’t know he’d changed his mind about that. But as you say, it was a speculative view so not as big a deal as changing your mind about something you previously were “all in” on.
Pingback: Why I voted as I did in our controversial ecological ideas poll | Dynamic Ecology
Pingback: How to assess “importance”? | Dynamic Ecology
Pingback: Bonus Friday link: controversial ideas in evolutionary biology | Dynamic Ecology
Pingback: Happy 6th birthday to us! | Dynamic Ecology
Pingback: Poll results: Which classic topics do ecologists care about? And which ones do they think ecologists collectively should care about? | Dynamic Ecology
Pingback: What’s the biggest change of mind any ecologist has ever had about any ecological idea? | Dynamic Ecology
Pingback: Fighting about frequency and randomly generating fitness landscapes | Theory, Evolution, and Games Group
Pingback: How do ecological controversies typically end? | Dynamic Ecology
Pingback: Poll: which ecological and evolutionary “laws” actually deserve that title? | Dynamic Ecology
Pingback: Poll results: which ecological and evolutionary “laws” actually deserve that title? | Dynamic Ecology
Pingback: Ask us anything: what theories should a beginning ecology student read up on, and how? | Dynamic Ecology
Pingback: On what important ecological research topic do non-experts have the most outdated view? | Dynamic Ecology
Pingback: Blast from the past: the most controversial ideas in ecology | Dynamic Ecology