Wanted to revisit a perennial topic of conversation around here: success and failure in ecology. And the surprisingly difficult task of distinguishing one from the other.
One reason scientific success or progress is sometimes difficult to identify is that scientific ideas have various desirable features that don’t always go hand-in-hand. So an idea can be successful in some ways but unsuccessful in others. Here are some ways in which a scientific idea might be considered successful:
- Make correct predictions.
- Make testable predictions, whether or not they’re correct. We learn something from testing predictions that don’t pan out as well as from ones that do.
- Identify or discover a new possibility or phenomenon. For instance, Edward Lorenz’s discovery of chaos.
- Explain a pattern or phenomenon.
- Provide understanding or insight.
- Unify seemingly-unrelated phenomena or special cases.
- Ask new questions.
- Ask better questions. For instance, taking an existing vague question and making it precise.
- Focus research effort. Insofar as you think scientific progress requires lots of people working on the same problem (whether collaboratively or not), you’re going to want to see scientists focusing their efforts. Arguably, at least some of the credit for focusing research effort should go to the idea on which the effort is focused.
- Be influential. Scientists tend to have a lot of respect for influential ideas (ideas that prompt a lot of work), even if those ideas turn out to be totally wrong and are eventually abandoned. Personally, I don’t share that respect, because I think that if influential-but-totally-wrong idea X hadn’t been proposed, the scientists who worked on it would’ve just worked on something else instead. And some of that work might’ve turned out to be based on correct ideas. So I don’t think being influential, independent of correctness or other desiderata, is a mark of a successful scientific idea. But I recognize that I’m probably in a minority on this.
- Other possibilities I haven’t thought of.
(Aside: all of the above require elaboration. For instance, there’s such a thing as too much unification. Sometimes, “focus of research effort” is just a phrase for “bandwagon“. Etc. Those sorts of caveats are another reason why “success” isn’t always easy to identify in science. But I wanted to keep the post short so didn’t elaborate much.)
I’m interested in how different ecologists define “success” in ecology. So as a conversation starter, below is a little poll. For each of a number of different big ideas in ecology, you have to say if it was successful, unsuccessful, a mixed bag, or if it’s too soon to tell (there’s also an option for don’t know/not sure/no opinion). I also ask you to provide your career stage, since I’m curious whether junior and senior people differ in their evaluations.
Don’t read anything into my choice of ideas. I just picked some big ideas that I have opinions about, and on which I’m curious about others’ opinions. I tried to include a range of different sorts of ideas–verbal and mathematical ideas, older and newer ideas, etc.
For purposes of the poll, define “success” however you want. I’m betting we’ll get a pretty wide range of views on most of these ideas, in part because different people define “success” differently. Even though all of the ideas on my list are famous ideas, it’s not obvious that they’re all successful. For instance, you know what I think of the IDH. The hump-backed model of diversity-productivity relationships has been debated for forty years, which arguably isn’t a sign of success. A bunch of prominent ecologists think R* theory is unsuccessful while optimal foraging theory and metabolic theory are successful–but that’s a very debatable view. There are ecologists wondering if neutral theory has just been a distraction. The ideas of r/K selection and limiting similarity have come in for a lot of criticism over the years. Etc. So hopefully this poll will be a good conversation starter. In the comments, I encourage you to share why you voted as you did.
p.s. Note that calling an idea “unsuccessful” doesn’t imply anything negative about those who proposed or worked on the idea. Great scientists can have unsuccessful ideas.
Dynamic Ecology 
Very interesting topic! — in somewhat similar vein, I’ve been wondering lately out of the key questions posed in early 1970s (e.g. first editions of Krebs and Ricklefs textbooks), which of them have actually been answered, versus which are still in play versus which have been swept under the carpet.
Ooh, that’d be another good post topic!
It’s interesting, and difficult, to think about how much credit for the success or failure of an idea should be given to the idea “itself” (whatever that means!). For instance, in an old post I argued that it’s to R* theory’s credit that the vast majority of attempts to test it have been very well-designed experiments. In contrast to, say, limiting similarity or neutral theory, for which many empirical tests have attempted to infer process from pattern and so been quite poor (i.e. the “test” couldn’t possibly tell you anything one way or the other about the theory, no matter what the result of the test). But on the other hand, there certainly are effective ways to test, say, neutral theory. So maybe you could argue that the theory itself was successful, but was “let down” by ecologists? Kind of like how a great baseball player on a bad baseball team is still a great player? Hmm…maybe I need to do a post on “Ernie Banks ideas”–great ideas that were let down because their “teammates” (the scientists who tried to test or further develop them).
Its interesting. I think neutral theory, R* and MaxEnt are all equally strong theories, making many precise quantitative predictions. So the degree to which they’ve been strongly tested or not definitely depends on “the teammates” (other scientists), not just on the theory themselves. Although I think you could argue that R* and MaxEnt have received a higher proportion of strong tests because the original authors in their initial presentations presented strong tests where as Hubbell definitley just presented a series of isolated pattern recreations which many (but not all) of the ensuing tests followed
“Although I think you could argue that R* and MaxEnt have received a higher proportion of strong tests because the original authors in their initial presentations presented strong tests where as Hubbell definitley just presented a series of isolated pattern recreations which many (but not all) of the ensuing tests followed”
*Very* interesting suggestion that the initial tests of a theory “set the tone” and determine how others subsequently test it.
The hump-backed model might be another (unfortunate) example here. Initially tested by eyeballing a few small observational datasets. Fast forward several decades, and what are the people who work on this arguing about? Whether there’s “really” a hump in this or that observational dataset.
What about island biogeography? Initial test (or one of them): Simberloff’s famous mangrove island size manipulation. Great experiment–but did it really “set the tone” for subsequent tests of island biogeography? It’s not as if lots of other people went out and did their own island size manipulations. So if island biogeography theory was successful (and so far, most voters say it was), it seems like it was successful for different reasons, or in different ways, than (say) R* theory?
Looking over my answers some (not too surprising) general trends emerge.
Patterns that I thought were successful:
a) are highly predictive
b) usually have specific processes in mind
c) are quite often out of evolutionary ecology sensu latu (optimal foraging, r/K, IFD)
Patterns that I thought were unsuccessful:
a) often started as a prediction of a correlation (BEF, hump shaped productivity-diversity, limiting similarity), diversity-stability
b) conversely rarely had a specific process in mind
Although there are exceptions. Neutral theory is highly predictive and has a specific process in mind but I voted only a mixed bag which I think is ultimately because its process was so obviously wrong. MaxEnt also has no process, but like the optimality program, invokes an extremely general overarching principle (so I voted successful).
For the record, my own votes, with some comments:
-IDH: mixed. You could probably talk me into “unsuccessful”. I do deplore the ongoing influence of zombie ideas about how disturbance affects coexistence and diversity, and I remain puzzled that the IDH could hang on as it has despite a lousy empirical track record. But the idea did give rise to some good modeling work (competition-colonization models; the recent work of Miller, Roxburgh, and colleagues). And also a number of very good observational and experimental studies that collectively have given us a pretty good answer to the basic descriptive question “How are diversity vary as a function of disturbance?” (Not that the answer–“in any ol’ way”–is a very satisfying one, but it *is* an answer). I think it’s valuable to have a sufficiently large body of studies, all addressing the same simple question using sufficiently-similar methods, that in the end you’ve answered the question and the answer isn’t likely to change in future.
-hump-backed model: mixed. It was a coin flip with “unsuccessful”. I went with mixed for two reasons. First, trying to bend over backwards to be generous towards ideas of a type that I just dislike on a gut level. Second, because I think of the large body of experiments examining effects of nutrient enrichment on plant diversity as having been motivated at least in part by the hump-backed model. I like that body of experimental work, I think it’s another example of a critical mass of experiments that gives us as definitive an answer as we’re ever going to get to a simple, straightforward question. But on the other hand, I have little time for the hump-backed “model” itself (trying to “test” verbal “models” is something ecologists should’ve quit doing decades ago), and little time for ongoing arguments about whether there’s “really” a hump in global survey data of grassland plant diversity if only we measure productivity in just the right way, and include enough massively-anthropogenically-enriched sites, and treat the fact that enrichment explains something like 1% of the variation in diversity as a minor caveat.
-R* theory: success. Read Jim Grover’s book on R* theory–basically *every* direct test of the theory has come out in the theory’s favor! And many of those tests are quite severe sensu Deborah Mayo–they’re not the sort of tests a theory could pass just be getting lucky. Yes, I know those tests have mostly come from algal and microbial microcosms, and grassland plants living in very nutrient-poor soil. But as I said in a comment above, to my mind that’s not a failing of R* theory, it’s just a reflection of the fact that people have mostly tested it in those model systems where direct tests are tractable. If anything, I think it’s to R* theory’s credit that hardly anybody’s ever tried to test it using highly-indirect or merely-suggestive line of evidence.
-island biogeography: success. Though now that I think of it, I’m not actually all *that* familiar with the history of research on island biogeography. So I can’t actually articulate why I think of it as successful in anything like the same amount of detail as I could with, say, R* theory.
-neutral theory: unsuccessful. You might be able to talk me into “mixed”. But I think ecologists were far too easily swayed into taking neutral theory far too seriously and literally as a “null model” that needs rejecting. And I think the insights that have come out of neutral theory (mainly, that certain famous patterns in ecology are consistent with either neutrality or the lack thereof) aren’t *that* big a deal. But I might change my mind down the road if I see signs that neutral theory has helped increase ecologists’ collective preference for process-based models over other sorts (MaxEnt being a special case; see below). Or that it eventually spurs community ecologists to develop a pop gen-style theoretical framework for the entire field. And you might be able to convince me that I’m blaming neutral theory for the mistakes of the ecologists who set out to test it.
-limiting similarity: unsuccessful. The most unsuccessful idea on the list, in my view, and it’s not a particularly close contest. Now a zombie idea.
-metabolic theory: mixed. You could talk me into “too soon to tell”.
-optimal foraging theory: success. One of the best examples in ecology of sustained, tight linkage and feedback between theory and experiment. Which perhaps just goes to show that you can’t entirely separate the “idea itself” from how people have tried to test it or otherwise make use of it.
-ideal free distribution: I said not sure, just because I don’t really know the IFD literature that well. But in light of the IFD’s close connections to optimal foraging theory, I wouldn’t be surprised to learn that it’s considered successful in much the same way optimal foraging theory has been. Even though I doubt that there are all that many systems in which organisms exhibit an IFD.
-r/K selection: unsuccessful. If you wanted to argue otherwise, I think you’d have to give r/K selection credit for successes of other, better ideas in life history theory that sort of resemble r/K selection if you squint at them.
BDEF: mixed. You could talk me into success, because the most influential strand of BDEF research (“random draws” experiments with grassland plants) is another good example of a critical mass of experimental work giving us a pretty definitive answer to a simple, straightforward question.
-diversity-stability: mixed, because (infamously) this isn’t really one hypothesis or idea so much as a bunch of semi-related or even unrelated ideas that happen to use some of the same key terms in different ways.
-MaxEnt: too soon to tell. I like the general idea of specifying the range of possible forms one’s data could possibly take on. It’s important to recognize that the answer to the question “why do my data look the way they do?” might be “because it would take rather improbable or special circumstances for them to look any other way.” I admire Steven Frank’s very deep, almost philosophical work on where “patterns” in data come from, and that work has strong links to MaxEnt. I really like the MaxEnt stuff that’s been coming out of Ethan White’s group recently. And it seems like people are making progress connecting MaxEnt to better-understood statistical approaches. But I also worry that MaxEnt as an approach is open to abuse because you’re free to treat anything as a “constraint”. We’ve already seen one dead end (?) from a creative (probably too creative) attempt to use MaxEnt with unconventional “constraints” (Bill Shipley’s trait-based MaxEnt work). I think Steven Frank’s on the right track in his focus on fundamental mathematical constraints. I’m suspicious of the idea that we can just treat highly-empirical things like average body size or average metabolic rate or etc. as “constraints” in MaxEnt and thereby discover something. But it’s still early days, and it’s not really my field anyway, so maybe I’m off base.
I gave limiting-similarity ‘success’ (if I remember correctly) on the grounds that it stimulated huge amounts of work through which we learned a lot – even if one of the things we learned is that there generally isn’t limiting similarity. That is, a theory can be successful without being true. I think you anticipate this in your definitions of “success” at the start of the post, but maybe didn’t give it much weight in your answer? I might give r/K ‘mixed’ for the same reason, although I penalize r/K heavily for not being internally consistent. Limiting similarity is internally consistent; it just isn’t complete.
“stimulated huge amounts of work through which we learned a lot – ”
Hmm…except that a lot of the work that the notion of limiting similarity stimulated was based on flawed premises. So whatever we learned from it (and I’m not sure we learned a lot–what do you think we learned?) was kind of by accident. The IDH mostly isn’t true either–but at least the body of observational and experimental work testing the IDH makes sense. I think that’s in contrast to, say, work measuring Hutchinsonian body size ratios to see if they’re >1.3.
BTW, at the risk of giving too much of the polling results away, I can tell you that your view on the success of limiting similarity puts you in a small minority among faculty. Which isn’t to say you’re wrong, of course! But you’re clearly an independent thinker. 🙂
Jeremy – perhaps I have to think “creatively” like a pretzel to count limiting-similarity a success. I do so because of its failure, not despite it… Limiting similarity was such an obvious idea for explaining diversity in the face of competition that its failure forced us into developing other ideas to explain coexistence – things like aggregation, variance/mean partitioning, etc etc. So maybe this just isn’t what any reasonable person would count as “success” – maybe this is like calling Ptolemaic astronomy a success because its failure forced the Copernican system! I may have to rethink my vote… 🙂
That’s like the lemming suicide theory of idea success. We need an unsuccessful idea to fail so that a better idea can succeed. 🙂
I’ve stumbled across your blog entries recently and I’m thrilled to see others in ecology have some of the same doubts as I do. I’ve working toward finishing a dissertation in historical ecology, largely inspired by my reading of McIntosh’s 1985 book and my experiences working with some of the raw material of ecology (university scientific collections).
I’m tending to agree with one part of Sheiner’s 2013 essay that there is a disconnect between the actual data of ecological studies and the theories. The data itself is highly heterogeneous and embodies important conceptual problems. For example, the species concept. If understanding biodiversity depends on identification of species, there is already a huge challenge to understand what species exist on a plot of land, or a meter cubed of seawater, and that is not even trying to account for any dynamic processes involved!
Ecology and biology both have important historical baggage associated with terminology (“invasive species” comes to mind). Terminology is crucial to understanding what should be or could be measured in an experiment or extracted from meta-data analyses.
I feel strongly that change will only come if the scientific practice of ecology and biology are revised to include fundamental concepts from chemistry and physics about repeatability, quality control and reference systems, and standardization of definitions. It’s probably not yet another statistical test that will solve the problem – ecology and biology need to establish a solid basis for comparisons instead of hiding behind the every experiment (or every field site) is unique syndrome which is prevalent in the literature.
Thanks for this, interesting to have comments from a historian. I’m kind of embarrassingly ignorant of the history of the field myself; reading Mackintosh and Kingsland is on my list for this year.
I think the issues you raise of standardization of definitions and repeatability are very important. You’re absolutely right that ecologists mostly haven’t discussed them, preferring to use statistics to quantify variation rather than standardization and other approaches to reduce variation.
This is sometimes referred to (or at least, it’s related to) the difference between Galileo and R. A. Fisher. In a Galilean experiment, you make everything identical across your experimental units, except whatever factor that you want to vary. In a Fisherian experiment, you randomize everything across your experimental units, except whatever factor that you want to vary. Obviously, there are lots of situations in which Galilean experiments really are impossible, and so you have to adopt a Fisherian approach. But the Fisherian approach often is a lot more powerful if you make your experiments as Galilean as you can.
I do ecology in laboratory microcosms in part so that I can have the control I need to study the phenomena I want to study. That makes me pretty unusual among ecologists. Gause pioneered this approach in the 1930s. There’s a great line in his book The Struggle for Existence (https://dynamicecology.wordpress.com/2011/07/09/non-zombie-ideas-in-ecology-gauses-the-struggle-for-existence/) about how ecology can be like physics if only ecologists would adopt physicists’ methods and control variation.
Actually I know Gause very well. His work is a central part of my own research which re-examines the theoretical underpinnings of accepted ideas in ecology and the concrete implications of applying untested ecological principles to marine conservation problems. The debate you’ve launched here (and in the follow-up) with an apparently simple question has produced some really good examples about how theories are transformed and transmitted through scientific practice. Like when the names of the theories vary for different reasons – see the appendix from Michael Palmer in 1994 where he lists 120 co-existence hypotheses (Folia Geobot Phytotax 29: 511) – it is difficult to know when we are all on the same page!
Pingback: Poll results: which big ideas in ecology were successful, and which unsuccessful? | Dynamic Ecology
Pingback: Advice for undergraduates on contacting supervisors in ecology: Aim high and understand that it will be an anxious experience | Weltbummler Ecology