The latest in our “ask us anything” series…
Is the ecological literature “idea free” (cf Sam Scheiner’s recent piece on the frequency of theory-driven ecology)? (Amy Parachnowitsch)
Brian: Focusing just for a minute on the phrase “idea free” I think ecology has the opposite problem – too many ideas, not enough decisive rejections of some ideas and embracing of others. We have lots of ideas, we just aren’t very good at weeding our ideas. Now I don’t think ecology will ever become Plattian (sensu Platt’s 1964 piece on Strong Inference in the journal Science). Things are just too complicated. There are always four or five important things going on so the notion we’ll have a decisive test that says its A, not B is not going to happen (usually its A and B and C and D). But I think we could still be a lot better at killing off ideas.
My favorite example is MacArthur’s broken stick. He put this idea forward in 1957 (PNAS), developed a theory of which it was one alternative in 1959 (AmNat), by the early 1960s the theory had been elaborated, well tested and mostly empirically rejected. In 1966 MacArthur (in response to Pielou) said essentially “it was a good idea, but it didn’t work out – can we please stop wasting time on it” – all in less than a decade. Three cheers for the efficiency of ecology. Except there are still papers being published in the 2010s testing the broken stick. Boo for ecology and the inability to let ideas go. That is why I think Jeremy’s zombie idea model is so important. Ecological ideas NEVER die.
Turning to Scheiner’s sense of “idea free” … I can be immensely critical of where theoretical ecology has gone and how large a fraction of it is irrelevant, but I can also be immensely critical of field ecologists who keep measuring the same things over and over again without decisively answering questions. I do think the failure to bind theory and empiricism together in a successful give and take dance is a real problem in ecology. So I largely agreed with Scheiner’s piece (but don’t think it said what a lot of people claim it said – namely that field ecologists are the only source or the problem).
But I would argue that the most underlying problematic attitude in ecology is not the merger of theory and empiricism. It is the lack of a problem solving mentality. Ecologists are very happy to gather information and generate ideas. I feel if we had more of a problem solving motivation we would advance better as a field. Problem solving because the metric for ideas – does it help solve problems? Then keep it. Is it pretty but useless? Then put it on a shelf and admire it only occasionally. If we were motivated by solving problems then we would necessarily bind theory and empiricism. But this binding is an outcome, not the main goal. This is closely tied to ideas of ecology needing to become more predictive (see my four posts, the first of which is here).
Jeremy: First, let me say that I love that Sam Scheiner went to the trouble to compile time series data on the frequency of theory-driven papers in ecology. As scientists, we’re supposed to be all about putting our ideas to the test with data. Except that, when it comes to our own behavior–what sort of papers do we write, how well does peer review work, how common is scientific misconduct, whatever–we all have really strong opinions based on little more than our own anecdotal experiences. Well, our own experiences plus whatever people on social media happen to be saying, which isn’t any better.
My main reaction to the results of the Scheiner piece is that it’s one of these glass half empty/glass half full things. His data show that theory-driven papers (as he defines them) are increasing in frequency–but he’s disappointed because he doesn’t think they’re increasing in frequency as fast as they ought to in some ideal world.
I think papers like the Scheiner piece play a useful role. It’s useful to have people who, when they are really bothered by something, call attention to it rather than just taking for granted that “that’s the way it is”. They wake everybody else up, keep everybody thinking and talking, maybe even shift the “Overton Window” and create the possibility of real change and improvement. Ecology may not change or progress as fast as the frustrated idealists would like–but thanks to the frustrated idealists maybe it changes and progresses a bit faster than it would without them. Just to be clear, I don’t personally share Sam’s disappointment at the frequency of theory-driven papers. But I have my own pet causes–things that really bug me, and that I think ought to bug others. Of course, if there is something that really bugs you, it’s up to you to make a convincing case that it ought to bug others as well, that it’s not just a personal hangup of yours. I think sometimes I’ve done a pretty good job of that, as with my zombie ideas posts that eventually turned into a paper I’m quite pleased with. But sometimes I haven’t done such a good job (although I hope I’ve never done as poor a job as, say, Lindenmayer & Likens).
Getting back to the question asked: more or less what Brian said. I agree that ecologists often are too slow to give up on ideas. In part because we’re too often satisfied with empirical work that’s at best suggestive–that only tests our ideas rather weakly or indirectly. I’ve talked a lot in the past about how I’d like to see ecologists focus more on model systems. I’d like to see ecologists rely less on putative “shortcuts” to insight. I’d like to see ecologists get better at combining different lines of evidence so as to thoroughly test both the assumptions and the predictions of their theories, and so as to distinguish between alternative theories. Finally, I’d like to see ecologists focus more on testing proper mathematical theories rather than “verbal models” or “conceptual models” that aren’t sufficiently well-defined to even be testable (which is why I mildly disagree with Brian that a laser-like focus on making and testing predictions will help us all that much. I see lots of emphasis on making and testing predictions in the literature–but making and testing predictions isn’t very helpful if those predictions aren’t derived from a well-defined model).
Lest I sound too negative about the state of ecology, let me hasten to add that I’m not! I actually think we’re better at ecology than we ever have been, and that we’ve improved a lot over the last few decades. So when I talk about the ways in which I’d like to see ecology get better, what I mean is that I’d like to see it keep getting better.
Hi Brian and Jeremy,
I like your thoughts and agree with much of what you said! As a young ecologist I am struggling with the state of ecology. Specifically, I am concerned (as Brian pointed out) that too many of us “are very happy to gather information and generate ideas”, without a strong focus on a well articulated question. In my opinion this problem is being perpetuated in graduate school by a lack of guidance in the scientific method and too little time spent developing a testable question (including the appropriate tests). Some of this problem may also be explained by the peer-review system putting too much emphasis on novelty.
My question, however, is about Brian’s statement that “things [in ecology] are just too complicated” to become Plattian. This same excuse is used to defend the notion that ecology can’t develop unified theories. I don’t know much about fields that do a good job developing unified theories (e.g., physics), but I doubt these fields are simple (or whatever the opposite of complicated is). I’m guessing for someone with little knowledge in ecology, ecology may also look simple. The devil is in the details. So, I’m questioning whether you really think this is true, or is “things are jut too complicated” a way to defend the fact that we haven’t yet been able to develop a widely recognized set of principles. For some, I feel like this is an excuse to continue to chase poorly articulated questions (Note: I DO NOT think Jeremy or Brian are these people, in fact quite the contrary, I’m just bringing it up here because of Brian’s statement).
If you do really think “things are just too complicated”, then what should ecology be striving for (OK, maybe this last bit is too much to ask)?
Thanks for all that you do to keep this blog interesting,
Personally, I think that whether ecology looks too complicated is up to us to decide. If ecology looks complicated, that may just mean we’re asking the wrong questions. I have an old post on this (one of my best efforts, I think):
On the other hand, there are a few phenomena that may just be “inherently” complicated:
Hadn’t seen the “synthesizing ecology” post yet. Very interesting. It has got me thinking about how people perceive the complexity of different scientific disciplines.
As you note, any discipline can always be made to more complicated by trying to answer more detailed questions. This is true, but probably only from the perspective of basic science. In applied science the questions of interest are more or less fixed (how much energy will it take to move A from B, how will species X respond to Y), so the disparity between the types of questions we can answer in a discipline, and the ones we want to answer probably determines how “hard/impossible/complex” a discipline is. Because scientific disciplines are mostly judged by their applications (even most defenses of basic science boil down to some version of “you never know what applications this could eventually lead to”) our perception of complexity is determined by the ability to address applied questions.
So maybe ecology seems like a much more complex science because we aren’t able to answer useful questions nearly as well/easily as we can in physics.
Good point, I hadn’t thought of it that way.
Just to be clear I don’t think ecology will ever be Plattian. But Platt was basically talking about biochemistry and he was looking for decisive tests like “is DNA or RNA or Protein the genetic code” (which was “decisively” answered as DNA – of course we’re finding its more complex than Watson and Crick ever imagined including roles of RNA, Proteins binding DNA, methylation of DNA and etc so maybe cell biology isn’t Plattian either). But I don’t think we’ll ever have a decisive test “which is more important competition or predation”
But I definitely do NOT think that is an excuse to give up on ecology. It just means we need different approaches. I’ve got my name on a couple of papers with the word “unified theory” in the title, so I’m not going to disagree with you on the need to go there! On the whole I think ecology needs to be more context dependent – but that is really a whole new post.
But bottom line – there is a lot of space for good science between Platt’s very simple (and maybe not even applicable to cell biology) view and giving up.
As I’ve mentioned here before, I think its worth thinking about
1. complexity of the field of ecology v. physics
2. theory v. experiment v. observation
3. prediction v. explanatory models
with “On the tendencies of motion” (google it for the link) in mind, since these are the issues that were being addressed. Is ecology really like things in motion though? Does ecology only seem complicated because we haven’t discovered general laws? And how close does prediction get us to discovering these general laws?
Pingback: Does theory in neuroscience have any empirical content? | neuroecology
Hi Brian and Jeremy,
Thanks for answering! You both provide some really insightful responses. A few of us followed up Scheiner’s paper and also classified the papers in the July 2012 issue of Ecology. It was a useful exercise, mostly because through it we all realised how difficult it can be to determine whether papers are ‘theory motivated’ or not. Everyone got different answers than Scheiner and they spanned the whole range (some fell out with fewer theory motivated papers, some more). So despite the small sample size, we all likely have different ideas about what theory really is. However, there were a number of papers that clearly fell into one camp or another. Although I started this exercise thinking maybe Scheiner was a bit hard on ecology, I came away with a different perspective. I wouldn’t say we should completely give up on observation based studies but being more explicit about the theory tied to our work can only strengthen the field. Well, as long as it doesn’t mean resurrecting zombies in the process!
What a great experiment to conduct (analyzing papers in a journal into the categories). I’ll have to try that myself. I know this paper is stimulating a lot of discussion. It came up again in my grad seminar yesterday. In the end if we’re having the discussion that is a good thing.
Interesting Amy. In any exercise like Sam’s there will always be a grey area. But I wouldn’t have expected that it would be that challenging to classify so many of the papers.
“Everyone got different answers than Scheiner and they spanned the whole range (some fell out with fewer theory motivated papers, some more). So despite the small sample size, we all likely have different ideas about what theory really is.”
Exactly. You run immediately into problems of definition and subjectivity, which was also my point to Jeremy a couple days ago regarding trying to make clear distinctions between what is, and is not, a philosophical problem. Unless you clearly define things and calibrate that across people, It’s largely in the mind of the beholder.
The reluctance of many ecologists to embrace hypothesis testing is fascinating. I have two related observations that I think are germane.
First of all, I am 52 years old. I remember in the mid 80’s at U Arizona, a colleague of mine and I polled grad students from my EEB department and from Cell/Molec across the hall with a simple question: “Why are you in graduate school?”. All invoked childhood predispositions and memories. The Cell/Molec grad students (ca. n=7) answered with variations of “I always like to take things apart/solve puzzles/figure out how things work”. The EEB students (ca. n=10) all answered with some memory of an early encounter with an organism or an ecosystem. I left that afternoon convinced that a lot of us do what we do not because we are focused on predicting, or solving problems, or generalizing, but because we really like our critters/ecosystems and this job is a great way to hang out with them and find out more about them. At least back then, testing hypotheses to contribute to a more generalizable framework wasn’t a priority among many, likely a majority, of ecologists.
There is a related constraint to using hypotheses, particularly strong inference. Tom Schoener once told me, tongue in cheek, that “The perfect paper in ecology is 10 pages long and has one good idea.” Papers are getting shorter and pithier for all the reasons we need not belabor. But one result is that a Platt strong inference approach, in which a data set is used to evaluate multiple hypotheses (and I’m now including complementary hypotheses), is just harder to get through the review process. Multiple hypotheses lengthen the introduction and force the reviewer to work harder. After one particularly dispiriting bout of reviews, I became convinced that the probability of getting a paper through review was proportional to a/n, where “a” is a random number between 0 and 1.0, and “n” is the number of hypotheses tested.
Pingback: Stats vs. scouts, polls vs. pundits, and ecology vs. natural history | Dynamic Ecology
Pingback: On progress in ecology | Dynamic Ecology
Pingback: Why AIC appeals to ecologist’s lowest instincts | Dynamic Ecology