(Note: I wrote most of this post before seeing Brian’s recent post on how there is no one true way to do science. There is definitely overlap between the posts!)
Recently, I attended a symposium at the Kellogg Biological Station (KBS) that celebrated 50 years of research in aquatic ecology based at KBS. The event featured talks by former faculty and students, spanning the entire timespan of research at KBS. While everyone did research (at least at one point) in aquatic ecology at KBS, otherwise it was a fairly mixed group: evolutionary ecologists through ecosystem ecologists; research on small local streams vs. on Lake Baikal; junior folks vs. retired folks. But there were a few themes that kept emerging throughout the talks, one of which I want to focus on in this post. That theme? Tackling complex ecological questions requires a diversity of approaches. This theme came up over and over again.
For my own research program, I feel that the questions I’m interested in require a combination of observational studies on natural populations, experiments (in the field and lab), and mathematical modeling to tackle. First, I’ll go through why I use these approaches in my personal research. Then, I’ll come back to the general topic of the importance of diverse approaches.
In my research, I use observational studies to ground my research in natural systems. My goal with these studies is to identify phenomena in natural populations that are interesting and seem to warrant further study. For example: why do we see positive associations between some predators and levels of parasitism? Or, this parasite has shifted hosts and now infects an invasive species. What evolutionary changes allowed that? Sometimes these patterns are ones we find when we set out to look at that particular question. But, as I’ve written about in the past, sometimes these patterns emerge when we are studying something else.
Next, we often carry out experiments aimed at testing a hypothesized mechanism underlying a particular pattern. Of course, it isn’t always possible to manipulate the hypothesized driving factor, but, when it is, this allows one to directly test causality. In my case, these experiments are often done in the lab, at scales ranging from beakers to buckets. Other times, the experiments are done out in the field, using whole-water column enclosures. In all cases, the goal of these experiments is to test the effects of particular factors.
Finally, my research often uses mathematical models that help me link results from different scales, generalize results, and/or try to understand how a factor that can’t be experimentally manipulated might be influencing results. Can the shift in body size and infection risk that we observe in the lab when we manipulate a predator explain the observed links between that predator and epidemics in the field? If the parasite didn’t castrate its host, would we still expect the same effects on population dynamics? Those sorts of questions require mathematical modeling to tackle.
So, in the end, many of my papers end up having a combination of field patterns, experimental tests, and mathematical results. I feel like I can’t tackle the questions I’m interested in without using these different approaches. And, based on the symposium at KBS, it was clear that many other people also feel that they need to use a variety of approaches to study ecological questions. The particular approaches that would be used will differ between people, sub-disciplines, etc., but the need for diverse approaches seemed very consistent. In my opinion, if studies employing a variety of different approaches all point to the same answer, that gives me much more confidence in that answer. And, if there are discrepancies between the results found when using different approaches, uncovering the sources of those differences can be very important.
Now, back to the general topic of the importance of diverse approaches. There is no way that one person can do everything. In my case, I am finding, more and more often, that the questions I’m interested in require genetic skills that I do not possess. And I am very, very good at getting myself in over my head with a planned theoretical analysis. This is part of why collaborations are so valuable. (Another advantage to collaborations, in my opinion, is the diversity of perspectives that result.)
So, my advice to grad students is to try to develop a variety of skills, techniques, and approaches with which you are comfortable. In addition, develop skills that will make you a good collaborator, because the days of a scientist doing innovative, exciting, important work on his/her own are long over. (Brian made this point, too, in his rather lengthy footnote.) The important questions in ecology will not be tackled with a single approach.
Dynamic Ecology 
Hi Meg, yours and Brian’s commentaries and Brian Maurer’s chapter in The Nature of Scientific Evidence (almost ten years old but I’m only at 2004 on my ‘catch up on the literature’ resolution) make a similar argument, that there are many ways to do good science. And while I agree with the sentiment I’m not certain what problem the argument is aimed at – articles like Liken’s and Lindenmayer’s? Personally, I love a good polemic…and I’m not sure it doesn’t serve a very valuable purpose even when it’s occasionally misguided or wrongheaded. It forces those on the other side to develop clear, cogent arguments in defense of their position and those clear statements of position may be promising avenues for progress. And are there really people out there who will argue that there isn’t a place for experiments, observational studies and theoretical, mathematical models? Maybe there are but it’s a pretty small group.
But what bothers me more is that this argument could be interpreted as saying that all ways of doing science are equally good. Science needs to be tolerant, pluralistic, and accommodating with respect to scientists but we need to be hard on ideas. Disciplines can be both too prescriptive and not prescriptive enough and, if anything, I think ecology should be spending more time telling people how to do science well rather than telling them that all ways are fine. These prescriptions are often going to be a little off the mark but they will force us to think more clearly about what we do (I do a lot of meta-analytic work but Brian’s commentary helped me with my response to people who are suspicious of meta-analyses). I also think they provide a way for disciplines to get back on track when they drift a little.
I think ecology would be better off with more people taking clear positions on what they think bad science is (or, at least, explicitly identifying the limitations of different methods) than suggesting we have to be more pluralistic. I don’t think our biggest problem is that good science isn’t allowed to find a forum, it’s that we aren’t rigorous enough about what qualifies as good science. And calling something bad science (even when it’s not) shouldn’t be thought of as arrogant or presumptuous or insulting – it should be a conversation starter. Straighten me out if I say your science is bad and it isn’t. Give me an opportunity to straighten you out if you say my science is bad and it isn’t. I think there is more opportunity for progress if we err on the side of challenging ideas than on the side of accommodating them. Best, Jeff Houlahan
Yes, we absolutely should be critical, but I think the point Brian and Meg are emphasizing is that the question of good vs bad science is totally independent of the approach. Like any other technology, observations, experiments, meta-analyses and theory can each be used in productive or unproductive ways. I guess you could argue that integrating a diversity of approaches can be unproductive, but I tend to agree with Meg that carefully (maybe that is the key word) integrating multiple approaches increases the chances of doing good science.
Hi Peter, I don’t quibble with Meg’s statement but, who does? It seems like a prescription in search of a malady. Even Likens and Lindenmayer weren’t suggesting that meta-analyses are wrong – they were just suggesting that the downsides weren’t being considered carefully enough and that an over emphasis on meta-analyses might have negative implications for ecology. Is there a clear thinking, practicing ecologist out there that doesn’t agree with the statement that “experimental, observational, meta-analytic and mathematical techniques are all useful.”? But, Peter, you state that ‘good vs bad science is totally independent of the approach’ – what’s the evidence for that? I’m happy to believe it, if it’s true. The two statements that are back-to-back here are not the same – ‘totally independent’ sets a different bar than ‘useful’. The question we might disagree on is the relative utility of different approaches in different contexts and, in my opinion, that needs more rigorous debate in ecology than we are currently having.
So, here I am arguing for prescription and then being critical of Meg’s prescription but this feels like the weakest kind of prescription – any and all approaches are good. And while that may be true in the strictest interpretation I doubt that they are all equally good. This kind of statement is analogous to the weakest kind of hypothesis test where almost all possibilities are still included after the test.
But, why care? Is it just spoiling for a fight? Well, it’s true I love a good fight, but, I think this actually gets at a ‘domestication’ of science that I’m not sure is good for scientific progress. I think we occasionally confuse tolerance, inclusiveness, civility, and plurality with respect to scientists with similar concerns about ideas – we need to be kind, open, supportive, and encouraging to scientists and prepared to kick their ideas around the room like jockeys in a barfight.
And it sounds to me like you think I may be spearing the straw man myself, Peter, and perhaps I am. But I have deep concerns about the progress we have made in ecology. My concerns are completely unoriginal, A Critique for Ecology written more than 20 years ago, expressed most of the concerns I still have today. What progress has ecology made in understanding how nature works? And I’m not looking for a list of all the things that ecologists claim we understand, I want the evidence that we actually understand something. (And that is a sincere challenge – evidence of increased understanding of the natural world that has come from ecology. I know there is some, I’m just interested in actually seeing how much there is). I wasn’t there, but as I understand it, Tony Ives can ask “Should ecology be about general laws’ at ESA and split the room. We can’t even decide if there are general laws let alone what they are. Do you think there is a Physics or Chemistry conference in the world where that question would split the room? And I’m not saying it isn’t a legitimate question but if ecology is that much different than other disciplines don’t you think we should know it by now? Or shouldn’t that be the fundamental question we are addressing? And I think some of the blame for this lack of progress falls on ecologists for not be demanding enough of themselves and their colleagues, not challenging established approaches more often and more rigorously, not doing exactly what L&L have done in demanding we defend our approach.
So, while I don’t disagree with Meg’s point, L&L’s rigorous challenge of meta-analyses seems like a more likely way forward for ecology than a call that ‘all approaches are fine’. Best, Jeff H.
I certainly didn’t mean to imply that all ways of doing science are equally good (and I think Brian made that point explicitly in his post). My post was motivated by the observation at the symposium I attended that everyone, regardless of the questions they were looking at, stressed the importance of using diverse approaches. It seemed clear that, if you’re a one trick pony, your ability to do rigorous ecology would be limited. Not all approaches are equally good for a given question, but the odds that an important question can be addressed by one approach is low.
How diverse are our approaches in ecology, from a philosophical as well as methodological standpoint? I recently put up a blog post suggesting a possible method for quantifying and describing how diverse the field actually is: http://tinyurl.com/qc5j6yv
It’s a bit coarse, naturally, so I would be happy to hear what alternatives people might have in mind that accomplish something similar.
As a current graduate student, I am finally starting to be able to tie these different approaches together in my head (and trying in publications). I wonder though if there should be more focus during the undergraduate phase on understanding why and how vital these approaches are. I know (shamingly so) that I was a student who sat in calculus wondering why the heck I was learning integrals when all I wanted to do was get a job playing with insects. Maybe undergraduate seminars that really focused on introducing, laying out, applying and understanding why their course plan is set up the way it is would help students formulate why a diverse background is essential to any field. This may also force universities to analyze if the course plan they have laid out is the most beneficial.
Yes! I totally agree. I also didn’t see why math skills would be useful in ecology until I got to grad school. I took Calc II to meet the requirements, and then didn’t even consider taking another math class (even though I had liked math and done well). I really wish I had gotten the message that math was important for ecology and evolution.
One reason a diversity of approaches is powerful is that it usually involves checking your ideas/hypotheses/models from all angles. Developing a mathematical model of your idea rather than just relying on intuition. Testing assumptions as well as predictions. Testing a range of different predictions, not just one. Etc. The most convincing stories in science often are those where all the i’s are dotted and t’s crossed, and where many different lines of evidence are mutually consistent and point towards the same conclusion. Relying on a single approach is a bad idea if it means giving short shrift to other relevant lines of evidence gleaned from other approaches.
I really like the point of this post and those that inspired it. It can’t be repeated enough in a variety of ways and venues. It stands in stark contrast to the MacArthur lecture given by David Tilman. It was a while ago, but the one thing that I recall now is that he told all of us that we all needed to do more research just like him, and that his approach was best.
Jeff, I can’t figure out how to get my reply to show up in the right place… You wrote “But, Peter, you state that ‘good vs bad science is totally independent of the approach’ – what’s the evidence for that? I’m happy to believe it, if it’s true.” I’m pretty confident that no matter what approach you look at, you’ll find a few examples of its application leading to real gains in understanding and/or predictive ability, and lots of examples of it leading to little progress. That’s why I say that asking “which approach is best?” is the wrong question. I wish that the adoption of a particular methodology would consistently lead to “better” ecology, but I’m afraid it isn’t that simple. It’s much easier to recognize good science in retrospect than to identify a formula that will replicate it in the future.
An old post that’s relevant here:
In thinking about the science I admire most, I often find it difficult to briefly summarize the “approach”. In part this may be because the best science combines different approaches. But in part I think it reflects what I wrote in that old post. The best stuff in many cases can’t easily be reduced to a simple summary or formula, and so isn’t easily emulated. The best people, doing the best work, are (i) quite creative, and (ii) quite good at customizing their approach depending on the nature of the problem, the system in which they’re trying to solve it, and other factors.
I certainly think we should be open to critiquing one another’s approaches, that’s just part of doing good science. But I also think it’s worth recognizing the limits of such critiques–they’re never going to produce a recipe that anyone can follow to become a Meg Duffy. Believe me, if such a recipe existed, I’d follow it in a heartbeat! 🙂
Pingback: Modelling cyclic populations: thoughts on the workshop | Dynamic Ecology
Pingback: Establishing New Field Sites | Dynamic Ecology
Pingback: Why ecologists might want to read more philosophy of science | Dynamic Ecology
Pingback: Stats vs. scouts, polls vs. pundits, and ecology vs. natural history | Dynamic Ecology
Pingback: Friday links: Bob Paine passes away, and more | Dynamic Ecology
Pingback: Poll results: what are the biggest problems with the conduct of ecological research? | Dynamic Ecology
Pingback: Please share your favorite examples of complementary studies or research approaches in ecology! | Dynamic Ecology