Objections to microcosms in ecology, and their answers

I work in protist microcosms. A typical experimental unit is a glass jar with 80 ml of nutrient medium, which I inoculate with bacteria, protists, and perhaps other microbes. Then I (actually my research assistants) follow the resulting population dynamics. Several features of this system combine to make it an excellent model system for asking fundamental questions about population and community dynamics. Protists have very short generation times (4-48 h), so you can collect hundreds of generations of data in a few months. Microcosms are easy and cheap to replicate highly, giving you a lot of statistical power. You can control and manipulate features of the system that are difficult or impossible to control and manipulate in most other systems. I tend to use the system to ask questions about the consequences of particular processes or combinations of processes that would be difficult or impossible to ask in other systems. For instance, how does productivity, in isolation from other factors, affect the occurrence of alternate stable states and assembly cycles (Fox 2008 Oikos)?

The advantages and drawbacks of protist microcosms, and of artificially-assembled communities more generally, have been debated in various places (see, e.g., papers in the April 1996 issue of Ecology and chapters in Bernardo and Resetarits (eds.) 1998). But over the years I’ve run into various objections that either haven’t been addressed publicly, or haven’t been addressed in quite the way I’d address them. Plus, although the debate over microcosms seems to have died down a while ago, who knows when it will flare up again, so I figured I might as well take a preemptive shot in the next Microcosm Wars.

So below are some objections to microcosms (in bold), and my own answers. I emphasize that I really have encountered all these objections (sometimes in unsigned reviews, so I can’t link to the sources); I’m not setting up straw men here. I also emphasize that, in answering these objections in the way I do, I’m speaking only for myself. Microcosm experiments are conducted for different reasons by different people, who would probably have different answers to these objections. Indeed, one overarching problem I have with these objections is that they’re blanket objections, and those who raise them often don’t seem to pay attention to the various quite distinct reasons why microcosm experiments are conducted.

I should also note that microcosms absolutely have their limitations. There are plenty of questions you just can’t ask using microcosms. For instance, protists lack stage-structure, so you can’t study stage-structured population dynamics with protists. Protists and other small organisms aren’t convenient for collecting individual-level data, and so if you want to scale up from individual-level data to population dynamics you need a different system (I understand that this is why my fellow Oikos editor Andre de Roos switched from working on zooplankton to working on fish). It’s hard to study the bacteria in a typical protist microcosm, which means I (like many ecologists) mostly treat the bacterial portion of the system as a ‘black box’ whose effects I hope I can summarize implicitly (e.g., by treating bacterivorous ciliates as growing logistically, rather than actually modeling their interactions with the bacteria). But those kinds of detailed considerations are of a very different character than the blanket objections to microcosms which I discuss below.

Microcosms just use organisms to solve equations. Yes, but so do non-microcosm systems. Anything can be described with equations, and so all organisms can be said to ‘solve equations’. The challenge in microcosms, just as in any other system, is to figure out what equations the organisms are solving, at least to a sufficiently good approximation to answer the scientific question of interest. (And if you insist that your system is so complex that it can’t be described with equations, how do you know humans can comprehend it?)

Microcosms are rigged; the results can’t help but confirm the theory. Boy, I wish that were true! Believe me, if I could rig my experiments so they always came out the way I wanted them too, I totally would (and my cv would look a lot more impressive!) I can actually see where this objection comes from, because there certainly are microcosm papers which report such clean results, conforming so beautifully to theoretical expectations, that it’s hard for someone who doesn’t work in the system to believe the experiment wasn’t somehow ‘rigged’. Gause’s (1934) competition experiments are famously well-described by the Lotka-Volterra competition model, and Kaunzinger and Morin’s (1998) test of food chain theory is a terrific recent example. But those kinds of theory-confirming papers not only aren’t rigged (more on that in a second), they’re far outnumbered by theory-disconfirming papers. You know how I just said that, in microcosms, the challenge is to figure out what equations the organisms are solving? Well, that’s a big challenge, because it’s often not the equations you thought they were solving. Take it from me, even though protists aren’t animals, they definitely obey the Harvard Law of Animal Behavior (‘under carefully controlled conditions the animal does what it damn well pleases’). For example, see Harrison 1995 and Fox 2007, to pick just two of many possible examples. Hard as it may be to believe for people who don’t work in microcosms, they do have the capacity to surprise. They can reveal new phenomena and unexpected behavior (e.g., effects of species loss on food web connectance in Fox and McGrady-Steed 2002; effects of dispersal on within-patch demography in Fox 2007). That capacity for surprise is what makes them useful. Microcosms are complex enough to surprise, but simple (and controllable, and replicable) enough that the source of the surprise hopefully can be identified. If your system is hugely complex and ‘noisy’ (which is often shorthand for ‘hugely complex’), it can be hard to tell if you’ve been surprised or not, because of the many factors that can obscure the signal you’re looking for. It can be even harder to figure out why you’ve been surprised. By throwing up ‘tractable surprises’, microcosms can suggest new ideas and teach us new things, not just serve a negative role as a first-cut test of theory. ‘If a theory doesn’t work in microcosms, it won’t work anywhere’ is a claim sometimes used to justify microcosm experiments, but it’s not a claim I personally buy (I used to, but not anymore).

And even when microcosms do conform to theoretical predictions, well, that’s generally a surprise too. Because microcosms never conform precisely to theoretical assumptions—that’s pretty much impossible. So even when the theoretical predictions are supported, the experiment demonstrates that those predictions are robust to the ways in which microcosms violate the relevant theoretical assumptions. For instance, Kaunzinger and Morin (1998) is a beautiful confirmation of Oksanen et al.’s (1981) predictions for how equilibrium biomasses on different trophic levels change along a productivity gradient—even though Kaunziner and Morin’s population dynamics were highly non-equilibrial (they averaged over a predator-prey cycle), even though their top predator seriously violated the assumption of a linear functional response, even though their basal trophic level in some treatments comprised a heterogeneous mixture of bacteria rather than the single species assumed by Oksanen et al., and even though their unshaken microcosms violated the assumption of a spatially-homogeneous system. Indeed, theoretical models that explicitly incorporate precisely these complications actually make different predictions than Oksanen et al. So a very interesting question is why the Oksanen et al. predictions are robust to these complications. One way to get at that question would be to manipulate features of the system like the number of species per trophic level and the spatial homogeneity, and indeed such manipulations have been conducted (e.g., Fox 2008 Oikos, Fox and Barreto 2006). I suspect this objection gets raised because my fellow microcosmologists and I often emphasize the ways in which our system conforms to theoretical assumptions while neglecting to emphasize the ways in which it doesn’t.

Microcosms aren’t ‘realistic’ or ‘natural’. I can totally see the point of this concern if (and only if!) the goal of a microcosm experiment is to mimic some specific natural system, or some specific feature of some specific natural system. For instance, if you put an individual predator and some prey in a small container in order to estimate the predator’s feeding rate in nature, you’d better be sure that the container doesn’t introduce any artifacts which bias your measurements (e.g., lacking prey refuges which exist in nature). This is the main reason why Carpenter (1996) objects to microcosms. But if that’s not the goal of the microcosm experiment, then this objection is simply irrelevant and frankly it frustrates and mystifies me when it’s raised. I wonder if those who raise this objection have really thought it through, and ask that they consider the following points:

a. In many respects, microcosms are realistic. For instance, all microcosm experiments use species found in nature, and most use species that coexist in nature. Nobody uses robots, or Jurassic Park-style lab-created species, or children pretending to be animals (but see Bell 2007). And the resulting dynamics are realistic in many respects too. For instance, the average strength of trophic cascades in protist microcosms is almost frighteningly similar to the average strength of trophic cascades in nature (Fox 2007 Oikos). Protist microcosm food webs exhibit pyramids of numbers and biomass, just like many natural food webs. Etc.

b. Sure, microcosms aren’t exactly like any specific natural system. But neither is any other natural system! As every ecologist surely knows, natural systems differ a lot from one another, and change over time. ‘Natural’ isn’t one state of affairs, it’s a massive range of states of affairs. So if you say microcosms aren’t natural, which natural system are you comparing them to? And don’t say ‘they’re unnatural compared to every natural system that’s ever existed or could exist’. No one knows what sort of natural systems could exist, and more importantly protist microcosms actually are reasonable analogues for small, naturally-occurring water bodies such as puddles and rockpools (McGrady-Steed and Morin 1996).

c. Many microcosms aren’t intended to be realistic or natural. Again, the point of my experiments is not to reproduce the behavior of any particular natural system, it’s to obtain information that couldn’t be obtained in any other way. Criticizing this kind of experiment as ‘unnatural’ is like criticizing a car because it can’t fly.

d. No experiment is ‘natural’ or ‘realistic’. That’s the whole point of experiments, including field experiments—to change nature, to create conditions that wouldn’t otherwise exist, i.e. unnatural conditions. We do that because a really good way to obtain information about how realistic systems work is to create unrealistic conditions.

e. You’re effectively insisting that we discard useful information which we could not obtain in any other way. Microcosms, by allowing you to create conditions which would be difficult or impossible to create in nature, allow you to expand your information base. For instance, it’s really useful to know that, under the conditions produced by Kaunzinger and Morin (1998), the Oksanen et al. model works. It mostly doesn’t work under other conditions, so comparing the systems where it does work to the systems where it doesn’t lets you develop testable hypotheses about the circumstances under which it works or doesn’t work. You wouldn’t be able to make such comparisons if all you had were systems where the model doesn’t work. So if you don’t think we should do microcosm experiments because they’re ‘unrealistic’, you’re effectively saying that you want less information, about a smaller range of systems and conditions. Please explain how that could possibly be a good thing! And don’t just explain it to me, explain it to Charles Darwin. Charles Darwin, the greatest naturalist in history, didn’t just rely on observations of unmanipulated nature for evidence for evolution by natural selection. One of his most important lines of evidence, the line of evidence with which he chose to begin the Origin, was the effects of artificial selection in domestic species. Artificial selection provided Darwin with evidence relevant to interpreting nature and testing his hypotheses, evidence that he could not have obtained any other way, even though artificial selection as practiced by breeders is highly unrealistic. For instance, selection coefficients imposed by breeders often are much stronger than those typically observed in nature, and breeders often select for traits that would never be favored in nature. My point here is not proof by authority; Darwin wasn’t infallible. My point is merely to ask those who have a blanket objection to all microcosm experiments as ‘unnatural’ or ‘unrealistic’ (e.g., Carpenter 1996) to carry their arguments to their logical conclusions.

f. If there’s some specific respect in which you think microcosms are unrealistic, it’s probably possible to manipulate that feature of the system and thereby turn your objection into a testable hypothesis. A constant, undisturbed environment is unrealistic? Ok, let’s manipulate environmental variation or disturbance (e.g., Warren 1996 Oikos, McGrady-Steed and Morin 1996 Oikos, Fukami 2001 Oikos). It’s unrealistic to mix species with no coevolutionary history? Ok, let’s manipulate coevolution (see, e.g., the work of Mike Brockhurst, among many others). It’s unrealistic for a community to be closed to immigration and emigration? Ok, let’s open it to dispersal (e.g., Holyoak and Lawler 1996, Warren 1996 Oikos). Etc.

Microcosms are too small-scale. Umm, you are aware that protists are really tiny, right? Population sizes in my microcosms are on the order of 10,000-1,000,000, which is as large or larger than the population sizes in the study areas most ecologists use to study  most macroscopic organisms (e.g., there are only 225,000 trees in the famous 50 hectare plot on BCI). So relative to the size of the organisms, microcosms are actually large-scale (and long-term, relative to the generation times of the organisms). And unshaken microcosms contain substantial spatial heterogeneity at spatial scales relevant to microbes (e.g., Meyer and Kassen 2007). Plus, spatial scale can be manipulated, for instance by manipulating vessel size, and by studying arrays of microcosms linked by dispersal (e.g., Holyoak and Lawler 1996).

Microcosms lack the full range of processes that occur in nature. The objection here, as best I can understand it, is that a system that includes only a subset of whatever goes on in any natural system cannot tell us anything useful about nature. Rather than extending our information base, microcosms just represent irrelevant ‘apples’ that can’t usefully be compared to natural ‘oranges’. Again, if the objection is only meant to apply to microcosms that try to mimic specific natural systems, then I can appreciate it. But if it’s meant to apply more broadly, then I don’t buy it at all, because it amounts to denying that ecologists can build up an understanding of how a complicated ecological whole works by studying its parts. Carpenter (1996) claims that studying parts of a system is informative in molecular biology (‘a molecular biologist who isolates ribosomes is working on ribosomes’), but not in ecology (‘an ecologist who isolates organisms in bottles may not be working on communities and ecosystems in any relevant sense’). Unfortunately, I confess I don’t fully understand his reasons for this claim. One of his reasons is that ‘there is general agreement about human health goals that rationalize most of [molecular biology’s] funding’. What this has to do with the scientific validity of ecological microcosms I have no idea. Another is that ‘relatively rapid replicated study is possible at several levels’ in molecular biology, which you would think would be an argument in favor of microcosms in ecology since rapid, replicated experiments are what microcosms are for. The claim that ecologists cannot build up an understanding of a complex ecological whole by studying its parts just strikes me as so obviously false that I don’t even know what to say in response.

Microcosms should not be studied for their own sake. The ultimate goal should be to understand nature. I don’t know anyone who studies microcosms for their own sake. I sure don’t. I mean, yes, I do play close attention to the results of previous experiments in microcosms, in order to help me design my own experiments. But that’s surely no different than what any field ecologist does—you build on what’s already known in order to learn something new. The mere fact that someone conducts a lot of microcosm experiments, and no field studies, is not evidence that he or she is just interested in microcosms for their own sake. I ride the bus a lot too, but I don’t ride the bus for its own sake. A frequently-employed means to an end is still a means to an end, and contrary to Carpenter (1996) I don’t believe there’s any serious risk that that a frequently-employed means will ever be mistaken for an end. Heck, field experiments are only a means to an end, but you don’t see anyone worrying that, if we focus too much on field experiments, we’ll start treating them as an end in themselves. (Come to think of it, maybe we should start worrying about that…) (just kidding)

As an aside, while the ultimate goal of microcosm work, including my own, often is to understand nature, worthwhile work in ecology can have other goals (e.g., Caswell 1988 talks about how theoretical ecology has ‘a life of its own’, independent of empirical data).

It’s irresponsible to train people to do microcosms; to solve real-world ecological problems we need field scientists with a deep appreciation of natural history and a ‘sense of the ecosystem’. Carpenter (1996) says this (and yes, he actually uses the word ‘irresponsible’). I’m glad he says it, because I suspect this gets to the heart of the matter. All the above objections, I suspect, really spring from the feeling that, if you’re not doing it in the field, and if your starting point is not natural history, you’re not doing proper ecology.

If you buy this objection, I doubt there’s anything I or anyone can say to change your mind. But a few remarks are in order.

First, if solving real-world ecological problems is your overriding goal, you should seriously consider going into law, politics, or economics rather than field ecology. The ultimate causes of anthropogenic impacts on the environment are not ecological. And while ecological knowledge is essential for addressing those impacts,  lack of knowledge typically is not among the biggest impediments to addressing those impacts.  But that’s a discussion well beyond the scope of this blog entry.

Second, I personally am a crap natural historian. So the fact that I work in microcosms is no loss to natural history-driven field ecology. 😉 (By the way, this is emphatically not true of everyone who works in microcosms. Peter Morin for instance is an amazing natural historian)

Third, no one I know trains students to be ‘microcosmologists’. Personally, I like to think that I train students to be scientists. For instance, one of my current graduate students entered my lab with a strong interest in working in microcosms, and I steered him towards alpine plant-pollinator interactions because that seemed like a better system in which to address the questions he wanted to address.

Fourth, I can tell you, from my own personal experience, that training students to work in microcosms does not significantly reduce the pool of students who want to train as field ecologists. I struggle to attract graduate students (and undergraduate honors students), and the ones I do attract mostly don’t end up working in microcosms. The vast majority of undergraduates interested in ecology get into ecology because they love the outdoors and wild nature, and they choose graduate supervisors and careers that reflect that love (which I think is great, by the way). Let me tell you, it’s very hard to convince such students to do ecology indoors. If we are indeed, as Carpenter (1996) claims, short on students with the natural history background to do field ecology, it’s not because students who have that background are getting sucked into doing microcosms instead. The students I mostly attract are the rare ones who, although they often love wild nature, also love big, general questions that are relevant to many systems. These students feel, as I do, that if a question could in principle be asked in any system, we may as well ask it in a model system that has features that make it easier to get a clear-cut answer.

Fifth, I respectfully disagree that, in order to solve the world’s ecological problems, and train the next generation of ecologists who will help solve those problems, that we should only train field ecologists. Natural history and a sense of the ecosystem are hugely important, but they cannot answer all the questions we ought to be asking, and indeed can’t even identify all the questions we ought to be asking. For instance, the ideas of alternate stable states, hysteresis, and critical slowing down, which have tremendous management relevance and on which Steve Carpenter himself is currently working, derive from dynamical systems theory and weren’t originally discovered by field ecologists. The competitive exclusion principle, which is implicitly or explicitly part of every field study of competition (e.g., between invasive species and natives) was not discovered by a field ecologist. The many surprising and counterintuitive behaviors of nonlinear, stochastic dynamical systems (e.g., chaos), which are a major plank in the argument for ‘adaptive management’, were not discovered by field ecologists (for instance, think of ‘stochastic resonance’, which may explain outbreaks of many pest insects). The fact that ‘diversity’ does not necessarily promote, and can even inhibit, ‘stability’, was not discovered by a field ecologist (indeed, field ecologists pretty much thought the opposite until Bob May came along). Modeling global climate change and its ecological consequences requires lots of mathematical and computer programming skills that could not realistically be part of the standard training for field ecologists. I could go on, but you get the idea. Good ecology, even good conservation ecology, and even good system-specific conservation ecology, often depends on ideas and skills drawn from other sources than natural history. A ‘sense of the ecosystem’ is necessary but not sufficient to conserve the ecosystem, because ecosystems are far too complex for that.

29 thoughts on “Objections to microcosms in ecology, and their answers

  1. Pingback: Interview with Steve Ellner « Oikos Blog

  2. Pingback: Why do fundamental research in a world with pressing applied problems? « Oikos Blog

  3. Pingback: From the archives: defending microcosm experiments « Oikos Blog

  4. I agree with you Jeremy, that microcosm experiments have their place in Ecology, just like many other approaches. As long as we accept their limitations, we should not ignore their utility either.

  5. Jeremy, your post forced me to re-examine my own mild skepticism about micro-organism laboratory experiments (although I want to emphasize that the skepticism is mild – I think that Rees Kassen, Graham Bell, Peter Morin, Owen Petchey, yourself, write papers that I am always eager to read). And so where does the skepticism come from? It turns out it comes from the same place that my general skepticism about ecology comes from – that ecologists often make claims to understand the world but are rarely able to demonstrate that understanding (and I put myself solidly in the camp of scientists that have provided no evidence that they have contributed, even incrementally, to understanding the natural world better). That’s a fairly sweeping claim (although not a new one – the late Rob Peters made this point explicitly in A Critique for Ecology) but, I believe, a true one.
    I make the claim based on two assertions, one logical and the other semi-empirical. The first assertion is that there is only one way to demonstrate understanding and that is through prediction. An ecologist, a natural historian, a farmer, a fisherman, a hunter can all make claims to understand how the natural world works but the only way they can demonstrate that they truly have understanding is to make predictions about the natural world that are more accurate than we would expect by chance. So, while understanding and prediction are not the same, understanding has one diagnostic and it is prediction (we could argue about whether prediction is sufficient to demonstrate understanding but it is absolutely necessary). The second assertion is that ecologists rarely make predictions (at least, relevant ones) and when they do, the predictions are usually poor. (This is going to get back to microcosm experiments – I swear.) That probably means I have to define what a relevant prediction is but I think you made the point in your post about microcosms – when push comes to shove we all have to acknowledge that our work usually only makes sense if we are trying to use it to understand what happens in natural systems. By extension, I would say that to demonstrate understanding about the natural world we have to make predictions about the natural world (not predictions about what happens in aquaria or beakers or field mesocosms or perhaps, even manipulated’natural’ systems).
    So, the skepticism I have about all experimental ecological research is proportional to the degree to which I think it is likely that the experimental units are representative of nature. More particularly, the degree to which those results will help me make predictions about the natural world. So, for micro-organism laboratory experiments that means there are two major sources of skepticism for me – one, if you are hoping to make general inferences to natural systems made up of microorganisms, are the ‘lab’ communities representative of natural communities? Now, I am less skeptical of microcosm lab experiments than I am many lab experiments because, as you point out, ‘lab’ communities contain large numbers of organisms and it is at least plausible that the spatial and temporal scales that these ‘lab’ communities are constrained to is not dramatically different that the ‘natural’ spatial and temporal scales that the processes of interest operate at in nature. The other source of skepticism is only relevant if researchers choose to make inferences beyond micro-organisms because now there is the additional concern that, even if the patterns observed in the lab also occur in nature, do they scale up? Are the relationships somewhat scale-invariant so that large differences in body size don’t result in qualitative differences in the processes being studied?
    But the proof of the pudding is in the eating, just as with any experiment (lab, field or ecosystem; big or small body) is anybody checking to see if the results from these experiments allow us to make predictions about natural systems that are more accurate than we would expect by chance? There can’t be any claim to better understanding of the natural world until that is demonstrated. And I would say understanding is not strictly a qualitative characteristic it can be quantitative – that understanding can be measured by the precision and accuracy of our predictions.
    So, I think you make a great case for the utility of microcosm experiments – if every experiment was held to the same standard ‘how likely are these experiments to contribute to improved predictions about the natural world’ I suspect that microcosm experiments would come out ahead of many other kinds of experiments. Having said that, microcosm experiments have the same failing that the vast majority of ecological research has – few tests to see if we understand the natural world better than we did before the experiments.
    I’ll finish with an example of how pervasive I think this problem is. Species-area relationships are as well-studied as any pattern in ecology. But could I use that century (or more) of know-how to predict how many fish species are in a randomly chosen lake in New Brunswick? What slope should I use? What intercept should I use? Are there any general principles that would allow me to select the appropriate slope and intercept and make a reasonably good prediction of the number of fish species in the lake (and by reasonably good, I only mean as good as could reasonably be expected given that we are using only a single variable to predict richness). I suspect we might be able to agree on the slope (0.25? 0.20? 0.30?) but what about the intercept? Would it depend on body size? Probably but I’m not sure how. Would it be different in aquatic than terrestrial systems? Not sure. What about homeotherms versus heterotherms? I don’t have a clear answer. My guess is that most ecologists would conclude that a century of research has led us to a place where we could say “a big lake will have more species than a small lake”. And we’re not even getting at mechanism here.

  6. Hi Jeff,

    I consider it the highest compliment to be told that I’ve prompted someone to re-examine their beliefs, so thank you very much for that.

    Your comment is a lengthy one that probably deserves at least an equally lengthy reply. By the way, nothing that I say below is intended as criticism of your very thoughtful comment.

    (i) Re: Peters’ work, it’s true that he emphasized the importance of prediction, but that came with a lot of other philosophical baggage with which I very strongly disagree. I’m not an instrumentalist (hardly any practicing scientists are). That’s just an aside. I take that, in mentioning Peters in passing, you didn’t mean to imply that your comment springs from an instrumentalist philosophy of science. (If you did, that would be a whole ‘nother conversation!). Certainly, plenty of non-instrumentalists have also attached great importance to prediction (e.g., the notion in philosophy of science of “inference to the best explanation”)

    (ii) Another passing remark about Peters, which is only tangentially relevant to your comment: the regression models of the sort Peters favored are infamous for not doing very well at predicting new data, especially when any sort of extrapolation is required. So if successful prediction is the one and only hallmark of successful science, Peters-style instrumentalist ecology isn’t very successful.

    (iii) Given the importance you attach to successful prediction, I take it you’re particularly impressed by those cases in which our theoretical models *do* successfully predict the outcome of microcosm experiments (e.g., Kaunzinger et al. 1998)? Or does that somehow not count because it’s “only” predictions about microcosms, not predictions about nature? This kind of gets back to the issue of the “realism” of microcosms. I kind of get the sense that you still feel like microcosms somehow aren’t “realistic”, but that you’re having trouble articulating this sense in a way that isn’t vulnerable to the counterarguments in my post.

    (iv) Are you worried about lack of successful predictions in ecology, or lack of successful *general* predictions? For instance, if you want to predict the species richness of a lake in New Brunswick at some point in the near future, you could perhaps do pretty well if you studied that particular lake for a while. But if you want to predict its richness just based on its area and the average richness-area relationship for all lakes in the world, yeah, that prediction probably won’t be as accurate. But I’m not sure why that should worry us. That just says that different ecological systems are different. Our failure to make precise-yet-generally-applicable predictions in ecology is surely a sign of *success* if such predictions are actually impossible! I mean, we’d be fooling ourselves if we thought otherwise, wouldn’t we?

    (v) Can you give a few examples of what you consider to be successful predictions in ecology? In particular, I’m curious about how precise you think successful predictions ought to be, or how you think the appropriate level of precision varies from case to case. From your example at the end, I get the impression that you think only very precise predictions (or at least quantitative predictions with precisely-specified confidence intervals), with very precisely-specified domains of applicability, will do.

    (vi) Re: prediction as the *only* way to demonstrate understanding, that is quite a strong claim. I’d only agree if “prediction” were interpreted in a much broader way than I suspect you intend (although I’m not sure–you don’t elaborate on what you mean by “prediction”). For instance, consider what’s known in philosophy of science as the “old evidence” problem. Here’s a standard example: the precession of Mercury was known before Einstein developed relativity theory. Should the fact that relativity theory implies the precession of Mercury count as a successful “prediction”? If so (and I, like the physicists of the time, do think it counts), that indicates that “prediction” doesn’t just mean “correct statements about data that have yet to be collected”. As an example that’s closer to home, in the Origin of Species Darwin famously makes no predictions about data yet to be collected (and certainly no quantitative predictions, or predictions to which confidence intervals could be attached). Rather, he shows how many already-known, apparently-unrelated features of the living world (including not just “nature” but also, e.g., domestic animals) are actually related: they can all be explained by the hypothesis of evolution by natural selection from a common ancestor, together with various ancillary facts and hypotheses (e.g., regarding geology). Did the Origin improve our understanding of nature?

    What I’d say is that good science can have various epistemological and cognitive goals, which are distinct albeit often intertwined. I agree that successful prediction (where “prediction” is interpreted suitably broadly) is one important goal, but it’s only one. I also agree that successful prediction is a powerful way to demonstrate understanding, but I don’t think it’s either necessary or sufficient for demonstrating understanding. For instance, it’s perfectly possible to make good predictions without any understanding of why they work. Think of the purely statistical predictions of the sort Peters loved. If I sample from a population of lakes and find such-and-such a regression relationship between primary productivity and zooplankton biomass, I can use that relationship to predict zooplankton biomass in a newly-sampled lake from that population, without having a clue about why the regression relationship holds. This is basically a slightly fancy version of what philosophers of science call the “straight rule” of induction–all you’re saying is “I predict the future will be like the past”. All the swans I’ve ever seen have been white, so I predict the next one I see will be white. Even though I have no idea why swans are white. As a second example of successful prediction without understanding, think of the use of an incredibly complicated, detailed, and realistic simulation model, like a GCM, to make accurate predictions. The model is so complicated that it’s a black box to us. It’s as complicated as nature itself (or at least close enough to make good predictions), and so we don’t understand how it works any better than we understand nature itself.

    Conversely, I think there are ways to improve understanding without making any successful predictions, or even any predictions at all. Think again of the Origin of Species. Or, as an ecological example, consider the discrete time logistic model. This model illustrates how time-lagged density dependence can destabilize population dynamics (many other models with time lags of course illustrate the same thing). Knowing that time-lagged density dependence is destabilizing improves my understanding of how population dynamics work, even if in nature there are no time lags, or there are time lags but their destabilizing effects are not observed for whatever reason (e.g., because they are overridden by stronger stabilizing forces). If you don’t know about the destabilizing effects of time lags, there is a gaping hole in your understanding of population dynamics. In other words, the discrete logistic, and other models with time lags, improve my understanding of how population dynamics work even if they don’t make any successful predictions.

    All of which is to say that I think microcosms can help us achieve various epistemological and cognitive goals without necessarily making successful predictions about nature. I get the impression that you and I might disagree, at least a bit, on the relative importance we attach to those various goals. I don’t think failure to make precise, quantitative predictions about data yet to be collected indicates that ecologists don’t understand nature, or that ecological science is weak science.

    (vii) Re: your question about whether microcosms are “representative” of nature, I’m not sure what else to say besides what I said in my post.

    (viiii) The view that “microcosm experiments are just as useful as field experiments–i.e. not very useful at all!” isn’t one I’ve encountered before. That’s a rather depressing point of view! It sounds like what you need isn’t so much a defense of microcosms as a defense of ecology! 😉 Or else a beer or something to cheer you up! 😉

    (ix) One of my grad students insists that successful prediction is the one and only legitimate goal of science, the only thing that matters. Responding to your comment has been good practice for arguing with him. 😉

    (x) Have you read Everything is Obvious (Once You Know the Answer) by Duncan Watts? A very good book on how rare truly successful prediction is, and on the many cognitive biases and other obstacles that prevent successful predictions. If you think ecologists are bad at prediction, well, they can join the club! Watts argues powerfully that, except in certain branches of the physical sciences and engineering, or in contexts in which the “straight rule of induction” works (“I predict the sun will rise tomorrow”), successful prediction is basically impossible. Not sure if that conclusion will cheer you up (“At least ecology’s no worse at prediction than most other sciences!”) or depress you further (“Good god, most other sciences are as terrible at prediction as ecology is!”).

  7. Pingback: Instrumentalism, Dali, My Dad’s Photographs, and Me | aquaticbiology

  8. Pingback: “I want to be the [famous non-scientist] of science” « Oikos Blog

  9. Pingback: My rejected Princeton Monographs proposal, or what makes a good scientific book? | Dynamic Ecology

  10. Pingback: If you’re not an activist you’re an inactivist | Matthew Oldach

  11. Pingback: Advice: on choosing your own path in science | Dynamic Ecology

  12. Pingback: Musings on the culture of ecology | Dynamic Ecology

  13. Pingback: Microcosm experiments have unlimited potential for community and ecosystem ecology (guest post) | Dynamic Ecology

  14. Pingback: When, if ever, is it ok for a paper to gloss over or ignore criticisms of the authors’ approach? | Dynamic Ecology

  15. Pingback: This isn’t the blog you’re looking for | Dynamic Ecology

  16. Pingback: On the value of simple limiting cases: Lotka-Volterra models and trolley problems | Dynamic Ecology

  17. Pingback: Modelling cyclic populations: thoughts on the workshop | Dynamic Ecology

  18. Pingback: The fox and the hedgehog | Dynamic Ecology

  19. Pingback: Why ecologists might want to read more philosophy of science | Dynamic Ecology

  20. Pingback: Ask us anything: how do you critique the published literature without looking like a jerk? | Dynamic Ecology

  21. Pingback: What’s the greatest ecology experiment in history? | Dynamic Ecology

  22. Pingback: Stats vs. scouts, polls vs. pundits, and ecology vs. natural history | Dynamic Ecology

  23. Pingback: Laboratory research will never be substitute for fieldwork | Prairies To Patch Reefs

  24. Pingback: How selective are you in choosing prospective graduate students? | Dynamic Ecology

  25. Pingback: Ecology is f*cked. Or awesome. Whichever. | Dynamic Ecology

  26. Pingback: The uncanny valley of theoretical models and microcosm experiments | Dynamic Ecology

  27. Pingback: Poll results: what are the biggest problems with the conduct of ecological research? | Dynamic Ecology

  28. Pingback: Friday links: #pruittdata rolls on, philosophy vs. Jeremy’s papers, Kate Winslett vs. Mary Anning, and more | Dynamic Ecology

  29. Just wanted to let you know that I really appreciate you writing this all out. I’m currently running a long-term experiment using microcosms and get constant flack that my study is not “truly representative of all lakes”. I know my study system has its limitations, but I’m trying to identify something that’s really difficult to do so without a controlled environment. Anyway, thanks again! I might just refer to this page next time this argument comes up.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.