Should theory published in general ecology journals have to be “realistic”?

At last year’s ESA meeting there was some discussion among members of the Theoretical Ecology section about how, with the exception of Am Nat, leading general ecology journals seem not to publish much theory. And further, that general ecology journals increasingly seem to demand that the theory they do publish be “realistic”. Meaning in practice that there needs to be data supporting the assumptions, estimating (or at least constraining) the model parameter values, and/or testing the model’s predictions.

Which seems problematic, at least to me.* I think leading general ecology journals should seek to publish the best work that ecologists do, including theoretical work. It’s not true that only “realistic” theory is of interest to empiricists, with “pure” theory belonging in theory journals. I don’t think it’s fair to expect theoreticians to test their own models, but not expect empiricists to develop their own models. And think of all the important theoretical papers that rightly have a had a big influence on all of ecology without being “realistic”. Bob May’s work on chaos. The Rosenzweig-MacArthur model. Charnov’s marginal value theorem of optimal foraging. Many others. Yes, I know that none of those were published in general ecology journals–but the idea that they couldn’t or shouldn’t have been because they weren’t “realistic” (in the sense of being tightly linked to data) bothers me.

I wonder if the issue here isn’t just an empiricism-vs.-theory thing. I wonder if some of it also reflects the increasing popularity of models over theory in ecology. I wonder if we’re so keen to link models and data, and getting so good at it, that we’re coming to see models not linked to data as either of little value, or as some separate thing that belongs in its own journals.

I’m curious about whether you share my admittedly-anecdotal impression here, and if so, whether you think it’s a bad thing. So here’s a little three-question poll, which I encourage both theoreticians and non-theoreticians to take:

*Although I admit that, when serving as a reviewer for general ecology journals, on at least one occasion I’ve asked authors of a theory paper to add in data demonstrating the real-world applicability of their approach.

30 thoughts on “Should theory published in general ecology journals have to be “realistic”?

  1. “And think of all the important theoretical papers that rightly have a had a big influence on all of ecology without being “realistic””

    All of the examples you cite, Jeremy, are decades old and predate the explosion of freely available ecological data being exploited by, for instance, the macroecologists. Perhaps that’s the difference now, which reviewers/editors are (subconsciously or not) basing their decisions?

    • It occurred to me to try going through back issues of Ecology, Oikos, JAE, and JEcol to see how much data-free theory they publish and if/how it’s changed over time. But that seemed like work.

      • Note that even these data wouldn’t necessarily answer the question asked in the post, because perceptions can become self-fulfilling prophecies. If theoreticians *think* that general ecology journals don’t want to publish pure theory, and so don’t submit theory papers to general ecology journals, general ecology journals end up not publishing any theory. Which then creates/reinforces the perception that they don’t want to.

        I’d be very curious to see data separating perception/self-fulfilling prophecy from reality here. But I don’t see any easy way to obtain such data…

      • @jeff:

        “Could make a good undergrad project”

        Actually, Sam Scheiner may have a bit of data on this from his study of whether ecology journals are publishing more hypothesis-testing research these days. Will have to go back and look.

      • Aha, found Sam’s piece, which I’ve linked to before:

        Not quite the sort of data I was looking for. Sam does break out a category of papers that ‘develop’ theory and finds that such papers are reasonably common in general ecology journals. But it’s not clear what fraction of those papers are data-free, or what fraction develop theory in the form of equations as opposed to verbal or graphical “theory”.

  2. Jeff’s thought about availability of data occurred to me also. But another important distinction that May makes is tactical vs strategic models. Strategic models are very simple and general. Tactical models incorporate more detail and are general specific to a system/place and should be tied to data. All the models you cite are strategic, and by their generality it might not make sense to parameterize with data (Lotka Volterra competitive exclusion is a general result) (although those models had a lot more traction after Gause did parameterize them for a specific case!). I think it is much harder of find strategic models these days. The frontiers are such that models are incorporating 3-4 factors at the same time. You could have an interesting debate about whether these are still strategic or if their very complexity makes them tactical. But if the latter, we should expect them to be parameterized.

    For me personally, I buy your claim that the theoretician shouldn’t necessarily have to be the one to do the tie to empirical data (although I also agree with Jeff that this is a lot easier to expect these days). But I think the issue a lot of people have is there are so many models being published in theoretical journals that are so clearly never even intended to be tied to reality. That to me is a math exercise, not an ecological model.

    • Interesting suggestion that there’s just not much “strategic” theory being published any more in ecology. I agree. It’s interesting to speculate about why that is. Have we exhaustively explored the space of strategic models in ecology, so the only thing left to study is more complicated tactical models?

      Good question as to whether a model that incorporates several factors necessarily is “tactical” and so should be expected to be parameterized. I’d say no. I’m thinking for instance of a lot of Peter Abrams’ stuff. A lot of his work was about introducing complications into very simple strategic models of competition and predator-prey dynamics. But the resulting models, while sufficiently complicated that they had to be analyzed numerically (you often couldn’t even solve for the equilibria algebraically), weren’t tied to any particular system, and definitely weren’t the sort of thing you’d ordinarily be able to parameterize with data.

      So, do general ecology journals still publish Peter Abrams-style stuff? And should they?

      • One place to look for strategic theory is at the nexus of ecology, behavior and physiology. Optimal foraging, borrowing heavily from economics, was so successful it was swallowed whole, disappeared, and reemerged as nutritional ecology and, to some extent, functional trait ecology. Ecological stoichiometry relies on a similar kind of logic (as did, I suppose, R-star) with models continuing to provide for a more nuanced view of multiple nutrient limitation. Finally, the metabolic theory of ecology has gotten some attention,😉 , and arises from a desire to see how much we can push cellular biology through to ecosystems. All three share a focus on integrative biology, predicting ecological patterns through an understanding of individual behavior and constraints.

      • Although as discussed in some old posts, the metabolic theory of ecology arguably is more of a data-linked model than a theory:

        Re: nutritional ecology and stoichiometry, I’d call that a broad area within which there’s both modeling tightly linked to data, and theory that’s not tightly linked to data. My work with Dave Vasseur on coevolution of species competing for nutritionally non-substitutable resources is an example of the latter sort of work.

        I don’t know how to define “trait-based ecology” well enough to say what sort of work it represents. It’s too broad a tent, encompassing too wide a range of totally different stuff, to allow succinct definition.🙂

  3. Well, I have anecdotal evidence of a reviewer for a general ecological/biological journal who just does not believe that theory yields valid conclusions. Rather theory is limited to hypothesis formation:

    > Throughout the MS the authors confuse modelling results with observations and controlled experiments. The results of models cannot serve as an evidence. At most they can be used as hypotheses that have to be tested against field data. All along the introduction and discussion the author fail to distinguish between model results and experimental results or observations.

    I accept the critique that model results and theoretical results should be labelled as such and distinguished from empirical results. But don’t the comments reveal a disbelief of the reviewer in theoretical work?

    > Refs. 1 and 2 deal with models of ecosystem dynamics and not with observations. Therefore, citing these articles in order to demonstrate [XY] is misleading.

    Comments like these come along with more specific ones discrediting the assumptions or the parameters of the model.

    This is the most disappointing review for a work that took two years of deep thinking, talking to empiricists, development and parameter testing. The reviewer just did not make an effort to understand the rationale of the study because … well … just because.
    Disappointing also because the (reputable) journal occasionally publishes purely theoretical or modeling work. Did we just have bad luck? I think, empiricists reviewing in general journals should have a broader picture and definition of the scientific method.

  4. Via Twitter, Ben Kerr sends us to philosopher of ecology Jay Odenbaugh, arguing that ecological theory need not make accurate predictions to be successful science:

    Strong echoes here of Bill Wimsatt’s “false models as means to truer theories”, and various other things we’ve linked to over the years:

  5. You have selected a wonderful topic, Jeff! I recall back in the 80s, when I was a grad student at the University of Alabama, we had a seminar focused specifically on theoretical ecology. One day, the prof leading the session posed a really interesting question: Who, and who does not get to publish papers in major ecological journals without data? While it’s been so long since we had the discussion, I cannot recall any specifics- so my apologies for not being able to provide more depth. But the take home message was that only the “heavy weights” were able to do that… and anyone else a few rungs down the ladder would never get that kind of paper accepted in a major journal. I am curious if you have noticed any kind of trend like this in recent times.

    From my own experience, having abandoned a highly applied profession as a research faculty guy in bio-medicine, and then transitioning to theoretical ecology just 6 years ago, I found myself very intimidated- to the point that I forced myself time & again to be absolutely certain data not just supported my models… but did so with the kind of fidelity that might be somewhat uncommon in ecology. I also insisted on “triangulating” my theoretical models, such that several independent analyses supported them. As a biochemist, working with the most controlled of environments in the lab, obviously those kinds of outcomes (e.g., p values less than 0.001) were commonplace. In ecology, when we work with natural systems wherein the lion’s share of variables are beyond our control, things like correlation coefficients and whatnot tend to have a log more wiggle.

    Having gone through this transition of professions, I have adopted a personal approach now, wherein I let the data be my guide. That is to say, I will develop an initial model, and then test it using data. Invariably, there are problems concerning fit… but then I will use those outcomes to tweek the model and get it to a point wherein it becomes a strong predictive tool. Due to the fact I am a recovering biochemist, I do not believe I would ever be comfortable in publishing a model absent any data.

  6. Hi Jeremy,

    Thanks for raising this interesting issue. Perhaps this begs the converse question: if theory papers are increasingly expected to test/fit/tie their mechanistic models directly to empirical data, then have we also seen the increasing expectation for empirical research to tie their data directly to theoretically motivated mechanistic models?

    FWIW, I completely share your perspective on this one. My sense of this trend is similarly only anecdotal but real data would be interesting.

    I don’t think that the greater availability of data provides any justification for this as a general trend. I believe a researcher needs a good understanding of the data they are working with, and that does not just happen because the data becomes easily available. Indeed, it almost too easy to ascribe some biological interpretation to a parameter and attempt to estimate or fit said parameter to some data, any data because DATA and REALISTIC and all.

    Certainly the ready availability of data should change the expectation for theorists in the particular context where a sub-field has a well defined data structure they consider and the theorist seeks to study some property about the underlying model(s) currently being used in that sub-field, and propose changes to those models on some theoretical basis. Patterns in macroecology are a good example, but there are many others — e.g. phylogenetic comparative methods have a reasonably well-defined data structure where one can revisit the existing models and explore the consequences of theoretical modifications to them using data that has already been studied with the simpler models. Perhaps this is becoming more common. I’m not sure such models would truly be considered ‘tactical’ models in the May sense though.

    The majority of ecological research doesn’t work with well-defined data structures and models though. In such cases, an appropriate empirical test may not be a reasonable expectation, while a superficial “tie” to “data” may only be substituting a more general result with one that is now contingent on both the model assumptions and the specific data; instead of being contingent on the assumptions alone. Surely we can still agree that good theoretical research can provide a far more general statement than a single model superficially “tied to some empirical data”; to take a recent Am Nat example,, the author might have attempted to insert specific parameters (say, the dispersal coefficients or connectivity matrix) from some existing data and made the same argument for the specific example; but that would tell us only less, not more than having the general result.

    I think it is easy to overstate the realism provided by data. If realism is the objective, effort would be better spent on showing the assumptions are reasonable (look — rock-paper-scissors competitive relationships really exist in these lizards) rather than on external estimates of parameters or statistical fits to quantitative data; which is no more ‘real’ than the assumptions of both the model and the statistics.

    • “have we also seen the increasing expectation for empirical research to tie their data directly to theoretically motivated mechanistic models?”

      Well, that Sam Scheiner piece shows that at some broad level, the proportion of papers with *some* sort of hypothesis or other theory link is increasing in leading general ecology journals. But Sam defined “hypothesis or other theory link” pretty broadly, including e.g., verbal theory, I think. I’m sure there are fewer papers with tight links between data and theoretically-motivated mechanistic models expressed in the form of equations.

      I think some of this also gets back to our older discussions of the increasingly-blurred line between theoretical and statistical models. One illustration (of many that could be given) is in phylogenetic work. People are fitting increasingly parameter-rich evolutionary models to phylogenies to try to answer questions like whether phenotypic change is concentrated around speciation events, whether net rates of diversification are diversity-dependent, whether “key innovations” lead to adaptive radiations, etc. Are those models statistical models, or theoretical models, or some intermediate? Hard to say.

      • ” I’m sure there are fewer papers with tight links between data and theoretically-motivated mechanistic models expressed in the form of equations.”

        I would guess the exact opposite. This has never been very common (its just that the papers that do this are good papers and stand out in our mind 20 years later). But my money (apparently loosely allied with Sam’s data) is that model testing is increasing.

      • @Brian:

        We actually agree. By “fewer” I meant “not all that many”, not “fewer than there used to be”. Sorry for the ambiguity.

      • The phylogenetics example is not a good one. I would not say that the “theoretical models” you refer to in that area constitute theory. They’re very nice stochastic models that provide a nice connection between data and biological interpretability, but I hardly see their application to data sets as any sort of theoretical exercise.

  7. Hi Jeremy, it seems to me that the goal of ecology (actually any scientific discipline ) is to better understand how the world works. If you buy that it, seems a natural corollary that theory should help us make progress towards the goal of understanding the world. That implies that theoreticians should be proposing models that are at least plausible descriptions of how the world works – descriptions that would be worth testing. That doesn’t mean that one couldn’t explore models that end up being implausible but if you begin from the premise that ‘nothing I am be about to describe would ever be worth testing’ that sounds like a waste of time. So, realistic in the sense that the model is potentially plausible and therefore potentially worth testing – yes. May’s work is a good example because he demonstrated that a simple deterministic system could still result in chaotic population dynamics. Well, if we can get chaotic dynamics out of such a simple system then it it certainly implies that chaotic population dynamics would be plausible in nature. It didn’t get us any closer to understanding how the world DOES work but for most ecologists it expanded our sense of how the world COULD work. And without understanding how the world could work we risk, like the drunk who looks for his keys under the street lamp rather than where he dropped them, not looking in the right places to discover how the world does work. So, I think there is value to theoretical work that isn’t tested with data…in the short term. In the long term, theory has to be tested. But, not necessarily by the developer of the theory. As you say, empirical ecologists test theory without developing it, it seems fair that theoreticians should be able to develop theory without testing it. But, this seems to be a strong argument for theoretical and empirical ecologists to collaborate. Best, Jeff H.

  8. I’m coming here as a PhD student in atmospheric science, without any knowledge of the current state of affairs in ecology. Maybe that’ll make this perspective different in some interesting way, or maybe just meaningless.

    Before making a general comment, I should clarify some definitions I have in mind: ‘data’ is some numerical output, classification, description, or collection of results- it is not synonymous with “observations.” That is, one can sensibly speak about “model data.” A ‘model’ is the theory (or collection of theories, or assumptions) written in the form of an equation or encoded on some computer. Part of the model may be empirically motivated, although may limit its utility in out-of-sample prediction.

    “Field data” is not necessarily the good standard. The goal is not to get models to match observations. It’s to get models and observations to match reality, and more generally to understand that reality (and the range of conditions over which your theory applies). The “observations” may just be one realization of many possible realities, either in being very sensitive to “initial conditions” (like weather) or in the statistics of its behavior if we change the problem. What if we were to start talking about ecosystem dynamics on some distant exoplanet? How does photosynthesis behave on a planet around an M-type dwarf stars? How could ecosystems have adapted to near or complete global glaciation 700 million years ago? These are interesting questions, and are scientific in nature, but the experimental sciences can only reach so far in tackling them. One of the reasons why we bother modeling is to test the assumptions made in inferring results from a specific observation.

    Observations themselves do not tell you how a system behaves. Of course, observations need to be interpreted by the observer and usually involve decisions, judgments, and our own biases/priors are inevitably interwoven in results. This isn’t an inherent problem in science- indeed, it is for this reason that reproducibility and testing for the robustness of a result to subjective choices is so valuable. Similarly, we can construct a hierarchy of model complexity- from back-of-envelope relationships to million line computer code, and this becomes our laboratory to explore parameter space and the robustness of a result to the inclusion or exclusion of processes (Isaac Held has drawn analogy to the biologist studying a hierarchy of organisms, such as bacteria or a rat, not always or usually for its own sake but to find results that carry over to more complex systems- except in modeling we construct the hierarchy of models ourselves, as opposed to nature constructing the whole “simple to complex” taxonomy that we can call upon to understand “life”).

    Should be develop “theories” then that don’t include “field data?” The obvious answer is yes. Of course, it’s always nice (and more elegant) when you can make contact between a theory and an observable metric, but one could also challenge empiricists to detect the signature of some hypothesized behavior in what is currently thought out-of-sample (e.g., in the exoplanet example, we have “potentially falsifiable” hypotheses).

    Finally, it should not be taken for granted that a theory that looks like it’s not applicable to reality (or contradicting “field data”) actually is. In my field, there’s a long history of people interpreting field-derived data wrong, or the data just not being very good, when the prevailing models were closer approximations to truth (and vindicated later!). Alternatively, there are mathematical exercises one can go through in simple models of climate that don’t seem to make any connection to reality, but then in the geologic record (or across other planets) actually end up doing so!

    • I am really glad we have perspectives such as yours from outside ecology. In one of Jeff’s previous blogs, I raised the issue of cross-disciplinary approaches. Jeff was very astute to point out that in today’s age, it is virtually impossible to do that anymore. I know, because I’ve gone through the process, and you sacrifice a lot to become fully educated in disparate fields.

      Your perspective as an atmospheric scientist I believe is very different from that of biology. I don’t think the lines are as distinct concerning theory v. application in biology. Also, your perspective on “data” in biology generally might be somewhat restrictive. Not all models are based on “numerical outputs”. Molecular models, for example, are largely based upon observations of protein-protein or protein-nucleic acid interactions, which sometimes, but not always, are quantified numerically.

      Your comment on “field data” not being a good standard is provocative. For me, anyway, the process of theory begins with observations made from field data. As biologists, we frequently encounter what are very interesting relationships in our data. But not all interesting patterns are worthy of pursuit. That is where I believe empiricism takes over, along with a heck of a lot of experience. Choosing the proper fork in the road is tricky enough as it is, so having some indication of something biologically relevant in your data is of immense help. Eventually, and hopefully, you close the circle by testing the theory with data.

      I realize it is difficult, if not impossible to test the kinds of models you have suggested. Obviously we have similar quandaries in biology- such as how life began. But I believe if data are available, or can be obtained, then they should be part of the process of developing theoretical models. Your point about data being insufficient to test hypotheses is a very good one. We’ve seen so many instances where that has occurred. Certainly had Mendel not selected the garden pea, but some other organism with less discernible traits of inheritance, he would have had no ability to develop the initial theoretical models of genetics. In my own work, I spend inordinate amounts of time developing the protocols to obtain “field observations,” i.e., data. And it is absolutely the case that had I not done so, my theoretical models would have appeared to not function.

  9. My response is twofold: first as a member of the editorial board of Ecology and second as a theoretician. The official point of view of the editorial board of Ecology is that the journal wants to publish all types of ecological research as long as it is of the right quality. And as far as I can see the editorial board members that are pure theoreticians (apart from me including for example Bruce Kendall) and hence deal with theoretical papers are doing a good job in judging theoretical papers on their merits (at least we try hard). But while evaluating a manuscript I (and I know for a fact also Bruce) clearly ask ourselves whether a paper offers new ecological insight or not. And that is sometimes a hard call. It all depends on the theory that is being developed. To give an example of what is not likely to make it into Ecology: if a manuscript introduces one or the other non-linear modification of a Lotka-Volterra predator-prey model and finds either that it can suppress cycling, promote chaotic dynamics or lead to alternative stable states, I would personally not find that a major advance in ecological theory and hence worthwhile publishing as such in Ecology. For the simple reason that we know that in general non-linear interactions can have these effects. At this point I might indeed ask the authors whether they can provide data that show that the non-linear modification that is being introduced in the model is a reasonable (not realistic!) representation of an interaction.

    A second point I want to make here, is that now already for the 2nd year in a row at the annual Editorial Board meeting of Ecology we (the board members) have asked ourselves, why we get so few submissions of good theoretical papers. We definitely want to publish them and definitely want to have a fair representation of this ecological discipline in our journal, but we simply get very few submissions (although if I look back over the papers I have handled and accepted for Ecology there are some dazzlingly complex theory papers among them that got to publication without problems).

    In my other role, as a scientist doing theory and definitely not modelling, I have had no problems whatsoever to publish my purely theoretical papers in either American Naturalist, Ecology or Oikos without any data whatsoever. In fact, the minority of my papers have appeared in theoretical journals. So, my experience is quite different, but that may depend (as I also express above) on the type of theory that is presented in a paper.

    Perhaps my comment simply shows that a general statement about theory versus modelling fails because there is not agreement on what is judged as “theory that offers new ecological insight”.

    • As a theoretician, I will say that I do feel that Ecology has a reputation of not being theory friendly but that we should perhaps be giving it more of a shot. As Andre points out, representation on the editorial board is very good in this regard. Somehow Am Nat just feels like safer territory though…

  10. This is a great discussion, enough for me to leave what I think is my first-ever comment on this blog, although I have been reading for a long time! I am someone very interested in ecological theory in the pure data-free sense. But I think often strategic (general) theory is in the realm of evolutionary ecology, or even just evolution (behavior or speciation theory, for example). Truthfully, I read Am Nat, Evolution, JEB and Proc B much more faithfully than Ecology because I have noticed they tend to publish more insightful theory (including my own, admittedly I am biased!). I also have not been to ESA in 10 years for related reasons. However, I am not someone who reads many abstract theory papers – I do data papers too. I think what I really appreciate is integrative work that is motivated by empirical problems that may not be answerable with a manipulative experiment. Strategic and tactical theories both have their place, depending on the question.

    • Very interesting suggestion that general evolution journals are happier to publish “pure” theory than general ecology journals. Why do you think that is? Does it reflect some difference in attitude towards theory between ecologists and evolutionary biologists?

      • I don’t entirely know why, but I think there is clearly a greater appreciation for theory in evolution – although maybe I just know more evolutionary theorists and am more likely to read/review their papers! I do think that many ecologists have a strong preference for manipulative experiments – quadrat science – and the journal Ecology loves that kind of science too. Whereas evolutionary biologists can’t do that, unless, say, they do experimental evolution. Not trying to criticize, but that has been my experience, and I have tried to reach both audiences at various times with very different results.

      • Probably because evolutionary biologists are more mathematically literate on average. There is also a deeper historical *connection* between theory and empirical work in evolution (especially population genetics), no?

  11. Pingback: Ecologists think general ecology journals only want “realistic” theory. And they think that’s bad. | Dynamic Ecology

  12. Pingback: Weekly links round-up: 27/3/15 | BES Quantitative Ecology Blog

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s