Being influential doesn’t compensate for being wrong

All scientists get things wrong. And not just matters of empirical fact. Hypotheses, methods, conceptual frameworks, and even entire subfields that initially seemed promising can turn out to be unfounded, nonstarters, or dead ends. Even very widely-believed empirical claims, well-studied hypotheses, popular methods, established conceptual frameworks, and popular subfields can turn out to be wrong.

When a very influential idea turns out to be wrong, it’s often argued that the idea’s influence compensates for its wrongness. “Ok, that idea turned out to be wrong–but look at all the good science it inspired!” I’ve heard something like this said about neutral theory in ecology, a commenter on this post said something like this about much of Stephen J. Gould’s work, and one could imagine saying something like this about lots of other influential-but-wrong ideas in ecology and evolution, like the intermediate disturbance hypothesis.

I don’t buy it. I don’t think the fact that an idea is influential compensates in any way for it being wrong.

Yes, influential-but-incorrect ideas often prompt lots of good science, which may well retain some value after those ideas are recognized as incorrect or otherwise flawed. But here’s the thing: had that incorrect idea never been proposed, the people who did that good science would have done different good science instead, under the influence of other ideas, some of them correct. Insofar as incorrect ideas are influential, they’re just robbing Peter to pay Paul, shifting the composition of our science but not affecting the absolute amount of it. Unless influential-but-incorrect ideas increase the absolute amount of good science of lasting value (e.g., by attracting into science money that otherwise would’ve been spent on non-science, and people who otherwise would’ve pursued non-scientific careers), I don’t see how their influence can be said to be a good thing. And frankly, I doubt that you can name an influential-but-incorrect idea in ecology that appreciably increased the absolute number of ecologists or the absolute amount of funding for ecology.

Now, often the only way to find out if an influential idea is correct (or useful or productive or whatever) is to pursue it. So it’s inevitable that some fraction of all of our science is going to be based on ideas that ultimately prove flawed. But just because that’s inevitable doesn’t mean it isn’t also unfortunate. Science based on flawed ideas may retain some value even after those ideas are exposed as flawed–but science based on sound ideas retains more value, all else being equal. Note as well that influential-but-flawed ideas also can prompt good science that doesn’t retain any value after those ideas are recognized as flawed. For instance, technically-sound attempts to replicate a flawed experiment often lack lasting value, at least lasting positive value. Nobody builds on them; their only value is the negative one of exposing the original experiment as flawed.

None of which means scientists shouldn’t care about being influential. I certainly want to be influential! But I don’t want to be influential for the sake of being influential, I want to be influential because I think I’m right. It’s not good for science if good, correct ideas have no influence. The point is that influence, independent of correctness, is valueless. What’s valuable in science isn’t being influential. It’s being right.

This is a lightly-edited version of a post that first ran a few years ago. Sorry for the rerun, I’m buying time to polish some good stuff.


30 thoughts on “Being influential doesn’t compensate for being wrong

  1. Nope, I disagree with you fundamentally. Hubbell’s Unified Neutral Theory is wrong; even Hubbell would concur on that point. What’s interesting is that, despite being wrong, it makes predictions consistent with real-world systems, and by employing fewer assumptions. By doing so it has challenged the way that ecologists interpret the assembly of communities and forced us to reassess the metrics we use. Something similar can be said of the Ideal Gas Law in physics — fundamentally incorrect in some of its assumptions, and only ‘true’ for a fictional ‘ideal gas’, but nonetheless delivering key insights.

    Any theory is only valid within the boundaries of its assumptions. It can be wrong in either its predictions, in which case it’s interesting to find out why the real world deviates from them, or in its assumptions, which forces us to reconsider our entire way of thinking about a problem. This to me is all not only good science, but essential science. An idea that’s logically wrong, however, is a waste of time. In that case, with something like the IDH, or the carbon-nutrient balance hypothesis, we can agree on the need to move on.

    • Re: Hubbell’s neutral theory, I think ecologists (including me) should’ve realized much earlier that the data his theory was trying to predict weren’t very difficult to predict correctly, because many different theories all lead to about the same predictions. That is, they should’ve realized much more quickly that *of course* neutral theories are among those that can predict, say, the shape of the species-abundance distribution. And so *of course* you can’t use the shape of the species-abundance distribution to try to distinguish between neutral and non-neutral theories.

      Now, I think you can still give credit to Hubbell’s theory for pointing out that neutral theories can predict some features of the world, even if you think (as I do) that it’s something that shouldn’t have needed pointing out. So I don’t disagree with you as much as you might think on Hubbell’s theory. I do think there’s an insight of lasting value that came out of Hubbell’s theory. I just don’t think we should’ve needed Hubbell’s theory to recognize that insight. And I don’t think we should’ve wasted as much time and collective attention as we did trying to use species-abundance data to test neutral vs. non-neutral models.

      As to *how* big an insight it is that a neutral model can reproduce (some features of) real data even though its assumptions are wrong, and even though it makes “simpler” assumptions than a non-neutral model, your mileage may vary:

      • I agree that we should have been able to foresee problem with SADs; Frank Preston knew that in the 1950s. Where I think neutral theory is interesting is in showing how much ecologists can throw out of their models, and identifying what else might need to be included. See for example this nice recent paper, which builds on the neutral framework to generate some — but not all — of the empirical patterns in diversity observed in BCI. I think it’s neat not because it’s a complete answer, but because it shows its workings and brings us a step closer. This is a new direction that was only made possible through Hubbell’s work:

        Many of the great theories in science come about through going back to basics; a process of paedomorphosis if you like. In this case it doesn’t matter whether Hubbell himself was right or wrong, more that his work generated unforeseen lines of enquiry.

    • That is a different discussion: when is a theory, framework, or hypothesis wrong or wright. Frameworks and hypotheses always make some assumptions that are not realistic. Theories go even further in boldness, as they are based on first principles, which are always very abstract and far from reality. All those ideas have boundary conditions and are not able to explain nature outside them. Nevertheless, an idea of any philosophical level proves to be not only valid, but wright, when predictions derived from it are confirmed in most or many cases. Therefore, the neutral theory may be considered right for its good predictive power. The intermediate disturbance hypothesis, on the other hand, is seldom confirmed through its predictions.

      • “Nevertheless, an idea of any philosophical level proves to be not only valid, but right, when predictions derived from it are confirmed in most or many cases. ”

        No, not when many different theories making different assumptions all predict the same thing. Well, in that case I guess you could say they’re all “right” in a sense. But I’d say that case is a case where it’s a bad idea to try to infer the correctness of a theory from the correctness of its predictions. See those two old posts I linked to in my reply to Markus.

      • You’re welcome, I love debates! Yes, I agree with you. Then we have another issue in question, which is the usefulness of an idea. Any idea may be (or not) valid, right, and useful. An all ideas need to prove their worth in all three aspects, necessarily in that order. Maybe in the case of the neutral theory what is under debate is its usefulness, not its validity or predictive power.

      • @Markus:

        “the difference at this moment in time is that I’m procrastinating from marking”

        Glad to be of “assistance”.🙂

    • There is something that has always confused me about this line of argument. The argument seems to be, “Hubbell’s Neutral Model is useful because it can predict SADs.” (I think? If that is more of a straw-man than a summary, please let me know) But, from what I understand, what it predicts is that SADs should follow a Fisher’s alpha distribution or a Preston lognormal distribution, right? And, we’ve known for half a century that most communities have that structure. So, it seems like Hubbell’s model doesn’t actually “predict” anything novel about SADs, it just proposed a mechanism that could produce a known pattern (and not uniquely, if memory serves there are other models, such as Maximum Entropy, that predict it as well). Thus, if we didn’t have the Hubbell model, it would still be possible to predict that most communities would follow a Fisher/Preston distribution (for some unknown phenomenological reason).

      Is there something I missed here? I know it makes other predictions, and I think that that is neat. And I think it’s neat that he showed how the Fisher and Preston distribution were basically the same. But, the best example I always hear are the SADs.

      • I don’t think you’re missing anything, at least not anything major.

        Brian could perhaps comment on why everyone fixated so much on SADs to the exclusion of trying to test other predictions. I suspect it was because lots of people had easy access to SAD data.

  2. Hi Jeremy; Kindly list a few theories in Ecology that you personally consider correct; maybe influential too. Then we can discuss what you require of a theory to be correct. and useful.

      • I wouldn’t call island biogeographical theory ‘correct’. It works well for a subset of systems, for a subset of taxa, and even then provides only a partial explanation for species richness patterns.

      • @Markus:

        “I wouldn’t call island biogeographical theory ‘correct’. It works well for a subset of systems, for a subset of taxa, and even then provides only a partial explanation for species richness patterns.”

        I agree, and these are issues the post glosses over. The domain of applicability of a model, and the importance of the factors or processes considered by the model relative to those the model omits.

      • Jumping off what Marcus said (not sure how to respond to his comment — sorry!), I’m interested in how you choose to value theories or concepts that only work, as Marcus said, for “a subset of systems, for a subset of taxa, and even then provides only a partial explanation…”
        In my opinion, these concepts can be powerful (the one that springs to my mind is the river continuum concept) though they work only for a specific subset of taxa and systems. I think that totally dismissing them as “not totally correct” would be an example of tossing the baby out with the bathwater.

  3. I could imagine that there is an added value from many scientists being focused on the same thing at the same time (E.g., does the fact that people jump on bandwagons imply that bandwagons are more productive per capita?). If so, the influence of an idea would actually have some value independent of its truthfulness.

    • Mike the Mad Biologist has made a version of this argument. Arguing that rather than spreading funding thinly among many investigators, all of whom then go and conduct their own low-powered studies on different topics, we need a few Manhattan Projects. We need to concentrate our grant funding on conducting a few massive, high-powered projects. That way we’ll at least make progress on those few topics, as opposed to spreading funding too thin and not making any progress on anything.

      I guess the key question is whether or when concentration of attention or other scarce resources is required for scientific progress. That’s an empirical question, and the answer surely varies from topic to topic. There are questions on which one investigator working on their own can conduct sufficiently high-powered studies to make substantial progress. That’s what I hope my lab does. And there are questions on which it takes many investigators all working on the same problem (and maybe all working *together* on the same problem), but those investigators don’t need massive financial resources. Think NutNet. And then there are questions on which progress requires a huge, expensive, coordinated effort by many people, like the Manhattan Project or the Human Genome Project.

  4. So, here’s another way to put the point I was trying to get at in the post. I do think scientific ideas can have other virtues besides being correct (or approximately correct, or correct in some circumstances, etc.). In particular, an idea might be “fruitful”. What bothers me is that, in practice, people often seem to identify “fruitfulness” with “influential”. I don’t like that, it amounts to saying that any idea lots of scientists choose to pursue is *by definition* a good use of scientific effort.

    But that raises the question of how you could tell how fruitful an idea is, or how worthwhile it is to pursue or build on, independent of how many people choose to pursue it or build on it. Which of course would require one to say something more precise about what “fruitful” is, and whether it’s even a property of an idea as opposed to a property of, say, the scientists working on an idea (this kind of gets back to my old post on how techniques aren’t powerful, scientists are: Brian and I had a conversation about this in the comments on that old post on successful vs. unsuccessful big ideas in ecology. To what extent is the success (or fruitfulness, or whatever) of an idea something that should be credited to the idea, as opposed to some other factor?

    I don’t have an answer. I know I don’t like identifying “fruitful” (or “successful”) with “influential”, and I don’t think “influence” in and of itself is usually a virtue. But I don’t know exactly what “fruitful” (or “successful”) is.

    • I really have to agree with this. If one is just happy with fruitful theories, and frutiful=influential=a lot of people spent a lot of time on it, then that is a circular self-justifying exercise that doesn’t tell us a lot about what good science is!

  5. Very thought and emotion provoking post, Jeremy.

    Do you think that it is fair to read this post as a lamentation for all the hours and careers lost on influential but incorrect theories and hypotheses?

    I understand what you are saying at this level: wouldn’t it be better if we magically only worked on correct ideas?! But I wonder if there is a prescription for what work scientists should do lurking here as well, something about avoiding bandwagons?

    I accept something like the following, on the basis of a Kuhnian or Lakatosian picture of science: the best way to know the conditions under which a theory is correct is for the theory to be well-developed. Some hypotheses don’t need much development, but many theoretical claims need a lot of development to either get the auxiliary assumptions correct or else exhaust a large space of auxiliary assumptions and show them incorrect. This seems best done in a research programme, and being influential means more people are in the research programme.

    Don’t continue to work on an influential theory once you know it is incorrect!
    This seems like a more reasonable, if still controversial, prescription.

    At this point we need to be more fine-grained about what ‘incorrect’ means because there are different ways for a theory to be incorrect and some of these are still useful, as Wimsatt showed.

    • “Do you think that it is fair to read this post as a lamentation for all the hours and careers lost on influential but incorrect theories and hypotheses?”

      You could read it that way if you like. Something that’s inevitable, and that doesn’t necessarily reflect badly on anyone, can still be lamented.

      “But I wonder if there is a prescription for what work scientists should do lurking here as well, something about avoiding bandwagons?”

      Maybe, but as you say it requires more fleshing out of what “incorrect” means, and distinguishing more and less useful forms of incorrectness. Taking my inspiration from Wimsatt, I’d say that ecologists as a group have wasted a fair bit of time trying to test ideas that aren’t usefully incorrect. So that we don’t learn anything from the tests, no matter what the outcome of the test. I have an old post on this in the context of the “hump-backed model” of diversity-productivity relationships:

  6. What about metabolic theory of ecology? It seems like there is a growing consensus that the mechanism for 3/4 scaling law proposed by MTE is wrong. However, I think the theory had a positive influence on ecology by getting people excited again about the importance of body size and energetics in ecology. Turns out that you don’t need to know the mechanism underlying 3/4 allometry to make useful predictions about populations, communities, and ecosystems.

    • Yes, assuming the consensus you refer to exists and is correct (I wouldn’t know), then the original WBE explanation for 3/4 power scaling of metabolic rate with body size seems like a good example of a productive, incorrect theory.

      As for the fact that you don’t need to know the underlying mechanisms in order to use allometries to make predictions, sure. A lot of our best predictive methods in ecology are just based on correlations. I’m thinking of Brian’s old posts on spatial autocorrelation here.

      Weird thought that just occurred to me: in science, we often talk about how productive incorrect ideas turned out to be. How come we never talk about how productive *correct* ideas turned out to be? Is it always (even necesssarily?) the case that a correct idea will be fruitful or productive? That it will attract a lot of attention, inspire a lot of good work that retains value indefinitely, etc.? I don’t think so. Which raises the question: what are the *least* productive/fruitful/etc. correct ideas in the history of science?

  7. Jeremy, One long rant, just for fun[ sorry it cant be in pub over beer]:You certainly don’t describe research as I know it; research is picking an important question that one can, or thinks one can, make some distance on…theoretically, experimentally, etc., and going after some TRUTH, maybe small, maybe big. But a lot of time will be spent in blind alleys, and more importantly will be spent not knowing if ones present efforts will pan out. If I knew what was true in area XXXX, why would I spend time/effort working there? of course I will often be lost; of course I will live with unrealistic assumptions; of course I will hope they are not critical. Of course I want to write influential pubs, and I assume others will pay attention if I help them do what they think is important.
    ESS theory, think sex ratio theory, is more successful than any other ecological theory, and in its practice relies upon a model of genetical control we all know is false …. we also know that our genetical models capture enough of the important/key features that our phenotypic predictions are quite likely correct[ they often are]; this quasi-independence from genetical details is very important. all the early ESS models of animal fighting relied on simplified, unrealistic models that developed concepts. we then tossed the model away, and kept the concepts.
    The art of doing great science always does this, and it drives philsophers of science bonkers. Just listen to a few interviews with the most creative physicist of the last 50 yrs, and his quest for truth :
    With reference to MTE, the original model of networks was so exciting not just because it got 0.75 exponent, but because it predicted many other exponents of the network , and they were correct to. And it changed forever the way we search for metabolic scaling explainations: think networks for distribution of materials. And think exponentials for putting in temperature, at least till the beast fries. Not all of this was original, but in the hands of folks like van savage, have led to real progress. Of course many broader issues did not depend upon knowing the correct network model, just that it was power function. science is to messy for a cook book.
    You describe science a much too tidy; its certainly is not. al least not for me.
    ric charnov

    • Hi Ric,

      You seem to have misunderstood the point of the post, which is my bad for not writing it better.

      I agree with you that sex ratio theory and metabolic theory are successful, productive ideas. I don’t think there’s a tidy, black and white division between correct and incorrect ideas, or successful and unsuccessful ideas, or whatever–it’s a gradient, or multiple partially-orthogonal gradients (scientific ideas can have various features that are partially or entirely independent of one another). I don’t think it is or should be obvious immediately when some new idea is worth pursuing or further developing. I don’t think progress is, should, or can be linear or tidy. And I don’t think all model assumptions have to be realistic or mechanistic or even vaguely true (on that last point, see:

      As I tried to clarify above (, what bothers me is the notion that any influential idea *necessarily* is a productive or fruitful idea. And the related notion that the influence of an idea, *in and of itself*, somehow redeems other failings.

      Of course, you could argue that I’m attacking a strawman, that basically nobody thinks that influence per se is an infallible sign of a productive, fruitful idea.

      And you could argue that I’m worrying about something that’s not worth worrying about, or isn’t helpful to worry about. You could argue that scientific life can only be lived forwards, and that the past history of science doesn’t hold many lessons for how to make scientific progress in future. Maybe scientific progress is so messy that every instance of it is sui generis. Maybe, as you say, scientific progress is an art, not a science, and further an art that can’t be learned by studying how past artists went about their business. It’s a creative process, so it’s inscrutable. I don’t think this is entirely true (for either art, or science), but it’s a debatable point.

      • One can learn a lot from past scientists and how they went about their business. which is why I provided a link to several hours of interviews with Murray Gell-Mann [ the interviewer is Geoff West, inventor of metabolic theory of ecology, which Murray discusses in the segment on the accomplishments of the Santa-Fe Institute][One could also listen to interviews with john Maynard Smith for something closer to home]. How one tests ideas is a much cleaner business than where ideas come from; Its the latter that I was writing about. you worry that ‘wrong ideas’ wont be weeded out, and will be influential far beyond their time. I worry that ideas will die in toddlerhood because they will be judged too unrealistic to pursue,….. because they will indeed appear in quite unformed, ill fitting, partly wrong, etc form. Thats the nature of science. Its very hard to teach students to live with this state of affairs in developing their ideas. I think you do indeed worry too much about Zombies [ and people getting credit for them anyway], and not enough about the care and feeding of embryos.
        A short historical story, about how hard it is to change people’s minds. In the 1970s real botanists hated the idea of plant sexual selection, sexual conflict, and sex allocation,… individual strategic thinking [ I could not get funded by NSF, for example]. they already had explanations about the events I , david Lloyd, the Charlesworth’s, mary willson,Kamil bawa [ and others] where trying to explain. But in the long run, say now 35-40 yrs later, current opinion is a mixture of our ESS resource allocation ideas and the older ‘avoidance of selfing’ ideas, the latter transformed into individual benefit rather than species level benefit. so some of their ideas survive, in revised form.
        I think it is as hard to get new correct ideas accepted as it is to get wrong old ones discarded. i once had dinner with GL Stebbins[ 1981] and tried to convince him of this new approach to plant breeding systems; I failed. Calling his ideas zombies , even conceptual Zombies [ they were group selection] did not help at all. what mattered in the long run?: simple, we explained lots of old & better yet,new stuff. Our logic was better too.

      • @ric charnov:

        “How one tests ideas is a much cleaner business than where ideas come from; Its the latter that I was writing about”

        And it’s the former I was writing about.

        “you worry that ‘wrong ideas’ wont be weeded out, and will be influential far beyond their time. I worry that ideas will die in toddlerhood because they will be judged too unrealistic to pursue,….. because they will indeed appear in quite unformed, ill fitting, partly wrong, etc form.”

        That’s true about our respective worries. But I don’t think they’re in any great tension with one another. I don’t criticize zombie ideas because I think they should’ve been strangled in toddlerhood! Zombie ideas are good ideas that were well worth further developing and pursuing when they were first proposed–and that *have* been further developed and explored (for decades!) and turned out to be incorrect, irredeemably flawed, or dead ends. If you look back at what I’ve written over the years, I don’t think you’ll ever find me arguing that a *new* idea should be killed off before it gets too far!

        (Aside: you will find me criticizing some “new” ideas as old wine in new bottles. Fair enough if you wanted to argue that in those cases I’ve failed to pay sufficient attention to the ways in which new work improves on past work, and too much attention to the ways in which new work repeats past mistakes.)

        If I understand you correctly, you’re also suggesting that there’s no point to criticism of ideas in any case because it won’t change the mind of anyone committed to the old incorrect ideas. That like it or not, the only way to get wrong old ideas discarded is to replace them with new correct ideas on the same topic. I agree: In response, I’d say that the primary audience for criticisms of ideas isn’t those who already hold those ideas–it’s students. Group selection might be an example here. G C Williams, Maynard Smith, et al. didn’t change Stebbins’ mind–but I bet their criticisms of group selection caused a lot of students to stop and think, and to do different and better work than they otherwise would’ve done.

        I’d also say that, when I do my own science, I already do as much as I can to try to push new correct ideas (at least, *I* think they’re new and correct!). Blogging doesn’t take away from that, and time I spend blogging is time I’d otherwise spend on something non-scientific. And if you reply, well, why don’t you also use your blog to push new correct ideas, the answer is twofold. First, sometimes I do (e.g.:, Criticizing zombie ideas is actually a small fraction of my total blogging output. Second, I find I have no control over what I happen to feel inspired to write about. I couldn’t choose to blog very differently than I do, even if I wanted to.

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s