“Putting your money where your mouth is”, also known as “betting your beliefs”, is a form of honest signalling: it’s a way of demonstrating that your beliefs are sincerely held, and that you’re willing to stand behind them. As economist Alex Tabarrok says, “a bet is a tax on bullshit”.
Of course, bullshit is relatively rarer in science than in other fields (Tabarrok’s discussion concerns political punditry and political election forecasting). But more broadly, a bet is a tax on being wrong. Insofar as your beliefs about the world are mistaken (or at least, more mistaken than the beliefs of whoever takes the other side of the bet), you expect to lose money. This is the logic behind prediction markets. Perhaps their most famous use is election forecasting, but they’ve been used to allow betting on scientific topics such as the reality of cold fusion.
Ecologist Paul Ehrlich and economist Julian Simon famously wagered on whether the inflation-adjusted prices of five metals would increase or decline during the 1980s. Ehrlich thought prices would increase because human population growth was outstripping growth in resource supply. Ehrlich lost. And while one could argue that in a larger or broader sense Ehrlich was right about the consequences of human population growth (indeed, many have argued precisely that), part of the point of a bet is that it forces the bettors to state their claims precisely enough to make a wager possible.Vagueness is where our biases, implicit assumptions, and faulty logic hide. In forcing you to be precise, betting is rather like mathematical modeling, and that’s a good thing. And if you find you’re unable, or unwilling, to make your beliefs precise enough to be willing to bet on them, well, that in itself is useful for you (and others) to know.
One famous scientific bet involved Alfred Russel Wallace, but it was nothing to do with evolution. Wallace bet a flat-earther named John Hampden that he could prove the Earth was round. Wallace, a qualified surveyor, won the bet with a famous surveying study now known as the Bedford Level experiment. The bet was for a lot of money (the equivalent of 35,000 pounds sterling today!), but Hampden never paid and instead took to threatening Wallace. More recently, Steven Hawking lost a bet on whether the Large Hadron Collider would discover the Higgs boson. Here is a list of prominent scientific bets, both historical and current, including many as-yet-unresolved ones. Many of them are from physics and none are from ecology, although a few involve bets or proposed bets on global warming. Physics bets are common enough that they’ve even been made the subject of a cartoon.
Far, far less notably, as a grad student I once made a bet with Hal Caswell and Mike Neubert over whether a particular food web model had a stable, feasible equilibrium or not. I said it did, they said it didn’t, I won, and they had to buy me a beer. In fairness, I don’t know that they were seriously betting their beliefs–I think it was more just a bit of fun and a way of testing me a bit (the condition of the bet was that I had to prove the stability and feasibility of the equilibrium myself; I couldn’t just go look up a paper on the model). Indeed, I now suspect that they may well have intentionally made a bet they knew or suspected they’d lose, much as a parent might let a child win at a board game. 🙂
I’m still trying to think of some ecological claims I’d want to bet on. Part of the problem is that my recent blog posts and paper on the zombie IDH have given away a number of “sucker bets” that I might have hoped to get someone to take (e.g., a bet on whether fluctuating mortality rates can make stable coexistence of competitors possible in a linear additive model). 🙂 Similarly, it’s too late to bet on the outcome of the NutNet study showing that the diversity-productivity relationship in grasslands is not humped. I kind of wish the NutNet folks had thought to make a bet with believers in the universality of humped diversity-productivity relationships before the NutNet study began. I think such a bet (or even just the exercise of trying to define a precise, mutually-acceptable bet) would’ve been very interesting. So I need to think of some as-yet-unresolved ecological controversies on which I’d be willing to bet on the resolution…
What ecological claims would you want to bet on, and at what odds? As Tabarrok notes, the odds are key–if you think the odds are fair, you should be willing to take either side of the bet. And if you can’t think of any, is that a bad sign for ecology? A sign that we’re bad at prediction, to the point where we’re too uncertain to be willing to make any bets?
Dynamic Ecology 
Great post and an awesome idea. Last night I read the first chapter Nassim Taleb’s new book “Antifragile” where he also moans and groans about economic forecasters who don’t (as he puts it) “have skin in the game”.
I can’t think of an ecological claim that I, personally, am willing to bet on, but I can think of a scenario where I wish the proponents of the idea would be willing to lay a wager. I would love to see evidence for the notion that protected areas selected using advanced reserve-selection techniques (systematic conservation planning) outperform those selected haphazardly.
I first assumed that it would be impossible to test due to the lack of a control group, but a quick google search suggests that others are already hard at work trying to answer this question (http://www.environmentalevidence.org/SR74.html).
If the odds offered are kind enough (say…8 to 1; any bookies out there?), then I would be willing to bet that carefully selected reserves perform no better than randomly selected ones because ‘where’ conservation takes place is less important than ‘how’ it is carried out.
Great comment! I was wondering if we’d get any commenters willing to offer bets–and the first one not only proposes a bet, but even specifies the odds!
Your bet is on a topic I don’t know much about, so I’m not willing to take you up on your wager. Though I do know of others who might have offered similar bets. I read Terbourgh’s Requium for Nature way back when, in which he argues that nature reserves in developing countries often are ineffective at conserving wildlife because of lack of enforcement. That is, ‘how’ conservation is carried out (more precisely, ‘how effectively’) is what matters.
On an irrelevant note, how do you like the Taleb book? I hear from one source that it’s rather a mess, and that it includes some loose analogies to ecology and evolution that don’t really stand up to scrutiny (the biological systems with which I’m familiar certainly aren’t “antifragile”). But I want to hear from other sources before deciding whether or not to read it.
Just to clarify: like all gamblers, my willingness to make a bet is based on a gut-feeling, not real information…
I have only read the first chapter of Taleb’s book so far, so I can’t really offer an opinion yet. I do get the impression, though, that it will be confusing if you haven’t read his earlier books because, even in the first 20 pages, he tends to refer to his previous ideas quite a bit without much context.
Hi Jeremy and all, I actually think that this points to one of the problems with ecology – even ecologists can’t think of a single prediction that they would be wiling to bet on. I’m guessing that the same wouldn’t be said for physicists or chemists (I’m sure there are predictions they wouldn’t be willing to bet on but I’m also certain they would come up with lots of predictions they would be willing to bet on). This is a very general statement and not true of all ecologists but isn’t it fair to say that ecologists relatively rarely test their models on new sets of data?
I would be willing to bet that any prediction of quantitative ecological model could not get within +/- 20% of the observed value for a data point that was independent of the data that the model was constructed from (if it was an empirical model). For example, I would bet against a species-area model for butterflies (or any more complex model that includes area and whatever other variables the predictor wanted to include) predicting the number of butterfly species within +/- 20% in an old field here in New Brunswick. I will add the corollary that ‘big old fields will have more species than small old fields’ is too vague a prediction to count. I understand that this is sort of arbitrary but it just sets a bit of a bar for ‘prediction’.
And I would rule out predictions that are related to ‘what results are possible under specific models and their assumptions’ because that only tells us if we understand our models not if we understand the natural world.
If I was forced to identify a systemic flaw in ecology is that we rarely ask how well our models (theoretical or empirical) predict independent data. I think that’s, in part, because our best models rarely predict very well for all kinds of good reasons (e.g. stochasticity, measurement error, historical contingencies, chaotic dynamics). But, that doesn’t change the fact that they only way to demonstrate understanding is by prediction. Our discipline might be better served by constantly having to confront how poorly we can predict.
I love the betting challenge and think it’s the perfect diagnostic for how confident scientists are in their discipline.Feynman makes the claim (and I haven’t heard it disputed) that quantum mechanics makes predictions that are analagous to predicting the width of North America to the diameter of a human hair. I don’t want to be held to that standard but the one I might be willing to bet against myself on is using Total phosphorus to make a prediction of chlorophyll levels in a lake. I think I might be able to make predictions that are within 20% of the real chl levels more times than not.
Jeff H
Hi Jeff,
Thanks for the lengthy comment. Brian will have much more to say in his upcoming posts on the issues you raise, so I don’t want to steal too much of his thunder. I will say that probably one sort of prediction we ought to be willing to make is based on statistical patterns in our data, of which we have many (species-area curves, relationships between lake TP and algal biomass, relationships between body size and many other individual and population properties, etc.). Predictions based on those sorts of data can of course be off by a lot, depending on a variety of factors including the difficulty of precisely defining the statistical population to which a given prediction applies. As you say, we don’t necessarily expect the species-area curve for butterflies in New Brunswick to precisely match that for, say, plants in China or whatever.
As for out-of-sample predictions being THE test of our understanding, I don’t know that I’d quite go that far, though they certainly can be a hugely powerful test. I’d also note that one can very well use statistical models for purposes of prediction–maybe even extremely precise and accurate prediction–without gaining any understanding at all. For instance, in his recent post, Brian noted the example of using the fact that species’ abundances often are spatially autocorrelated to make the prediction that “abundance here will be the same as abundance X km away”. That prediction might well be quite precise and accurate–but it doesn’t explain anything, doesn’t increase your understanding of the determinants of abundance, doesn’t tell you WHY abundances are spatially autocorrelated. Indeed, the late Robert Peters of course famously argued that ecologists should just be purely instrumentalist about their predictions. A point of view which I find troubling for various reasons (one of which is that there are many cases in which attempts to do purely phenomenological, statistically-based prediction fail very badly, as you’ve noted). But I need to stop here, as Brian won’t want me to steal his thunder! 🙂
Hi Jeremy, I’m looking forward to Brian’s comments but I will just pose this question – what other way can we demonstrate understanding of the natural world than prediction of the natural world?
It may not be sufficient but I think it’s necessary.
Jeff
Let me turn the question around: what predictions does Darwin’s theory of evolution by natural selection make? Darwin’s Origin is conventionally regarded as a remarkable *explanatory* success. Often also as a remarkable *unification* of many areas of inquiry previously seen as separate (and what “unification” amounts to and why it might be a good thing is a whole ‘nother conversation…). But *prediction*? Well, maybe–but if you regard the Origin as a major *predictive* success, then I think you need to at least elaborate on what precisely you mean by “prediction”. Since whatever successful “predictions” the Origin could be said to make are clearly of quite a different nature than, say, predicting the species richness of butterflies in New Brunswick…
Hi Jeremy, here are the sorts of predictions that I think arise from Darwin’s theories – so, if we are just talking about the theory of descent with modification we would predict that as we travel down through the fossil record organisms found in neighboring strata are going to be more similar than organisms from distant strata. We would predict that organisms that are found in geographic proximity to each other are going to be more genetically and phenotypically similar to each other than those found in similar environments but geographically far apart. There will be more phenotypic homologies in closely related than distantly related individuals. That there must be a process of inheritance that would allow for both change and conservation of traits. And that organisms with the highest mutation rates will have the highest rates of genetic/phenotypic change. That if we cross two very large dogs and two very small dogs that the large dogs will have larger offspring than the small dogs.
If we are talking about evolution by natural selection then we would predict that later generations will be more fit than earlier generations (i.e. if raised together the later generation will end up more abundant than the earlier generation). I would say that Richard Lemski’s work has provided a clear example of that. That if there is an orchid on an island where access to the nectar requires a very long proboscis we will find an animal on the island that has a very long proboscis. That if we find that such an animal once existed but no longer does, the orchid will decline in abundance or undergo relatively rapid evolution toward a physiology that allows easier access to the nectar. That, given enough time, bacteria will become resistant to all but novel antibiotics. That pests will become resistant to all but novel pesticides. That if we impose a fitness advantage for large organisms, an experimental population will evolve to be larger. If we impose a fitness advantage for small organisms…etc.
And although this might be heretical, I would also say that Darwin’s lack of understanding of the importance of other explanations for evolution such as genetic drift (and how could it be otherwise – he didn’t know genes existed) has probably led to a general perception among laypeople with an interest in science and evolution that evolution is all about natural selection. That we need to have a ‘fitness’ explanation for every phenotypic variant. I don’t know the history of Kimura’s work on genetic drift but I suspect it grew, in part, from a sense that evolution by natural selection couldn’t predict all the patterns that he saw around him. My sense is that it is difficult to test for the relative importance of genetic drift versus natural selection and, to the extent that we can’t make predictions to differentiate between the two theories, we also can’t make claims to understanding the relative importance of the two mechanisms.
I would say that the reason Darwin’s theories have stood reasonably well over time is because they made predictions that generally have been found to be true. Where they haven’t been tested, we can’t demonstrate our understanding of the natural world. It’s possible we have the understanding and can’t demonstrate it but, in science, not being able to provide evidence of understanding should be indistinguishable from not having it. That said, I will acknowledge that it is reasonable to say “This theory does a good job of predicting the things we have been able to measure and so we believe that it would also do a good job of predicting things we can’t measure yet.” but this is still an appeal to prediction. And it is much weaker evidence than direct prediction. And I’ll trot out the usual example of the danger of assuming prediction by association (provided by people like me who read neither widely nor deeply) – the accuracy and precision of some predictions based on Newtonian physics and the absolute inability of Newtonian physics to predict some phenomena.
So, I’m not sure how persuasive this has been but can I turn it around once more? If you are convinced that Darwin’s theories have increased our understanding of the natural world, what evidence would you provide for your conviction other than their ability to predict? Best, Jeff.
Thanks for taking the time to reply at such length Jeff.
In light of our previous discussions, I think several features of your list are interesting. One is that the predictions you list aren’t quantitative. I’ve gotten the sense that what you really want to see from ecology is quantitatively precise, accurate predictions. A really good numerical prediction of, say, the number of butterfly species in New Brunswick, not a qualitative prediction that, say, there will be more butterfly species in New Brunswick than in some smaller area. If you want ecology to be predictive, I think we can already make lots of predictions of the sort you attribute to Darwin. Or if you think we know a lot more today, and so should be able to make better predictions, would you also hold evolution today to the same standard? Because I don’t know that evolutionary biologists today are any better at making precise quantitative predictions than Darwin was. Rich Lenski for instance, whose remarkable work you note, and which I know quite well, has never to my knowledge quantitatively predicted new data in advance of collecting them. Now, Rich Lenski certainly has done a lot of hypothesis testing over the years. But as Brian noted in his recent post on the “insidious evils of ANOVA”, testing and rejecting hypotheses isn’t really prediction, or if it is it’s prediction in a quite different sense than predicting the number of butterfly species in New Brunswick.
I also think that it’s often unclear precisely what the assumptions are underpinning many of the predictions you offer. I say this not as a criticism–your list of predictions that a Darwinian might make is perfectly reasonable–but just to reinforce my point that, even when one wants to test predictions, one needs to be sensitive to their basis. For instance, you list Darwin’s prediction of a long-tongued moth to pollinate a long-spurred orchid as a prediction of his theory of evolution by natural selection. Are you sure? If I just say “organisms vary in their traits, those traits are heritable, and they affect an organism’s ability to survive and reproduce” (which is the conventional list of assumptions that together comprise the theory of evolution by natural selection sensu stricto), how do I derive from that the prediction of a long-tongued moth to pollinate that long-tongued orchid? Isn’t that prediction actually better thought of as being derived from other sources? For instance, couldn’t you make that prediction by just saying that “I observe that other similar species of orchid all are pollinated by long-tongued moths, so I predict that this long-spurred orchid will be as well”? Or take the prediction that large dogs will have larger offspring than small dogs? Is that really a prediction of “descent with modification”? Isn’t it actually a test of an assumption of evolution by natural selection (heritability)?
I’m sure I must seem like I’m being perverse or intentionally dense here–I don’t mean to be, sorry. As in other comments, I’m merely trying to drive home the importance of what I think is mostly a difference in emphasis between us. You see predictions as of overriding importance. I’m just gently trying to suggest that (i) testing predictions well often means paying a lot of attention to stuff besides the predictions that are ultimately of interest, and (ii) you’re implicitly defining “tests of predictions” in an extremely broad way, so as to lump together what seem to me to be a much broader range of things (tests of assumptions, tests of very different sorts of “predictions”, etc.)
As for how Darwin’s theory has increased our understanding of nature, besides its ability to predict, I’d say its ability to explain. Also its utility in directing and organizing our study–suggesting new questions to ask and new data to collect. Darwin himself famously called it “a theory by which to work”. Here I’m guessing you’d want to say it provided “lots of new predictions to test”, which I wouldn’t deny but which I don’t think does justice to the cognitive role the theory played.
I think we’re starting to go in circles and repeat ourselves here. Your comments are of course always welcome, I’m very glad that you find Dynamic Ecology such an interesting forum. But I’m afraid I can’t promise any more lengthy replies on the topic of “prediction”, not unless I think I have something really new to say that will really push the conversation forward.
Pingback: We need more “short selling” of scientific ideas | Dynamic Ecology
Pingback: Friday links: synthesizing data on “synthesis” in ecology, new blog on women in science, and more | Dynamic Ecology
Pingback: Friday links: how to do great research, what would Ben Bolker do, and more | Dynamic Ecology
Pingback: Friday links: scientific publishing is unfixable, “Big Replication”, new ecology teaching videos, and more | Dynamic Ecology
Pingback: Friday links: greatest syllabus ever, treemail, and more | Dynamic Ecology
Pingback: Ask us anything: investing in your scientific beliefs, and applied papers as corporate prospectuses | Dynamic Ecology
Pingback: Ask us anything: bet your beliefs | Dynamic Ecology
Pingback: Friday links: love letters to trees, are invasive species bad, ASN Young Investigator Award applications due soon, Barbara Kingsolver vs. Mary Treat, and more | Dynamic Ecology
Pingback: Friday links: de-extinction (of board games about extinction), and more | Dynamic Ecology
Pingback: Scientific bets vs. scientific influence | Dynamic Ecology