A while back, we invited you to ask us anything. Here are our answers to the next two questions, from Kevin Chase. Questions are paraphrased for brevity, click the link to see the originals.
Would you invest real world money on your published scientific predictions?
Do you think if we wrote more manuscripts like a company prospectus (outlining management focus, failures and successes over the past year or couple of years and foreseeable risks) we would be able to get stakeholders and policy makers to feel more confident about investing money into the conservation programs we so highly recommend?
Jeremy: Re: investing in your predictions, I have an old post on “betting your beliefs” in science, and a related suggestion that we need more “short selling” of scientific ideas. See also this discussion of “adversarial collaboration“, which can be viewed as two scientists taking opposing sides of a scientific bet and then collaborating to do the study needed to determine the winner. And here’s my review of a book about one of the most famous scientific bets ever.
Sorry, no idea if writing mss like company prospectuses would attract funding for conservation programs.
Brian: I have weighed in before (in the strong affirmative) on prediction which is really just a variant of betting without the at risk part. The at risk part is to me interesting though. I think most scientists would say that they are betting on their predictions. They invest enormous amounts of time, invest money which is admittedly other peoples money but which is a finite limiting resource for the scientist, and their reputation and future fundability depend on the outcomes of their work. But on a meaningful level, I don’t think scientists (or maybe I should just stick to ecologists) are putting much at risk on their predictions. After all, you get a paper whether the experiment succeeds or fails. And there is always a fresh perspective that lets you see results in a new light. So I think probably the key missing piece is not actually risk – scientists do risk things. Its just that we risk with a safety net – its almost as if we cannot fail. How many times do scientists declare an experiment as a failure – “it failed, and I moved on”. To me the essence of risk to a scientist is the potential to be spectacularly right or wrong. Popper talked of risky predictions, and his student, Lakatos, talked of bold conjectures. Thus I think the real issue is not so much that we aren’t betting, its just that we’re betting on extremely low stakes questions. And until we get away from “x will have an effect on y” predictions we will not be making bold risky predictions. Like MacArthur’s work or not, he was exceptionally good at making bold predictions that often were wrong (failed).
I was in business for 10 years before returning to graduate school. I think you have an unduly rosy view of the accuracy found in business prospecti! Some grant proposals (e.g. NSERC in Canda) and most tenure packages are written as long term assessments of direction although like businesses we tend to make an overly positive description. I guess it was the word “failure” that stuck out at me as the biggest difference between business and science. Businesses definitely acknowledge failures and fully kill things (even if they try to make excuses and hide them). Scientists don’t seem to do that.
So it seems across my two answers to your two questions the common theme is scientists would be better off if we could see things as failures. This has some echoes in previous posts (and especially comments) about why science doesn’t ever move on (not to mention Jeremy’s zombie posts).
Jeremy adds: An interesting issue under the surface in Brian’s answer and mine is whether it’s important for the advance of science as a whole for individual scientists to admit error and move on, or if it’s sufficient for science as a whole to admit error and move on even if (some) individual scientists don’t. There’s a loose analogy here to evolution: is adaptation at the population level necessarily maximized when individuals exhibit adaptive phenotypic plasticity? Or are there situations in which adaptation is maximized by letting evolution by natural selection operate on populations of individuals with fixed (non-plastic) phenotypes? /end arm-wavy blather
Thanks so much for investing time to answer my questions fella’s. I really appreciated the links to older posts and outside links as I had not come across these posts yet. I really like the idea of short selling science ideas…. maybe we need a journal dedicated to that.
Re: Brian’s statement: “To me the essence of risk to a scientist is the potential to be spectacularly right or wrong.” – I will certainly be quoting that time and again I think. As a young professional scientist, I have been internally wrestling with designing experiments that are “playing it safe” (guaranteed to get published in at least a low tier journal) or an experiment that will either rock the “entomological ship” or get sunk myself.
Re: Brian’s statement: “…scientists would be better off if we could see things as failures.” What do you think are the best avenues to see things as failures? I think it is rather hard to see our own failures and much easier to point out (or think upon) others failures, which is not always productive from a human and professional stand-point.